Benchmarking

Gelato@UNSW Work

Gelato@UNSW is undertaking a project on Ia64 LinuxScalability.

As part of this project we will be aiding the Scalability group in several areas of benchmarking, such as:

General Benchmarks

The following is a list of Benchmarks that have been identified as being useful for general benchmarking.

Tips

Specific Benchmarks

Kernel

Scalability Benchmarks

I/O Benchmarks

NFS

Database

Other

Quick Setup Guide

Run times are calculated on a dual 1500Mhz Itianium2 rx2600 running a 2.6.8-rc2 SMP Linux kernel.

Benchmark

Setup

Approx. Setup Time

Run Command

Approximate Runtime

Extract Results

Requirements

NFSSTONE

unpack, compile, run

1m

./nfsstone <path to nfs> <path to lock file>

0m 3.645869s

printed to stdout

Network file system access

Stream

unpack, make, run

1m

This script will run 10 iterations of all the gcc and icc generated binaries and pipe the output to file and stdout

1m 21s

printed to stdout

Intel's C compiler would be nice.

reaim7

unpack, bootstrap, configure, make, run

2m

reaim -c src/reaim.config -f src/workfile.alltests -g -i 10 -r 10

180m10.151s

file

Can be run from user space

LTP

unpack, make, run

export TOUTPUT=CONDENSE; sh runalltests.sh -o uname -r-ltp-01-output -l uname -r-ltp-01-log

55m 12.441s

file and stdout

root access to run all test cases, if root access is not available run times will be shorter.

LMBench2 Scale

unpack, make

make results; make rerun; make rerun; make rerun; make rerun

8m51.978s (single rerun)

cd results && make summary percent LIST=ia64-linux-gnu-first-run/*

none

lmbench3

unpack, make

make results; make rerun; make rerun; make rerun; make rerun

222m37.013s (single rerun)

cd results && make summary percent LIST=ia64-linux-gnu-first-run/*

none

SPECweb99

Start http server on system under test, . ./shrc, ./client -d, specperl manager

105m (defualt run)

http server with cgi access, user access to a cgi_bin

Not completed

Benchmark

Setup

Run Command

Approximate Runtime

Extract Results

Requirements

OSDL-DBT-1

see WiKi page

postgresql, postgresql development libraries and headers, user and group pgsql with password pgsql

OSDL DB benchmark

see WiKi page

postgresql, postgresql development libraries and headers, user and group pgsql with password pgsql

Base requirements

Applications

Users Account

Groups Account

Directories

Access

bash

none

awk

none

sed

none

postgresql

postgres, pgsql

pgsql

/tmp

wrx

Tuning and Configuration

There are several consideration required when tuning and configuring a system for benchmarking:

The TuningBenchmarks page has more information on these issues.

Statistics

Reporting your data correctly is very important

IA64wiki: Benchmarks (last edited 2009-12-10 03:14:06 by localhost)

Gelato@UNSW is sponsored by
the University of New South Wales National ICT Australia The Gelato Federation Hewlett-Packard Company Australian Research Council
Please contact us with any questions or comments.