2. Benchmark examples

Benchmark examples

Simple PC benchmarks:

• Windows / Mac OS: NovaBench
• Linux: sysbench

Notable synthetic benchmarks

• Prime identification is one example of a synthetic benchmark.
• Famous historical examples include:
  • Whetstone: Trigonometric and exponential functions
  • Linpack: Linear equations and double precision arithmetic
  • Dhrystone: String manipulations and integer operations

Simple synthetic benchmarks are easy to optimise so they do not give an honest measure of a system’s performance at general computing.

Benchmark suites

Benchmark suites are preferred to single synthetic benchmarks as:

• They are harder to optimise for
• They are potentially more representative of general computing patterns
• Benchmarks can be easily swapped in and out

Standard Performance Evaluation Corporation (SPEC) has one of the most widely used benchmark suites. It contains several programs for a wide range of integer and floating-point operations, arranged into two sets.

• CINT2017 is a set of programs for integer operations
• CFP2017 is a set of programs for floating point operations

This suite is regularly updated to keep up with general modern computing. https://spec.org/cpu2017/Docs/overview.html#benchmarks

The final result is the geometric mean of the run times for each program in the suite. Times from each program are different scales, so the geometric mean is more appropriate.