Dhrystone Benchmark on PowerPC 440 CPU

Performance Assessment using the Dhrystone v2.1 Benchmark on the Xilinx ML507 FPGA Platform
 

Table of Contents [Toc]

Overview
Test Setup
Hardware Architecture
   Clocking Infrastructure
   Glossary
The Dhrystone Benchmark
IBM PowerPC Performance Libraries
Test Results
   Measurement Results
   Inter-/Extrapolated Results
Cheated Results
   Measurement Results
   Inter-/Extrapolated Results
Summary & Conclusions
References
 

Overview   [Toc] [Top]

• Assessment and verification of the numerous Dhrystone performance numbers of the embedded PowerPC 440 CPU claimed by various press releases and data sheets, as those numbers are almost always listed without any compiler specifics [1, 2, 3].
• Impact analysis of the compiler's different code optimization techniques on the specifics of the Dhrystone benchmark.
• It is noted that the code size is intended to be kept within 32 kB in order to run entirely from cache, hence preventing any penalties due to cache miss and detrimental aspects of the hierarchical memory subsystem such as interconnect latencies.

Well, I'm aware of the paraphrase 'fake, lie, benchmark', therefore I seek to follow best practices by clearly specifying the overall setup employed including hardware system, software and compiler versions used, and employed compilation flags. Moreover, I do not intend to tweak any system specifics in order to reach highest scores, but rely on ordinary compiler flags such as '-O2' and '-O3'. The corresponding results are listed below, any interpretation is left to the prospective reader.

Note that in general, benchmark numbers are meaningless without proper specification of compiler settings and benchmarking conditions [4].

Last but not least, keep in mind Dilbert:

Test Setup   [Toc] [Top]

Hardware Architecture   [Toc] [Top]

The hardware architecture is depicted below. Only essential peripheral blocks have been attached to the CPU.

Hardware architecture for the performance assessment of the embedded IBM PowerPC 440 CPU using the Dhrystone benchmark. As peripheral blocks, 32 kB on-board memory, timer, interrupt controller and RS232 UART for serial communication are connected.

Clocking Infrastructure   [Toc] [Top]

The clocking scheme of the system architecture needs to adhere to a bunch of different rules imposed by various interconnect and device specifics. The applied clocking scheme and the corresponding important clock ratios are listed below.

Glossary   [Toc] [Top]

The Dhrystone Benchmark   [Toc] [Top]

The Dhrystone benchmark is a synthetic benchmark program and performs a series of CPU-centric operations such as integer arithmetic, comparisons, and logic and string operations. Being ported to the C programming language in 1988, the Dhrystone version 2.1 benchmark mostly relies on standard C library functions such as strcmp(), strcpy(), and memcpy(), and does not involve any multiply-accumulate or floating-point execution. The benchmark was mainly intended to characterize the integer performance of CPUs during the dawn of the Internet age in the 80's and 90's. In the end, the measured performance is reflected in a benchmark-specific metric called Dhrystones per second. This number can then be converted to Dhrystone MIPS (DMIPS).

What does 1 DMIPS stand for? The VAX-11/780 has been selected as the 'reference 1 MIPS machine', which scores 1757 Dhrystones per second in the Dhrystone benchmark, and hence, constitutes the reference for 1 DMIPS compute performance. As you can see from the measurement results below, the examined IBM PowerPC 440 CPU has an equivalent performance of 879 VAX-11/780. Did you get it? Well, even for me, the VAX-11/780 is an obscure ancient computing device from Digital Equipment Corporation (DEC), which was introduced in October 1977. It is kind of a computer dinosaur I haven't had the pleasure to become acquainted with...

The VAX-11/780 from Digital Equipment Corporation (DEC) introduced in 1977. It was running at 5 MHz and incorporated 32 bit addressing, 16 registers, 2 kB cache and 128 kB - 8 MB ECC RAM. This system achieves 1757 Dhrystones per second in the Dhrystone benchmark and represents the virtual metric for the benchmark by considering this system's performance as 1 Dhrystone MIPS (DMIPS). (Source: en.wikipedia.org / Digital Equipment Corporation)

IBM PowerPC Performance Libraries   [Toc] [Top]

Selected tests employ the compilation flag '-mppcperflib' which infers the IBM PowerPC Performance Libraries for optimized low-level integer and floating-point emulation, and optimized string handling routines [6]. According to Xilinx, the IBM PowerPC Performance Libraries may show an average of three times increase in speed on applications that heavily use these routines.

Caution: The IBM PowerPC Performance Libraries are only intended for improving the execution of emulated floating-point arithmetics and hence cannot be used in conjunction with floating-point hardware, i.e., with active '-mfpu' switch.

Test Results   [Toc] [Top]

Using separate compilation of files dhry_1.c and dhry_2.c, as intended by the Dhrystone benchmark.
Any interpretation is left to the prospective reader.

Measurement Results   [Toc] [Top]

IBM PowerPC 440 @ 400 MHz

The gray shaded table entries represent compiler settings violating the intended compilation rules of the synthetic benchmark.

Inter-/Extrapolated Results   [Toc] [Top]

Using 889 DMIPS @ 400 MHz as reference data:

Using 576 DMIPS @ 400 MHz as reference data:

Cheated Results   [Toc] [Top]

I was curious about the impact on Dhrystone performance if you merge files dhry_1.c and dhry_2.c into a single file, as you are not supposed to do [2]. The results are listed below. Again, any interpretation is left to the prospective reader.

Measurement Results   [Toc] [Top]

IBM PowerPC 440 @ 400 MHz

The gray shaded table entries represent compiler settings violating the intended compilation rules of the synthetic benchmark.

Inter-/Extrapolated Results   [Toc] [Top]

Using 1156 DMIPS @ 400 MHz as reference data:

Using 605 DMIPS @ 400 MHz as reference data:

Summary & Conclusions   [Toc] [Top]

By using gcc 4.1.1 and the IBM PowerPC Performance Libraries for an embedded PowerPC 440 CPU running at 400 MHz, it was possible to achieve up to 576 Dhrystone MIPS (DMIPS) with cache-adjusted code size and compiler optimization flags, which are officially allowed by the Dhrystone benchmark. Extrapolated to 550 MHz, the PowerPC 440 would achieve 793 DMIPS for the version 2.1 benchmark, what is well below the advertised single-core performance of 1100+ DMIPS from Xilinx [1]. In case aggressive compiler optimization techniques beyond the intention of the benchmark are used, the Dhrystone performance increases dramatically - ultimately owed to the fact, that the synthetic Dhrystone benchmark was both, not intended and not designed to cope with such compiler-based code optimizations [4].

However, the relevance of these performance numbers are in general questionable: The code must be running entirely from cache without any I/O transfers to show best results. As soon as larger code size, costly I/O transfers, and different compiler options are involved, these numbers are merely theoretical.

Last but not least, it is very impressive to see how different code optimization techniques of the compiler significantly influence the execution time of the identical piece of code. Here at last it becomes obvious, that a CPU performance measuring tool like a benchmark needs to be designed by keeping clearly in mind hardware, software and compiler architectures and capabilities.
 

References   [Toc] [Top]

[1] Xilinx Inc., Virtex-5 Family Brochure, Dec 2008

[2] Xilinc Inc., Virtex-5 Website

[3] Paul Glover, Running the Dhrystone 2.1 Benchmark on a Virtex-II Pro PowerPC Processor, Xilinx Application Note (XAPP507), July 2005

[4] Alan R Weiss, Dhrystone Benchmark - History, Analysis, "Scores" and Recommendations, White Paper, Nov. 1, 2002

[5] netlib.org, Benchmark Programs and Reports

[6] sourceforge.net, IBM PowerPC Performance Libraries

Last updated: 2012/12/30