Introduction

During efforts to migrate software elements from C and C++ to Java the issue of performance is often raised as a risk area. This intangible perception of performance risk can be addressed directly. In many cases people are operating from outdated information and perceptions about the current state of Java technologies.

In this short paper, a small survey of existing performance data is summarized, followed by the some specific informal tests. We certainly don't need another paper on this topic, so this paper tries to focus on interpreting available results instead of running a new study. The rationale is that we can average over many independent analyses and get a semi-statistically relevant result.

Survey

The following table summarizes many results into a single composite number. From each of the referenced studies, data were selected to represent a single platform (Linux where possible), and using the later Java and Gnu C++ compiler implementations. Also, when possible, the implementations in Java and C++ are similar in their object-oriented approach.

From each of these data points, a final average is taken, and we conclude that on the average, Java runs approximate 92% the speed of C++. Also of note is the speed of Java has been improving steadily with each release, so we can expect this gap to continue to close.

Performance Test	Java/C++ Performance factor
[lewis]	1.54
[saclay]	0.56
[bagley]	1.06
[wentworth]	0.51
Averaged result	0.92

The spreadsheet with the supporting data for this table are also available in these forms: HTML and OpenOffice.

Will your software exhibit these performance characteristics when ported from C++ to Java? There are many factors that can affect the specific performance you will see given a JVM, a machine, and an algorithm. For example, one dramatic effect is caused by whether the entire algorithm can fit within the processor's cache. If it can fit, it will run significantly faster, but generally real algorithms with significant data to process will not fit. So some very small tests may show that C++ will out perform Java dramatically, but only while the C++ implementation has little enough data to reside within the cache. Once it goes beyond those bounds, its performance is closer to Java again.

Informal Tests

The Java and ANSI C versions of the SciMark2 scientific benchmark [scimark2] produced these composite scores for runs on a DELL 2Ghz Pentium 4 laptop.

For large mode runs the C++/Java performance factor was: 0.74
For small mode runs the C++/Java performance factor was: 0.46

The execution output follows.

For the Java version, run in large mode (bigger than the cache) with JDK 1.4.2-b28:

java -classpath classes jnt.scimark2.commandline -large

SciMark 2.0a

Composite Score: 72.55501570968538
FFT (1048576): 12.60812371743371
SOR (1000x1000):   147.6241873945024
Monte Carlo : 20.731808537922987
Sparse matmult (N=100000, nz=1000000): 63.58668617601
LU (1000x1000): 118.22427272255783

java.vendor: Sun Microsystems Inc.
java.version: 1.4.2
os.arch: i386
os.name: Linux
os.version: 2.4.18-17.8.0

For the Java version, run in large mode with the Jikes compiler:

jikes -O -bootclasspath /h/l/j2sdk_nb/j2sdk1.4.2/jre/lib/rt.jar -d classes jnt/scimark2/*.java
java -classpath classes jnt.scimark2.commandline -large

SciMark 2.0a

Composite Score: 76.74625074191817
FFT (1048576): 16.553894841097797
SOR (1000x1000):   156.421378180101
Monte Carlo : 20.873675207341186
Sparse matmult (N=100000, nz=1000000): 65.57376975495468
LU (1000x1000): 124.30853572609624

java.vendor: Sun Microsystems Inc.
java.version: 1.4.2
os.arch: i386
os.name: Linux
os.version: 2.4.18-17.8.0

For the ANSI C version, run in large mode (bigger than the cache):

scimark2 -large

**                                                              **
** SciMark2 Numeric Benchmark, see http://math.nist.gov/scimark **
** for details. (Results can be submitted to pozo@nist.gov)     **
**                                                              **
Using       2.00 seconds min time per kenel.
Composite Score:           98.36
FFT             Mflops:    13.47    (N=1048576)
SOR             Mflops:   165.90    (1000 x 1000)
MonteCarlo:     Mflops:    31.36
Sparse matmult  Mflops:   152.38    (N=100000, nz=1000000)
LU              Mflops:   128.70    (M=1000, N=1000)

For the Java version, run in regular mode with JDK 1.4.2-b28:

java -classpath classes jnt.scimark2.commandline

SciMark 2.0a

Composite Score: 88.6567328037233
FFT (1024): 42.517564151904985
SOR (100x100):   153.1760746968527
Monte Carlo : 20.680696898630377
Sparse matmult (N=1000, nz=5000): 60.32400411175475
LU (100x100): 166.58532415947366

java.vendor: Sun Microsystems Inc.
java.version: 1.4.2
os.arch: i386
os.name: Linux
os.version: 2.4.18-17.8.0

For the ANSI C version, run in regular mode:

scimark2

**                                                              **
** SciMark2 Numeric Benchmark, see http://math.nist.gov/scimark **
** for details. (Results can be submitted to pozo@nist.gov)     **
**                                                              **
Using       2.00 seconds min time per kenel.
Composite Score:          192.66
FFT             Mflops:   132.14    (N=1024)
SOR             Mflops:   171.44    (100 x 100)
MonteCarlo:     Mflops:    31.51
Sparse matmult  Mflops:   295.21    (N=1000, nz=5000)
LU              Mflops:   333.01    (M=100, N=100)

References

[lewis] Lewis, J.P. "Performance of Java versus C++" [ link, local ]

[saclay] Lefevre, JeanPaul "Java versus C++" [ link, local ]

[bagley] Bagley, Doug "The Great Computer Language Shootout" [ link ]

[wentworth] Langan, David D. "Performance Evaluation: Java vs C++" [ link, local HTML local Zipped MSWORD ]

Introduction

Survey

Informal Tests

References

Related links