Monday, September 7, 2009

Cliff Click on Java-vs-C Performance

Cliff Click has a terrific post on Java-vs-native-code performance. Recommended.

Cliff is the Chief JVM Architect at Azul Systems, was the lead designer behind Hotspot's server JIT compiler, and is an all-around smart guy. His main drawback is that he (apparently) gets dragged into flamewars on YouTube's comment section.

7 comments:

Isaac Gouy said...

Yes he's "an all-around smart guy" but that doesn't mean he isn't "being sloppy in the numbers I report here"; it doesn't mean he isn't being sloppy in his comments about the benchmarks game.

Jeremy Manson said...

Without having dived into this too much, I do know that Cliff understands statistics pretty well. My guess is that examples he gave are more to prove a point than to demonstrate something conclusively.

My own opinion about the benchmark game is that it is (frankly) rather a waste of time. When I am deploying a server, I'm very interested in whether the code I have written can meet latency and throughput targets on the JVM in question, and not terribly interested in what SPECjbb2005 says. My experience is that the folks in charge of determining performance priorities for the JDK are more interested benchmarks than the average user's requirements.

Isaac Gouy said...

> My guess is that examples he gave are more to prove a point than to demonstrate something conclusively.

The trouble is he's trying to make several different points and his arguments conflict with each other.

His Java can be just as fast as C point conflicts with his too short to show something interesting point.

His different code ... so all the results have to be carefully inspected point is just as true for any Java vs C comparison that he makes.


> the benchmark game is that it is (frankly) rather a waste of time

Is it possible you are over-generalizing from your current level of experience and workaday tasks, rather than considering a broader range of inexperience and mistaken notions and ordinary ignorance?


> When I am deploying a server...

As in your application is the ultimate benchmark.

Jeremy Manson said...

Is it possible you are over-generalizing from your current level of experience and workaday tasks, rather than considering a broader range of inexperience and mistaken notions and ordinary ignorance?

Anything's possible.

The fact is that Java *can* be just as fast as C in practical applications (depending, of course, on the practical application in question). I've seen this in practice (much to the surprise of some of my C++-developing colleagues).

My opinion is that benchmarks are far too replied upon for JDK and JVM development. I know for a fact that there are optimizations in a variety of JVMs and JDKs that are targeted to improving benchmark numbers, from which few people will see an improvement in practice. Honestly, at this point, I think Sun's Java implementation's performance is good enough that in the short term, I would rather see JVM implementors focus on correctness and stability.

Isaac Gouy said...

> Anything's possible.

You are over-generalizing from your current level of experience and workaday tasks, rather than considering a broader range of inexperience and mistaken notions and ordinary ignorance.

The comments I see show people looking at the benchmarks game are surprised how fast Java is compared to C.

The comments I see show people have little or no basis for their notions about how relatively fast or slow programs written in different languages might be - it's outside their experience.


> The fact is that Java *can* be just as fast as C in practical applications...

I don't have a problem with that as a claim - the problem is that the claim isn't demonstrated, instead we have Cliff Click showing "a simple Sieve of Eratosthenes".


> optimizations in a variety of JVMs and JDKs that are targeted to improving benchmark numbers

There is of course a tradition of compiler writers doing that - JVM implementors no doubt focus on what they find interesting and amusing as much as they can.

Jeremy Manson said...

It doesn't sound as if we are disagreeing about anything, so I'll let your comments stand as written.

Anonymous said...

It is rather interesting for me to read the post. Thanks for it. I like such themes and everything that is connected to them. I would like to read more soon.