ETA: The G1 collector is now in the latest JDK6 release, which you can download from Sun here. You can enable it for testing with the flags -XX:+UnlockExperimentalVMOptions -XX:+UseG1GC. Original post follows.
For those of you who are interested in tinkering with new garbage collectors, the "Garbage-first" G1 garbage collector is in the new JDK7 drop from OpenJDK. I haven't tried it yet, but I am looking forward to seeing what it does.
(You will notice the link doesn't say anything about the new GC. Here's the appropriate bug.)
G1 is supposed to provide a dramatic improvement on existing GCs. There was a rather good talk about it at this year's JavaOne. It allows the user to provide pause time goals, both in terms of actual seconds and in terms of percentage of runtime.
The principle is simple: the collector splits the heap up into fixed-size regions and tracks the live data in those regions. It keeps a set of pointers — the "remembered set" — into and out of the region. When a GC is deemed necessary, it collects the regions with less live data first (hence, "garbage first"). Often, this can mean collecting an entire region in one step: if the number of pointers into a region is zero, then it doesn't need to do a mark or sweep of that region.
For each region, it tracks various metrics that describe how long it will take to collect them. You can give it a soft real-time constraint about pause times, and it then tries to collect as much garbage as it can in that constrained time. This is rather nifty (although it does add yet another layer of complexity to managing your GC).
Although this is likely not ready for prime time just yet, for the final version of JDK7, Sun is looking at making this the new "high performance" collector, replacing parallel newgen + CMS. Here is much more information about the collector.