tag:blogger.com,1999:blog-8405107760807432973.post4434950048433636102..comments2024-01-04T09:55:32.459-08:00Comments on Java Concurrency (&c): How Hotspot Decides to Clear SoftReferencesJeremy Mansonhttp://www.blogger.com/profile/04241094734813086257noreply@blogger.comBlogger22125tag:blogger.com,1999:blog-8405107760807432973.post-34131564300674757972013-12-30T00:59:54.075-08:002013-12-30T00:59:54.075-08:00Very useful information, but the parameter "h...Very useful information, but the parameter "how much free space there is in memory" is more ambiguous than one might suspect. This point has also been remarked upon by InJeNiErO: the "free space in memory" is calculated on the basis of what will be free after clearing all non-soft references. If that is a large amount (it may well run into the 100s of MB or even GBs) then the Arendhttps://www.blogger.com/profile/15624487515672426508noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-44953794226644840962011-01-10T12:44:27.251-08:002011-01-10T12:44:27.251-08:00@Luca: You can't run code when garbage collect...@Luca: You can't run code when garbage collection is actually happening. You can use finalizers to simulate some of this, but you have to do it carefully.Jeremy Mansonhttps://www.blogger.com/profile/04241094734813086257noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-74606833210044486342011-01-10T11:07:08.499-08:002011-01-10T11:07:08.499-08:00I'd like to catch when the GC passes and clear...I'd like to catch when the GC passes and clear soft references. Can I hook the GC? I tried to create a dummy SoftReference to intercept when it's enqueued (by extending the enqueue() method) but I got OutOfMemory and my enqueue() method is never called. Do you have a way to be called when the GC needs memory?Luca Garullihttps://www.blogger.com/profile/04523569178463279505noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-50610882466180493182010-11-26T10:45:11.095-08:002010-11-26T10:45:11.095-08:00Hi, I think it is worth to take a look at this rel...Hi, I think it is worth to take a look at this related bug to SoftReference and GarbageCollection: <a href="http://bugs.sun.com/bugdatabase/view_bug.do;jsessionid=cfd518f51afc7780e5188276b5f9?bug_id=6912889" rel="nofollow">SoftReferences cause worst-case garbage collection</a>InJeNiErOhttps://www.blogger.com/profile/02903880381509102311noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-57522473275881147282010-07-02T00:28:47.284-07:002010-07-02T00:28:47.284-07:00@Anurag - yes, thanks. Fixed.@Anurag - yes, thanks. Fixed.Jeremy Mansonhttps://www.blogger.com/profile/04241094734813086257noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-38343785388280343492010-07-02T00:23:31.542-07:002010-07-02T00:23:31.542-07:00I suppose instead of 5000 - 2000 <= 1 * 1000 co...I suppose instead of 5000 - 2000 <= 1 * 1000 correct expression would be 5000 - 3000 <= 1 * 1000Anurag Saxenanoreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-44109630518029770022009-12-14T11:20:07.613-08:002009-12-14T11:20:07.613-08:00@Ivan - I should have been more precise - I wasn&#...@Ivan - I should have been more precise - I wasn't really thinking about WeakReferences when I wrote that. It is true that WeakReferences can also refer to softly reachable objects when the VM decides to clear them. The WeakReferences will also get cleared if the VM decides to clear SoftReferences.<br /><br />This all happens atomically, though, so the atomicity property doesn't really Jeremy Mansonhttps://www.blogger.com/profile/04241094734813086257noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-52759894718790839872009-12-13T18:05:43.775-08:002009-12-13T18:05:43.775-08:00Hello, Jeremy.
When you say "The SoftReferenc...Hello, Jeremy.<br />When you say "The SoftReferences to an object are all cleared atomically. Therefore, no other thread will have a reference to the object when it is cleared."<br />You mean No-References-At-All or only No-References-and-SoftReferences?<br />Why i ask - because i read article = Jonathan Amsterdam, Adjunct Associate Professor of Computer Science at New York University Golovach Ivanhttps://www.blogger.com/profile/17934541017925930523noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-7856649752765658972009-10-08T19:49:50.536-07:002009-10-08T19:49:50.536-07:00For GC visualization, look at gcviewer:
www.tagtr...For GC visualization, look at gcviewer:<br /><br />www.tagtraum.com/gcviewer.html<br /><br />I used it a lot it 2006-2007. The zoomable diagrams it makes are very cool looking and worth studying.<br /><br />(When I used it in 2006-2007; it would complain about some log lines produced by newer GC modes, but was still useful.)Paul Baclacehttps://www.blogger.com/profile/01190461887856072497noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-65537964383238695812009-10-06T21:44:31.513-07:002009-10-06T21:44:31.513-07:00@maninder - Nope, sorry. Google has its own tool ...@maninder - Nope, sorry. Google has its own tool to do that, but we're unlikely to have the time / energy needed to make it open-source. Having said that, it isn't very complicated - it is just a python script to parse the output.Jeremy Mansonhttps://www.blogger.com/profile/04241094734813086257noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-57614892318380254922009-10-05T11:42:02.439-07:002009-10-05T11:42:02.439-07:00Apologies as this post does not directly relate to...Apologies as this post does not directly relate to the subject. I was wondering if there are any good open source tools for analyzing the output from gc details emitted –verbose GC -XX:+PrintGCDetails ?Unknownhttps://www.blogger.com/profile/10880114070254033807noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-17216494615421404452009-09-26T11:33:49.087-07:002009-09-26T11:33:49.087-07:00@Paul - it is only unsafe publication if multiple ...@Paul - it is only unsafe publication if multiple threads touch the SoftReference. In the post you mention, Bill was talking about making Reference and friends thread-safe. This was never done. There is nothing in the JavaDoc that guarantees its thread safety.<br /><br />Now, arguably, someone should take up the flag for this. I'll talk to some members of the core libraries cabal.Jeremy Mansonhttps://www.blogger.com/profile/04241094734813086257noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-43325847031723664702009-09-25T13:09:37.557-07:002009-09-25T13:09:37.557-07:00"referent should be volatile [...]"
Loo..."referent should be volatile [...]"<br /><br />Looking again at the Bill Pugh statement (from java memory model list, 2004):<br />``<br />The only cases that we really need to worry about are:<br /><br /> thread 1 calls clear, thread 2 calls get<br /><br /> GC clears reference, thread 2 calls get<br /><br />which are handled by making the referent field volatile.<br />'Paul Baclacehttps://www.blogger.com/profile/01190461887856072497noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-35432232557530662312009-09-23T23:32:06.094-07:002009-09-23T23:32:06.094-07:00@paul - GC itself doesn't create any happens-b...@paul - GC itself doesn't create any happens-before guarantees.<br /><br />When a SoftReference referent is cleared by GC, it is guaranteed that only SoftReferences refer to it (if you look around for the definition of "softly reachable", you will find this). The SoftReferences to an object are all cleared atomically. Therefore, no other thread will have a reference to the object Jeremy Mansonhttps://www.blogger.com/profile/04241094734813086257noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-26409071472398564762009-09-23T16:17:02.876-07:002009-09-23T16:17:02.876-07:00Here is a related question that I is resistant to ...Here is a related question that I is resistant to easy answers:<br /><br />GC interacts with Reference and ReferenceQueue. When a SoftReference referent is cleared by GC, is there a guarantee that no thread can see the old reference, except for PhantomReference? <br /><br />Does GC have any happens-before guarantees? <br /><br />The memory model docs and Java Concurrency in Practice (Brian Paul Baclacehttps://www.blogger.com/profile/01190461887856072497noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-58680982133385922862009-07-14T14:40:05.546-07:002009-07-14T14:40:05.546-07:00@mark - Any GC. Sorry about that!@mark - Any GC. Sorry about that!Jeremy Mansonhttps://www.blogger.com/profile/04241094734813086257noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-58716124699092505482009-07-14T14:28:35.905-07:002009-07-14T14:28:35.905-07:00Just to clarify the issue regarding SoftReferences...Just to clarify the issue regarding SoftReferences always being kept for at least one GC after their last access, rather than one full GC. Does this mean that the global <i>clock</i> variable is set on every GC rather than just on a full GC?Marknoreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-33614406489033623352009-07-09T02:35:17.254-07:002009-07-09T02:35:17.254-07:00This comment has been removed by the author.Brian Whitehttps://www.blogger.com/profile/05675893690256287871noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-71892839073219852502009-07-08T19:56:53.434-07:002009-07-08T19:56:53.434-07:00You actually can measure memory pressure directly ...You actually can measure memory pressure directly with the <br /><br />http://java.sun.com/javase/6/docs/api/java/lang/management/MemoryMXBean.html<br /><br />Using that you can watch the heap sizes post-GC and take actions based on the data you collect.Samhttps://www.blogger.com/profile/14121705654622595662noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-23884531209280884042009-07-08T09:13:57.020-07:002009-07-08T09:13:57.020-07:00@mindas - This is precisely the problem that my co...@mindas - This is precisely the problem that my colleagues encountered. They really want an LRU cache for what they are trying to do, and they tried to achieve it cheaply using SoftReferences.<br /><br />Sadly, there is no easy way to do what they do - clear memory in response to memory pressure. You could, for example, do it by checking the available memory every so often in a thread, and Jeremy Mansonhttps://www.blogger.com/profile/04241094734813086257noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-51425531640661986902009-07-08T02:03:24.188-07:002009-07-08T02:03:24.188-07:00Very good post!
IMHO using Softreferences in an ap...Very good post!<br />IMHO using Softreferences in an application server is evil, because of this very coarse grained way to configure their behaviour. <br />Note also that the mechanism implies that Softreferences will (in low memory situations) only be reclaimed after more than one full GC.Anonymoushttps://www.blogger.com/profile/06348916709650836506noreply@blogger.comtag:blogger.com,1999:blog-8405107760807432973.post-5555621645155443052009-07-08T02:01:11.069-07:002009-07-08T02:01:11.069-07:00You said "SoftReferences are generally used t...You said "SoftReferences are generally used to implement memory-sensitive caches". I was actually thinking about this myself - why caches don't get implemented using Weak/Soft references, then a good chunk of algorithm becomes so easy: you don't have to worry about item removal from the cache (map generally).<br /><br />So I asked this question to Gil Tene (Azul Systems) at mindashttps://www.blogger.com/profile/08826637355577644350noreply@blogger.com