[ http://issues.apache.org/jira/browse/LUCENE-529?page=comments#action_12377225 ]
Nicholaus Shupe commented on LUCENE-529: ---------------------------------------- I'm experiencing a memory leak in Lucene 1.9.1 that might be related to this problem. Looks like I'll be creating my own patched version of Lucene in the meantime. > TermInfosReader and other + instance ThreadLocal => transient/odd memory > leaks => OutOfMemoryException > ------------------------------------------------------------------------------------------------------- > > Key: LUCENE-529 > URL: http://issues.apache.org/jira/browse/LUCENE-529 > Project: Lucene - Java > Type: Bug > Components: Index > Versions: 1.9 > Environment: Lucene 1.4.3 with 1.5.0_04 JVM or newer......will aplpy to 1.9 > code > Reporter: Andy Hind > Attachments: ThreadLocalTest.java > > TermInfosReader uses an instance level ThreadLocal for enumerators. > This is a transient/odd memory leak in lucene 1.4.3-1.9 and applies to > current JVMs, > not just an old JVM issue as described in the finalizer of the 1.9 code. > There is also an instance level thread local in SegmentReader....which will > have the same issue. > There may be other uses which also need to be fixed. > I don't understand the intended use for these variables.....however > Each ThreadLocal has its own hashcode used for look up, see the ThreadLocal > source code. Each instance of TermInfosReader will be creating an instance of > the thread local. All this does is create an instance variable on each thread > when it accesses the thread local. Setting it to null in the finaliser will > set it to null on one thread, the finalizer thread, where it has never been > created. There is no point to this :-( > I assume there is a good concurrency reason why an instance variable can not > be used... > I have not used multi-threaded searching, but I have used a lot of threads > each making searchers and searching. > 1.4.3 has a clear memory leak caused by this thread local. This use case > above is definitely solved by setting the thread local to null in the > close(). This at least has a chance of being on the correct thread :-) > I know reusing Searchers would help but that is my choice and I will get to > that later .... > Now you wnat to know why.... > Thread locals are stored in a table of entries. Each entry is *weak > reference* to the key (Here the TermInfosReader instance) and a *simple > reference* to the thread local value. When the instance is GCed its key > becomes null. > This is now a stale entry in the table. > Stale entries are cleared up in an ad hoc way and until they are cleared up > the value will not be garbage collected. > Until the instance is GCed it is a valid key and its presence may cause the > table to expand. > See the ThreadLocal code. > So if you have lots of threads, all creating thread locals rapidly, you can > get each thread holding a large table of thread locals which all contain many > stale entries and preventing some objects from being garbage collected. > The limited GC of the thread local table is not enough to save you from > running out of memory. > Summary: > ======== > - remove finalizer() > - set the thread local to null in close() > - values will be available for gc -- This message is automatically generated by JIRA. - If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa - For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
