I avoided the cache by implementing the equals and hashCode methods appropriately in my custom ScoreDocComparator as described here: http://www.lucenebook.com/blog/errata/2006/03/01/memory_leak.html
Peter On 8/21/06, Aleksey Serba <[EMAIL PROTECTED]> wrote:
Hi Chris, On 5/17/06, Peter Keegan <peterlkeegan[at]gmail.com> wrote: > Suppose I have a custom sorting 'DocScoreComparator' for computing distances > on each search hit from a specified coordinate (similar to the > DistanceComparatorSource example in LIA). Assume that the 'specified > coordinate' is different for each query. This means a new custom comparator > must be created for each query, which is ok. However, Lucene caches the > comparator even though it will never be reused. This could result in heavy > memory usage if many queries are performed before the IndexReader is > updated. Lets suppose another custom sortings - random, using dynamically changed values from database, etc. Everytime lucene caches _useless_ custom comparators and application developer has no any control on it. Also, I have no evidence to close IndexReader if my index is static. Aleksey On 8/18/06, Chris Hostetter <[EMAIL PROTECTED]> wrote: > > : You can reproduce OutOfMemory easily. I've attach test files - this is > : altered DistanceSortingTest example from LIA book. Also you can > : profile it and see caching of distances arrays. > > An OutOfMemory error is differnet from a memory leak. Sorting with a > custom Comparator does in fact use a lot of memory -- and if your Heap > size is not big enough you may get an OutOfMemory -- but that doesn't mean > you have a memory leak. A Memory leak is a problem where memory is > allocated for objects, but not freed up once those objects are no longer > of use -- just becuase you aren't using those objects, doesn't mean they > aren't "of use" and it doesn't mean they won't be freed up once some > futher action is taken (ie: closing of an IndexReader) > > Are you sure you've found an actual memory leak? ... ie: do you have a > demonstratable test case that shows heap usage growing without bound over > time? If so then seeing *that* code would be helpful to figure out if > there is in fact a memory leak in Lucene, or if (perhaps) there is a > mistaken assumption in your code (ie; not closing an IndexReader) > > > > > -Hoss > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
