Peter, As i understand correctly, implementing the equals and hashCode methods let us avoid caching the same sorting values, i.e. if we have a few points to calc distance from then we can implement equals and hashCode methods based on point value and we'll have one cached comparator per different center point value.
However, if every new search we have different center point value then we still in enormous memory usage problem. Aleksey On 8/21/06, Peter Keegan <[EMAIL PROTECTED]> wrote:
I avoided the cache by implementing the equals and hashCode methods appropriately in my custom ScoreDocComparator as described here: http://www.lucenebook.com/blog/errata/2006/03/01/memory_leak.html Peter On 8/21/06, Aleksey Serba <[EMAIL PROTECTED]> wrote: > > Hi Chris, > > On 5/17/06, Peter Keegan <peterlkeegan[at]gmail.com> wrote: > > Suppose I have a custom sorting 'DocScoreComparator' for computing > distances > > on each search hit from a specified coordinate (similar to the > > DistanceComparatorSource example in LIA). Assume that the 'specified > > coordinate' is different for each query. This means a new custom > comparator > > must be created for each query, which is ok. However, Lucene caches the > > comparator even though it will never be reused. This could result in > heavy > > memory usage if many queries are performed before the IndexReader is > > updated. > > Lets suppose another custom sortings - random, using dynamically > changed values from database, etc. Everytime lucene caches _useless_ > custom comparators and application developer has no any control on it. > > Also, I have no evidence to close IndexReader if my index is static. > > Aleksey > > On 8/18/06, Chris Hostetter <[EMAIL PROTECTED]> wrote: > > > > : You can reproduce OutOfMemory easily. I've attach test files - this is > > : altered DistanceSortingTest example from LIA book. Also you can > > : profile it and see caching of distances arrays. > > > > An OutOfMemory error is differnet from a memory leak. Sorting with a > > custom Comparator does in fact use a lot of memory -- and if your Heap > > size is not big enough you may get an OutOfMemory -- but that doesn't > mean > > you have a memory leak. A Memory leak is a problem where memory is > > allocated for objects, but not freed up once those objects are no longer > > of use -- just becuase you aren't using those objects, doesn't mean they > > aren't "of use" and it doesn't mean they won't be freed up once some > > futher action is taken (ie: closing of an IndexReader) > > > > Are you sure you've found an actual memory leak? ... ie: do you have a > > demonstratable test case that shows heap usage growing without bound > over > > time? If so then seeing *that* code would be helpful to figure out if > > there is in fact a memory leak in Lucene, or if (perhaps) there is a > > mistaken assumption in your code (ie; not closing an IndexReader) > > > > > > > > > > -Hoss > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > >
--------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
