[jira] Commented: (LUCENE-388) [PATCH] IndexWriter.maybeMergeSegments() takes lots of CPU resources

2006-08-21 Thread Yonik Seeley (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-388?page=comments#action_12429525 ] Yonik Seeley commented on LUCENE-388: - No problem... I probably should have put "LUCENE-388" in the commit log (it allows JIRA to link up the related commits a

[jira] Commented: (LUCENE-388) [PATCH] IndexWriter.maybeMergeSegments() takes lots of CPU resources

2006-08-21 Thread Doron Cohen (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-388?page=comments#action_12429522 ] Doron Cohen commented on LUCENE-388: Oh - sorry for the 'noise' - got used to systems where a commit must be attached to an issue/defect - should have checked

Re: Custom sorting - memory leaks

2006-08-21 Thread Aleksey Serba
Chris, I see what you're saying now ... yes, for cases like this it probably would be useful to a way to prevent the Comparator from being cached ... That's what I'm talking about. I agree this is very uncommon case. perhaps by adding a SortComparatorSource.isCachable() method ... but the cha

ComparatorKey in Locale based sorting?

2006-08-21 Thread Chris Hostetter
I've never really looked at the javadocs for Collator much before yesterday, but I just noticed a comment about a class i'd never heard of: CollationKey ... > For comparing Strings exactly once, the compare method provides the best > performance. When sorting a list of Strings however, it is gene

Re: Custom sorting - memory leaks

2006-08-21 Thread Chris Hostetter
: > DistanceComparatorSource example in LIA). Assume that the 'specified : > coordinate' is different for each query. This means a new custom comparator : > must be created for each query, which is ok. However, Lucene caches the : Lets suppose another custom sortings - random, using dynamically :

Re: Custom sorting - memory leaks

2006-08-21 Thread Peter Keegan
I forgot to mention that the query threads are created only once. Peter On 8/21/06, Peter Keegan <[EMAIL PROTECTED]> wrote: Aleksey, My app has precise and predictable control over when the index reader/searcher gets refreshed. When this occurs, the sorting values for all the docs in the new i

Re: [jira] Commented: (LUCENE-436) [PATCH] TermInfosReader, SegmentTermEnum Out Of Memory Exception

2006-08-21 Thread robert engels
Correct, a soft reference will not work, since it CAN'T be allowed to be GC'd or all of the optimization would be lost. Review the archives, you will find the solution. As I have stated MANY times, we use Lucene in multiple productions environments, using 1.4.2 and 1.5 with HIGHLY interacti

Re: Custom sorting - memory leaks

2006-08-21 Thread Peter Keegan
Aleksey, My app has precise and predictable control over when the index reader/searcher gets refreshed. When this occurs, the sorting values for all the docs in the new index are generated by the app and cached. Each query thread has it's own ScoreDocComparator which contains a reference to the s

[jira] Commented: (LUCENE-436) [PATCH] TermInfosReader, SegmentTermEnum Out Of Memory Exception

2006-08-21 Thread Antony Scerri (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-436?page=comments#action_12429451 ] Antony Scerri commented on LUCENE-436: -- Lets forget that last comment I made, I was testing out a simple setup and forgot there wasn't going to be a hard refe

Re: Lock-less commits

2006-08-21 Thread robert engels
You are correct - I wasn't thinking of it that way. On Aug 21, 2006, at 10:44 AM, Yonik Seeley wrote: On 8/21/06, robert engels <[EMAIL PROTECTED]> wrote: Then keeping the segments in memory is not helpful, as every open of the writer needs to traverse the directory (since another writer still

Re: Lock-less commits

2006-08-21 Thread Yonik Seeley
On 8/21/06, robert engels <[EMAIL PROTECTED]> wrote: Then keeping the segments in memory is not helpful, as every open of the writer needs to traverse the directory (since another writer still could have created segments). For example, Computer A opens writer, modifies index, closes writer. Com

Re: Lock-less commits

2006-08-21 Thread robert engels
Then keeping the segments in memory is not helpful, as every open of the writer needs to traverse the directory (since another writer still could have created segments). For example, Computer A opens writer, modifies index, closes writer. Computer B opens writer (this must read the directory

Re: [jira] Commented: (LUCENE-436) [PATCH] TermInfosReader, SegmentTermEnum Out Of Memory Exception

2006-08-21 Thread robert engels
Your solution will not work, as it will be GC'd soon after it is inserted - since there is no longer a reference. There are other ways to do this, but as has been discussed on the list many times, the OOM is almost certainly due to something else (or a max heap size that is doesn't allow fo

Re: Custom sorting - memory leaks

2006-08-21 Thread Aleksey Serba
Peter, As i understand correctly, implementing the equals and hashCode methods let us avoid caching the same sorting values, i.e. if we have a few points to calc distance from then we can implement equals and hashCode methods based on point value and we'll have one cached comparator per different

Re: Custom sorting - memory leaks

2006-08-21 Thread Peter Keegan
I avoided the cache by implementing the equals and hashCode methods appropriately in my custom ScoreDocComparator as described here: http://www.lucenebook.com/blog/errata/2006/03/01/memory_leak.html Peter On 8/21/06, Aleksey Serba <[EMAIL PROTECTED]> wrote: Hi Chris, On 5/17/06, Peter Keegan

[jira] Commented: (LUCENE-436) [PATCH] TermInfosReader, SegmentTermEnum Out Of Memory Exception

2006-08-21 Thread Antony Scerri (JIRA)
[ http://issues.apache.org/jira/browse/LUCENE-436?page=comments#action_12429418 ] Antony Scerri commented on LUCENE-436: -- Would it not be easier to simply make the objects stored in the ThreadLocal a WeakReference. So in the case of TermInf

Re: Custom sorting - memory leaks

2006-08-21 Thread Aleksey Serba
Hi Chris, On 5/17/06, Peter Keegan wrote: Suppose I have a custom sorting 'DocScoreComparator' for computing distances on each search hit from a specified coordinate (similar to the DistanceComparatorSource example in LIA). Assume that the 'specified coordinate' is different for each query. Thi

Re: index update with database insertion

2006-08-21 Thread Michael McCandless
> In my project,I want to update the lucene's index when there has database > insertion operations,in this way,my users could search the fresh information > immediately if someone inserted the information into database.That's what I > need,could someone give me suggestions to implement my need?