Re: First search is slow after updating index .. subsequent searches very fast

2006-12-22 Thread Yonik Seeley
On 12/22/06, Mark Miller [EMAIL PROTECTED] wrote: So the first time you do a sort, the fieldcache is loaded up that stores the term to sort on for each document id. Right. The actual sorting appears to happen just like with relevancy score sortingusing a priority queue that is loaded as

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-21 Thread Bogdan Ghidireac
Otis, I am not familiar with the 'dd trick' to warm up the index. Can you please explain it ? Bogdan On 12/20/06, Otis Gospodnetic [EMAIL PROTECTED] wrote: To populate FieldCache, the number of matches doesn't matter. There is no need to be scrimy there - you don't really save anything by

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-21 Thread Otis Gospodnetic
Message From: Bogdan Ghidireac [EMAIL PROTECTED] To: java-user@lucene.apache.org Sent: Thursday, December 21, 2006 3:58:52 AM Subject: Re: First search is slow after updating index .. subsequent searches very fast Otis, I am not familiar with the 'dd trick' to warm up the index. Can you

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-21 Thread Joe Shaw
Hi, On Thu, 2006-12-21 at 10:21 -0800, Otis Gospodnetic wrote: Something like dd if=/path/to/index/foo.cfs of=/dev/null Basically, force the data through the kernel preemptively, so FS caches it. Run vmstat while doing it, and if the index hasn't been cached by the FS, you should see a spike

RE: First search is slow after updating index .. subsequent searches very fast

2006-12-21 Thread Bryan Dotzour
indices, one can also use the `dd' trick under UNIX. Otis - Original Message From: Bryan Dotzour [EMAIL PROTECTED] To: java-user@lucene.apache.org Sent: Wednesday, December 20, 2006 5:23:40 PM Subject: RE: First search is slow after updating index .. subsequent searches very fast One

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-21 Thread Doron Cohen
Something like dd if=/path/to/index/foo.cfs of=/dev/null Be careful not to mistaken with the 'of' argument of 'dd' - see http://en.wikipedia.org/wiki/Dd_(Unix) - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-21 Thread Mark Miller
new indices, one can also use the `dd' trick under UNIX. Otis - Original Message From: Bryan Dotzour [EMAIL PROTECTED] To: java-user@lucene.apache.org Sent: Wednesday, December 20, 2006 5:23:40 PM Subject: RE: First search is slow after updating index .. subsequent searches very fast One

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-20 Thread Otis Gospodnetic
after updating index .. subsequent searches very fast I'm investigating some performance issues with the way we're using Lucene in our web app and am interested if anyone could shed some light on what might be going on. Hopefully I can provide enough information, please let me know if there's more

RE: First search is slow after updating index .. subsequent searches very fast

2006-12-20 Thread Bryan Dotzour
: Re: First search is slow after updating index .. subsequent searches very fast All sounds good. Opening a new IndexReader can take a bit of time. If you use sorting of any kind other than default sorting by relevance, this delay on the first search is also probably caused by the lazy FieldCache

RE: First search is slow after updating index .. subsequent searches very fast

2006-12-20 Thread Bryan Dotzour
-Original Message- From: Otis Gospodnetic [mailto:[EMAIL PROTECTED] Sent: Wednesday, December 20, 2006 3:28 PM To: java-user@lucene.apache.org Subject: Re: First search is slow after updating index .. subsequent searches very fast All sounds good. Opening a new IndexReader can take a bit

Re: First search is slow after updating index .. subsequent searches very fast

2006-12-20 Thread Otis Gospodnetic
under UNIX. Otis - Original Message From: Bryan Dotzour [EMAIL PROTECTED] To: java-user@lucene.apache.org Sent: Wednesday, December 20, 2006 5:23:40 PM Subject: RE: First search is slow after updating index .. subsequent searches very fast One question about this, Otis... When warming up