Re: IndexSearch very slow after reopening the index

2010-10-15 Thread Ian Lea
I'm a bit confused about what exactly you are timing. Is the 46 ms for one search on one term with one hit, or for 100 similar searches or what? Perhaps a minimal self-contained search program demonstrating exactly what you are doing would help, with evidence of where it is spending time. -- Ia

Re: IndexSearch very slow after reopening the index

2010-10-14 Thread subwayne
Ok, I read the Wiki page related to improving the searching speed and adopted some advices. One of the slow queries is simply. Here are some: plaintext:guid 107.0 ms resultSet.totalHits = 1 plaintext:allianc 51.0 ms resultSet.totalHists = 1 plaintext:engin 46.0 ms resultSet.totalHits = 1 plain

Re: IndexSearch very slow after reopening the index

2010-10-14 Thread Ian Lea
OK, so it looks like we're down to a more general "why is searching slow" question. The number of docs is not very large by lucene standards. Work through http://wiki.apache.org/lucene-java/ImproveSearchingSpeed. If that still doesn't help, pick a slow query and post again with: . the output of

Re: IndexSearch very slow after reopening the index

2010-10-14 Thread Pradeep Singh
Many times when you run a search for the first time it has to load all field values IF the field is being sorted on. Subsequent searches use that cache and are faster. Does that happen in your case? From your description it doesn't look like you are sorting, although this kind of performance degrad

Re: IndexSearch very slow after reopening the index

2010-10-14 Thread subwayne
Hi Ian, thank you for your quick response. I am running Lucene on Ubuntu 10.04, 64 bit. I switched from MMapDirectory to NIOFSDirectory without any significant changes in performance. The Lucene version running is 3.0.2. I followed your advice and opened the IndexSearcher after I added all docume

Re: IndexSearch very slow after reopening the index

2010-10-14 Thread Ian Lea
Do the fast searches that you get while the app is running use the searcher you create before you add all the docs to the index? Surely that won't see the added docs. There are general tips on speeding up searches at http://wiki.apache.org/lucene-java/ImproveSearchingSpeed. There are some gotcha