At this size, really consider going to a single index. The lack of administrative headaches alone is probably well worth the effort....
I almost guarantee that the time you spend re-writing things to keep the searchers open (and finding the bugs!) will be far more than just putting all the data in a single index. But that might just be my preferences showing.... Best Erick On Wed, Jun 1, 2011 at 5:37 PM, Alexander Rosemann <alexander.rosem...@gmail.com> wrote: > Many thanks for the tips, Erick! I do close each searcher after a search... > I will change that first thing tmrw. and let you know how that went. > Multi-threaded searching will be next and if that hasn't helped, I will > switch to one big index. > All indexes together are rather small, ~200MB and 50.000 documents. > > -Alex > > On 01.06.2011 23:26, Erick Erickson wrote: >> >> I'd start by putting them all in one index. There's no penalty >> in Lucene for having empty fields in a document, unlike an >> RDBMS. >> >> Alternately, if you're opening then closing searchers each >> time, that's very expensive. Could you open the searchers >> once and keep them open (all 90 of them)? That alone might >> do the trick and be less of a change to your program. You >> could also fire multiple threads at the searches, but check if >> you're CPU bound first (if you are, multiple threads won't >> help much/at all). >> >> You haven't said how big these indexes are nor how many >> documents you're talking about here, so this advice is suspect. >> >> Do look at putting it all in one index though, let us know if you >> have some data indicating how big stuff is/would be. >> >> Best >> Erick >> >> On Wed, Jun 1, 2011 at 4:35 PM, Alexander Rosemann >> <alexander.rosem...@gmail.com> wrote: >>> >>> Hi all, I was wondering whether you could give me some advice on how to >>> improve my search performance. >>> >>> I have 90 lucene indexes, each having different fields (~5 per Document). >>> When I search, I always have to go through all indexes to build my result >>> set. Searching one index takes approx. 100ms, thus searching all indexes >>> takes 9s in total. >>> >>> How can I reduce the time it needs to search? >>> >>> I decided to create this many indexes because putting all data in one >>> index >>> would mean that a document would have ~400 fields, with most of them left >>> empty. Is that ok? Would a single index be faster compared to multiple >>> small >>> ones? >>> >>> Any pointers are much appreciated. >>> >>> Regards, >>> Alex >>> >>> --------------------------------------------------------------------- >>> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >>> For additional commands, e-mail: java-user-h...@lucene.apache.org >>> >>> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-user-h...@lucene.apache.org >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org
