Thanks Hoss,
I did figure out that I was putting about 400 stored fields per
document into my new index; more than my prior indexes.
Reducing the number of stored fields seems to have helped significantly.
I do call writer.optimize() after loading in documents, but not sure how I
would set the # of segments?
I think I will keep the IndexSearcher statically for all instances. The slow
times I was seeing, weren't even sufficient for that though.
Since this is a case of really only needing to search on one field and use
the index as a storage medium for the rest of the data(pretty much textual
data), I'm thinking it would make sense to get the latest version of lucene
and create a two field index.
Something like:
Field1: id
Field2: serialized data object.
Any reason why that wouldn't be fast?
I have been having elusive memory issues with my other usage, maybe you just
helped me find that solution as well.
Thanks,
-Gus
-----Original Message-----
From: Chris Hostetter [mailto:[EMAIL PROTECTED]
Sent: Wednesday, February 22, 2006 4:02 PM
To: java-user@lucene.apache.org
Subject: Re: IndexSearcher
: I have one index where the instantiation is very fast, to the point where
I
: don't need to do any pooling. A new index I have created, takes a very
long
: time to create the IndexSearcher object. With a 30mb index, it can take
: about 30 seconds just to instantiate an IndexSearcher(). It almost seems
: like it is reading the index at that point.
:
:
: The only difference between the indexes has been the # of fields indexed.
: The newer one only having one field indexed.
If i remember correctly, The IndexSearcher constructor doesn't do anything
but open an IndexReader ... IndexReader.open() opens a MultiReader on all
of the segments, and each of the SegmentReaders open up a bunch of files.
so off hte top of my head, one thing that can make a differnece in the
"new IndexSearcher" times, is how many segments you have in your index
(ie: is it optimized?) ... using the compound fileformat can probably make
a difference as well.
: Any ways to speed up that instantiation? Or do I have to use a pooling
: system?
Even if you get it down to 0.00001 seconds,i would still reuse the same
IndexSearcher as much as possible. See previous replies from me in
the archive about memory for my reasoning.
-Hoss
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
