A Searcher uses a Reader to read the index for searching.
- Mark
Harini Raghavan wrote:
Hi Mike,
Thank you for the response. I don't have readers open on the index,
but while the optimize/merge was running I was searching on the index.
Would that make any difference?
Also after the optimizing the index I had some .tmp files which were >
10G and did not get merged. Could that also be related to having
searchers open while running optimize?
-Harini
Michael McCandless wrote:
Harini Raghavan wrote:
I am using lucene 1.9.1 for search functionality in my j2ee
application using JBoss as app server. The lucene index directory
size is almost 20G right now. There is a Quartz job that is adding
data to the index evey min and around 20000 documents get added to
the index every day.When the documents are added and the segments
are merged, the index size increases and sometimes grows to more
than double its original size. This results in filling up the disk
space. We have allotted a f/s size of 50G and even that is not
sufficient at times. Is there an optimum vales for the f/s size to
be allotted in such scenario.
Any suggestions would be appreciated.
I believe optimize should use at most 2X the starting index size,
transiently, if there are no readers open against the index. And then
when optimize is done the size should be around the starting size, or
less.
If there are open readers against the index when the optimize occurs,
then, the segments that were merged cannot actually be deleted until
those readers close. Even on Unix, where it will look like the
segments were deleted, they are still consuming disk space because
open file handles keep them allocated ("delete on last close").
This means if you have open readers you should see at most 3X the
starting index size. Worse, if some (but not all) readers are
re-opening while the merge is underway it's possible to peak at even
more than 3X the starting size.
Do you have readers running against your index?
I will call this out in the javadocs for optimize, addDocument,
addIndexes...
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
