Hi Mike,
Thank you for the response. I don't have readers open on the index, but
while the optimize/merge was running I was searching on the index. Would
that make any difference?
Also after the optimizing the index I had some .tmp files which were >
10G and did not get merged. Could that also be related to having
searchers open while running optimize?
-Harini
Michael McCandless wrote:
Harini Raghavan wrote:
I am using lucene 1.9.1 for search functionality in my j2ee
application using JBoss as app server. The lucene index directory
size is almost 20G right now. There is a Quartz job that is adding
data to the index evey min and around 20000 documents get added to
the index every day.When the documents are added and the segments are
merged, the index size increases and sometimes grows to more than
double its original size. This results in filling up the disk space.
We have allotted a f/s size of 50G and even that is not sufficient at
times. Is there an optimum vales for the f/s size to be allotted in
such scenario.
Any suggestions would be appreciated.
I believe optimize should use at most 2X the starting index size,
transiently, if there are no readers open against the index. And then
when optimize is done the size should be around the starting size, or
less.
If there are open readers against the index when the optimize occurs,
then, the segments that were merged cannot actually be deleted until
those readers close. Even on Unix, where it will look like the
segments were deleted, they are still consuming disk space because
open file handles keep them allocated ("delete on last close").
This means if you have open readers you should see at most 3X the
starting index size. Worse, if some (but not all) readers are
re-opening while the merge is underway it's possible to peak at even
more than 3X the starting size.
Do you have readers running against your index?
I will call this out in the javadocs for optimize, addDocument,
addIndexes...
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
--
Harini Raghavan
Software Engineer
Office : +91-40-23556255
[EMAIL PROTECTED]
we think, you sell
www.InsideView.com
InsideView
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
