It would make me nervous to have Lucene insert that shutdown hook.
EG closing the IndexWriter could in general be a time-consuming
process. But if it's working for you, that's great. Though, if you
explicitly kill the JVM (eg kill -9) those shutdown hooks won't run.
You should use org.apache.lucene.index.CheckIndex to check the health
of your index.
Mike
On May 8, 2008, at 4:06 AM, Jamie wrote:
Hi Michael
I had in fact preempted you and moved the delete lock code to a
startup function. However, I found a nice little optimization that
seems to force the writer to close when the process is manually
killed. I added a JVM shutdown hook (i.e. using Runtime.getRuntime
().addShutdownHook(this) ) that closes the writer. This seems to
reduce the number of instances where the lock file is left lying
around. Perhaps, it is something that could be included in the
base lucene code. Also, are you aware of any chkdsk like utility
for Lucene? i.e. in the event that an index is corrupted through
disk error or otherwise?
Many thanks!
Jamie
Michael McCandless wrote:
OK, that sounds like a legitimate reason to forcibly remove the
write lock, but it would be better to do that only on startup of
your process rather than in every openIndex() call.
If ever you hit LockObtainFailedException in openIndex, even after
having deleted the write lock on startup, then that means there's
a bug (ie, two writers are in fact trying to open on the same
index, one of the two cases below).
Mike
Jamie wrote:
Hi Mike
Thanks for the suggestions. I've implemented all of them. The
main reason why I manually deleted the lock file was that
sometimes users kill the server process manually or there is a
hard reboot without any warning. In such circumstances, Lucene
leaves a lock file lying around as it was busy writing to the
index. Now, I understand that one shouldn't simply delete the
lock file, but what do you suggest my users do? The server must
continue running... the only way that I see how is to delete the
lock file, unless there is the equivalent of chkdsk for Lucene
indexes that I could run.
Regards,
Jmaie
Michael McCandless wrote:
On quickly looking through the code I think there are some
serious hazards that could lead to this exception.
First, in your openIndex code, if you hit a
LockObtainFailedException in trying to open your writer, you are
forcefully removing the write lock and then retrying. Yet, you
also open an IndexReader to delete documents, which acquires the
write lock. If ever you have this IndexReader open, and then
you forcefully remove the write lock and open the writer, that
would cause this exception.
Second, you have a deletIndex method, which first tries to use
the writer with create=true (good) but then falls back to
manually removing the files. Why is that fallback necessary?
If, for example, you are also hitting a
LockObtainFailedException, then forcefully removing files while
an IndexReader or IndexWriter holds the write lock is also
dangerous and would lead to this exception.
In general it's very dangerous to forcibly remove, or ignore,
Lucene's write lock. It really should only be necessary when
something catastrophic occurred (JVM crashed).
Also, note that IndexWriter can now delete documents. This
would simplify your code and possibly fix these two hazards.
Do you see any of the error/warnings that you send to your
logger? (They would be corroborating evidence here).
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
