Hi Michael
I had in fact preempted you and moved the delete lock code to a startup
function. However, I found a nice little optimization that seems to
force the writer to close when the process is manually killed. I added a
JVM shutdown hook (i.e. using Runtime.getRuntime().addShutdownHook(this)
) that closes the writer. This seems to reduce the number of instances
where the lock file is left lying around. Perhaps, it is something that
could be included in the base lucene code. Also, are you aware of any
chkdsk like utility for Lucene? i.e. in the event that an index is
corrupted through disk error or otherwise?
Many thanks!
Jamie
Michael McCandless wrote:
OK, that sounds like a legitimate reason to forcibly remove the write
lock, but it would be better to do that only on startup of your
process rather than in every openIndex() call.
If ever you hit LockObtainFailedException in openIndex, even after
having deleted the write lock on startup, then that means there's a
bug (ie, two writers are in fact trying to open on the same index, one
of the two cases below).
Mike
Jamie wrote:
Hi Mike
Thanks for the suggestions. I've implemented all of them. The main
reason why I manually deleted the lock file was that sometimes users
kill the server process manually or there is a hard reboot without
any warning. In such circumstances, Lucene leaves a lock file lying
around as it was busy writing to the index. Now, I understand that
one shouldn't simply delete the lock file, but what do you suggest my
users do? The server must continue running... the only way that I see
how is to delete the lock file, unless there is the equivalent of
chkdsk for Lucene indexes that I could run.
Regards,
Jmaie
Michael McCandless wrote:
On quickly looking through the code I think there are some serious
hazards that could lead to this exception.
First, in your openIndex code, if you hit a
LockObtainFailedException in trying to open your writer, you are
forcefully removing the write lock and then retrying. Yet, you also
open an IndexReader to delete documents, which acquires the write
lock. If ever you have this IndexReader open, and then you
forcefully remove the write lock and open the writer, that would
cause this exception.
Second, you have a deletIndex method, which first tries to use the
writer with create=true (good) but then falls back to manually
removing the files. Why is that fallback necessary? If, for
example, you are also hitting a LockObtainFailedException, then
forcefully removing files while an IndexReader or IndexWriter holds
the write lock is also dangerous and would lead to this exception.
In general it's very dangerous to forcibly remove, or ignore,
Lucene's write lock. It really should only be necessary when
something catastrophic occurred (JVM crashed).
Also, note that IndexWriter can now delete documents. This would
simplify your code and possibly fix these two hazards.
Do you see any of the error/warnings that you send to your logger?
(They would be corroborating evidence here).
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
