I was just trying to figure out why reopen() could fail ???
It would seem that on unix/linux the files could always be read
(since they don't delete until last close), and on Windows they would
be there (since they can't be deleted because they are open).
When "deleting segments", it would seem that the deleter should get
an exclusive open on segments.xxx file first, before deleting any of
the segments it refers to. If it does that, then it does seem that an
open/reopen can ever fail mid-open. The only thing need is during an
open/reopen is to back-off timeout if a read level shared lock cannot
be obtained on the segments.xxx file.
For us it doesn't make much difference (no multi-process readers), so
if everyone else is ok with the current behavior, we can move on...
Sorry if I wasted anyone's time.
On Nov 13, 2007, at 4:36 AM, Michael McCandless wrote:
I've become lost in this thread.
It started with the unit test in the reopen patch, suddenly jumped to
NFS and lockless commits, then sync() got pulled in as well.
The retry logic (part of lockless commits) already handles this case
of a segments file being partially written & falling back to the
previous one.
Robert what problem are you trying to solve here?
Mike
"robert engels" <[EMAIL PROTECTED]> wrote:
True. It seems that the Lucene code might be a bit more resilient
here though, using the following:
1. open the segments file exclusively (if this fails, updates are
prohibited, and an exception is thrown)
2. write new segments
3. write segments.new including segments hash & sync
4. update segments file including hash
5. delete segments that you can
Then if it crashes in step 4, it is easy to know segments is bad (out
of date) and use segments.new
If it crashes in steps 3, then segments.new is easily detected as
being corrupt (hash does not match), so you know segments is valid.
if there are segments that cannot be deleted in 5, every open can
check if it can delete them...
A similar technique can be used if using lockless commits, just need
to make it segments.XXX.new, etc.
On Nov 12, 2007, at 7:21 PM, Yonik Seeley wrote:
On Nov 12, 2007 7:19 PM, robert engels <[EMAIL PROTECTED]>
wrote:
I would still argue that it is an incorrect setup - almost as
bad as
"not plugging the computer in".
A user themselves could even go in and look at the index files (I've
done so myself)... as could a backup program or whatever. It's a
fact
of life on windows that a move or delete can fail.
-Yonik
--------------------------------------------------------------------
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
