What are you basing the "rename" is not reliable on windows on? That
a virus scanner has the file open. If that is the case, that should
either be an incorrect setup, or the operation retried until it
completes.
Writing directly to a file that someone else can open for reading is
bound to be a problem. If the file is opened exclusive for write,
then the others will be prohibited from opening for read, so there
should not be a problem.
All of the "delete on last close" stuff is a poor design. The
database can be resync on startup.
The basic design flaw is one I have pointed out many times - you
either use Lucene in a local environment, or a server environment.
Using NFS to "share" a Lucene database is a poor choice (normally due
to performance, but there are other problems - e.g. resource and user
monitoring, etc.) is a poor choice !.
People have written reliable database systems without very advanced
semantics for years. There is no reason for all of this esoteric code
in Lucene.
Those that claim, Lucene had problems with NFS in the past, did not
perform reliable testing, or their OS was out of date. What is
Lucene was failing for an OS needed an update, would you change
Lucene, or fix/update the OS??? Obviously the former.
Some very loud voices complained about the NFS problems without doing
the due diligence and test cases to prove the problem. Instead they
just mucked up the Lucene code.
On Nov 12, 2007, at 4:54 PM, Michael McCandless wrote:
robert engels <[EMAIL PROTECTED]> wrote:
Then how can the commit during reopen be an issue?
This is what happens:
* Reader opens latest segments_N & reads all SegmentInfos
successfully.
* Writer writes new segments_N+1, and then deletes now un-referenced
files.
* Reader tries to open files referenced by segments_N and hits FNFE
when it tries to open a file writer just removed.
Lucene handles this fine (it just retries on the new segments_N+1),
but the patch in LUCENE-743 is now failing to decRef the Norm
instances when this retry happens.
I am not very family with this new code, but it seems that you need
to write segments.XXX.new and then rename to segments.XXX.
We don't rename anymore (it's not reliable on windows). We write
straight to segments_N.
As long as the files are sync'd, even on nfs the reopen should not
see segments.XXX until is is ready.
Although lockless commits are beneficial in their own rite, I still
think that people's understanding of NFS limitations are
flawed. Read the section below on "close to open" consistency. There
should be no problem using Lucene across NFS - even the old version.
The write-once nature of Lucene makes this trivial. The only
problem was the segments file, which is lucene used the read/write
lock and close(0 correctly never would have been a problem.
Yes, in an ideal world, NFS server+clients are supposed to implement
close-to-open semantics but in my experience they do not always
succeed. Previous version of Lucene do in fact have problems over
NFS. NFS also does not give you "delete on last close" which Lucene
normally relies on (unless you create a custom deletion policy).
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
