Not just virus scanners: any program that uses the Microsoft API for being notified of file changes. I think TortoiseSVN was one such example.
People who embed Lucene can't control what their users install on their desktops. Virus scanners are naturally very common on desktops. I think we want Lucene to work in these cases. NFS (and other shared filesystems) is a convenient, if not performant, way to share an index. I think Lucene should work in such cases as well. Mike "robert engels" <[EMAIL PROTECTED]> wrote: > What are you basing the "rename" is not reliable on windows on? That > a virus scanner has the file open. If that is the case, that should > either be an incorrect setup, or the operation retried until it > completes. > > Writing directly to a file that someone else can open for reading is > bound to be a problem. If the file is opened exclusive for write, > then the others will be prohibited from opening for read, so there > should not be a problem. > > All of the "delete on last close" stuff is a poor design. The > database can be resync on startup. > > The basic design flaw is one I have pointed out many times - you > either use Lucene in a local environment, or a server environment. > Using NFS to "share" a Lucene database is a poor choice (normally due > to performance, but there are other problems - e.g. resource and user > monitoring, etc.) is a poor choice !. > > People have written reliable database systems without very advanced > semantics for years. There is no reason for all of this esoteric code > in Lucene. > > Those that claim, Lucene had problems with NFS in the past, did not > perform reliable testing, or their OS was out of date. What is > Lucene was failing for an OS needed an update, would you change > Lucene, or fix/update the OS??? Obviously the former. > > Some very loud voices complained about the NFS problems without doing > the due diligence and test cases to prove the problem. Instead they > just mucked up the Lucene code. > > > On Nov 12, 2007, at 4:54 PM, Michael McCandless wrote: > > > > > robert engels <[EMAIL PROTECTED]> wrote: > > > >> Then how can the commit during reopen be an issue? > > > > This is what happens: > > > > * Reader opens latest segments_N & reads all SegmentInfos > > successfully. > > > > * Writer writes new segments_N+1, and then deletes now un-referenced > > files. > > > > * Reader tries to open files referenced by segments_N and hits FNFE > > when it tries to open a file writer just removed. > > > > Lucene handles this fine (it just retries on the new segments_N+1), > > but the patch in LUCENE-743 is now failing to decRef the Norm > > instances when this retry happens. > > > >> I am not very family with this new code, but it seems that you need > >> to write segments.XXX.new and then rename to segments.XXX. > > > > We don't rename anymore (it's not reliable on windows). We write > > straight to segments_N. > > > >> As long as the files are sync'd, even on nfs the reopen should not > >> see segments.XXX until is is ready. > >> > >> Although lockless commits are beneficial in their own rite, I still > >> think that people's understanding of NFS limitations are > >> flawed. Read the section below on "close to open" consistency. There > >> should be no problem using Lucene across NFS - even the old version. > >> > >> The write-once nature of Lucene makes this trivial. The only > >> problem was the segments file, which is lucene used the read/write > >> lock and close(0 correctly never would have been a problem. > > > > Yes, in an ideal world, NFS server+clients are supposed to implement > > close-to-open semantics but in my experience they do not always > > succeed. Previous version of Lucene do in fact have problems over > > NFS. NFS also does not give you "delete on last close" which Lucene > > normally relies on (unless you create a custom deletion policy). > > > > Mike > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
