Doron Cohen wrote:
In my local changes (using numbered files) for lock-less commits, I've
implemented Yonik's suggestsion of opening segments in reverse order,
and this has definitely reduced the number of "retries" that the
searchers hit on opening the index. Even in highly interactive
searching (open searcher, do one search, close searcher, repeat) the
retry rate is low.
In this highly interactive search scenario is it true that every opened
searcher needs a directory listing? - If so is this a possible performance
hit for the searchers, similar to discussion in this thread for writers.
But we should worry more for searchers... In this case, how about
maintaining a separate version-file (as discussed) for allowing new
searchers/readers to easily detect the most current version that is stable
and should be used, without needing a directory listing. I believe that
NFS-wise, it should be as safe to use as a directory listing (more on NFS
below).
Yes in my current implementation I do a dir listing to locate the
current segments file.
The big problem is the client's caching. I've seen cases in my own
testing where the NFS cache on one machine remains stale for quite some
time (seconds) before "seeing" changes to a file on a server. I think
instead relying on a newly created file with the numbered approach (ie
never before used file name) will avoid the risk that a client-side
cache is presenting stale (or delayed) contents of a file.
I can't see why relying on files newly created is safer than relying on
files that were updated. In other words, I think we are ok with stales -
e.g. if the writer wrote 5 new files, and then wrote a 6th new file - the
appropriate segments file, it seems the numbered-files approach too is
counting on that the first 5 written files are visible to any client once
the 6th file is visible, otherwise readers might not find the files they
are looking for (the segment-infos just read from the 6th file). I
understand "staling" as - "although the 6th file was already written, a
reader might not yet see it yet, it would only see the first 5 files". But
this is not a problem, because the reader would attempt to use the previous
version. If that would fail, the reader would retry and now probably get
the newer version.
So it seems that the numbered-files scheme also relies on proper order of
operations, or I might misunderstand something here. Under this same
"visibility order assumption", I think the version-file should be safe to
use in NSF.
OK I think there are two issues here:
* Is using a newly created file actually safer than re-using a
filename?
I believe it should be: by using a newly created filename, there
is no risk that this file's contents is in the local (possibly
stale) filesystem cache. Ie the filesystem is forced to go to the
server to retrieve the contents of the file. Beyond NFS, I would
think we get ourselves into less trouble with filesystem caching
by never re-using a filename.
One other important reason: users have reported errors in renaming
segments.new -> segments (and deletable, but that file is no
longer used with my changes) on Windows. This is probably due to
the fact that a reader is reading the segments at that moment.
While will could insert a "sleep; try again" loop into the writer,
but that's adding possibly unnecessary latency.
* Relying on "order of operations": yes, I agree, the numbered file
approach does rely on a limited use-case of this (see below).
However, so does the current Lucene (with its current locking or
even once we get native locking implemented).
So in this regard numbered files is no worse than the current
Lucene -- "first do no harm" :)
The numbered file locking relies on a limited use case "creation
of new files in a single directory". I'm certainly no NFS expert
(but I'm rapidly getting tempted to buy NFS Illustrated!!), but
looking at how the NSF client/server protocol works, I would be
surprised if this specific use case fails to preserve of
order-of-file-creates. An NFS client client can't know about a
new file X until it contacts the server to do a READDIR or
READDIRPLUS, and at the point that it does so, it should get the
then-current current list of files in the directory.
Whereas, the current Lucene does rely on a more powerful "order of
operations" involving fresh contents of the "segments" file
against existence/freshness of the separate segments files. This
could indeed be the source of the FileNotFound exceptions users
see over NFS (or, it could also be that
java.io.File.createNewFile() is not actually atomic over NFS).
I agree that "order of operations" in general, if you span
multiple directories and you're including in the fresh contents of
re-used filenames, is very likely not guaranteed by NFS or other
remote filesystems: it's just very costly to do truly correctly.
> Under this no-order assumption, it is also possible that a file
exists but
> only part of it is available for the reader - to defend against this the
> reader should be able to verify that the segments file is complete, and
> that each of the other files is complete - using the file length or so,
> right?
Unfortunately, relying on accurate file sizes through NFS caching is
dangerous. In my local mods I take the same approach as before, ie,
write to a .new file and then rename it. This way the shear existence
of a segments_N file means it's now complete.
Mike
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
