On Jan 11, 2007, at 6:48 AM, Michael McCandless wrote:
I too am happy that we have no more commit lock :)
Not just that. :)
No more lock directory, since we can put write.lock in the index
directory itself.
No more lock file name munging, since lock files from different
indexes no longer need to avoid collisions within a shared namespace.
No more need to deal with any files outside of the index directory.
Those three changes have a bigger impact on Lucy than they do on
Lucene, and since I'm writing a lot of KS 0.20 code with the notion
that it will be submitted to Lucy, they're having an impact on what
I'm doing right now. C doesn't provide a number of the dependencies
needed to support the old lock system, so we would either have had to
include them, write them ourselves, or supply the needed
functionality via PITA callbacks to the host language (Perl, Ruby, etc).
Since the lock directory lived in the system's tmp directory, we
needed code to discover where it was. Now we don't.
The lock file name munging required a checksum string generator. We
don't need that now.
Lastly, a failure of imagination had left me blind to the fact that
we didn't need sophisticated, portable filepath manipulating
routines: just knowing a directory separator suffices. Previously,
I'd wrapped Perl's File::Spec::Functions to make catfile() and
canonpath() available from C. That hadn't been necessary, because we
could have built up the lockfile paths given the location of the tmp
directory and the dir_sep. However, as is often the case,
simplifying the implementation reveals unnecessary cruft, and when
all of a sudden everything ended up in one directory with a splash,
it became obvious that generating filepaths didn't require heavy
machinery.
But I have to say the lockless changes pale in comparison to what you
have done/are doing with KinoSearch, specifically the clean merge
model with an external sorter and other related file format changes
look very interesting.
Well, I look forward to seeing whether you can suggest improvements
on some of the algos I'll bring up in this forum once KS 0.20_01 is
out. :)
Marvin Humphrey
Rectangular Research
http://www.rectangular.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
