On Jan 18, 2007, at 2:59 PM, Doron Cohen wrote:
To my understanding the only remaining issue with NFS is: a reader
might get an IO exception in case writer removed an old file that
the reader is using.
It is not a possible corruption that we try to solve, right?
For that I think it is not worth to add that stuff again.
I agree, Doron.
I'd rather leave NFS as a problem case.
Now... how about having Readers establish advisory read locks when
the operating system supports them? That seems to me to be still in
the spirit having Readers be read-only.
Then our problem set is reduced even further: only NFS systems using
protocols prior to version 4.
It's probably not even worth it to perform the lock test I proposed
earlier. We just use file systems the way they're suppose to behave
and eventually NFS catches up.
Provisionally, this is what I intend implement for KS, unless
something better emerges from ongoing discussion.
A writer's "two steps" policy - delete only files that
"would have not been in use unless a reader did not refresh for X
minutes"
is "fair enough" I think.
By "two steps" I mean, start measuring time not from when segment
to be
deleted was created, but rather from when its "next generation" was
created.
Deletions are processed shortly after a new segments_N file gets
written (at least on KS, and IIRC also for Lucene). You'd always
have to leave deletes to the next commit.
Marvin Humphrey
Rectangular Research
http://www.rectangular.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
