Oh then I don't think you need a custom deletion policy.

A single NFS client emulates delete-on-last-close semantics. Ie, if the deletion of the file, and the held-open file handles, happen through a single NFS client, then it emulates delete-on-last-close semantics for you, by creating those .nfsXXX files on the server.

Test to be sure though!

Mike

Tim Brennan wrote:

No, I have multiple readers in the same VM. I track open readers within my VM and save those reader's commit points until the readers are gone.

--tim



----- "Michael McCandless" <[EMAIL PROTECTED]> wrote:

OK got it.  Yes, sharing an index over NFS requires your own
DeletionPolicy.

So presumably you have readers on different machines accessing the
index over NFS, and then one machine that does the writing to that
index?  How so you plan to get which commit point each of these
readers is currently using back to the single writer?

Mike

Tim Brennan wrote:

The bigger picture here is NFS-safety.  When I run a search, I hand

off the search results to another thread so that they can be
processed as necessary -- in particular so that they can be JOINed

with a SQL DB -- but I don't want to completely lock the index from

writes while doing a bunch of SQL calls.  Using the commit point
tracking I can make sure my appropriate snapshot stays around until

I'm completely done with it, even if I'm using an NFS mounted
filesystem that doesn't have delete-last semantics.

--tim




It seems like it should be pretty simple -- keep a list of open
IndexReaders, track what Segment files they're pointing to, and
in

onCommit don't delete those segments.

This implies you have multiple readers in a single JVM?  If so,
you
should not need to make a custom deletion policy to handle this
case

-- the OS should be properly protecting open files from deletion.

Can you  shed more light on the bigger picture here?

Unfortunately it ends up being very difficult to directly
determine
what Segment an IndexReader is pointing to.  Is there some
straightforward way that I'm missing -- all I've managed to do so
far is to remember the most recent one from onCommit/onInit and
use

that one....that works OK, but makes bootstrapping a pain if you
try to open a Reader before you've opened the writer once.

Also, when I use IndexReader.reopen(), can I assume that the
newly

returned reader is pointing at the "most recent" segment?  I
think

so...

Yes, except you have a synchronization challenge: if the writer is
in

the process of committing just as your reader opens you can't be
certain whether the reader got the new commit or the previous one.
If you have external synchronization to ensure reader only
re-opens
after writer has fully committed then this isn't an issue.


Here's a sequence of steps that happen in my app:

0) open writer, onInit tells me that seg_1 is the most recent
segment

1) Open reader, assume it is pointing to seg_1 from (0)

2) New write commits into seg_2, don't delete seg_1 b/c of (1)

3) Call reader.reopen() on the reader from (1)....new reader is
pointing to seg_2 now?

4) seg_1 stays around until the next time I open or commit a
writer, then it is removed.


Does that seem reasonable?


--tim









---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscri



---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]


---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]



---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]

Reply via email to