Re: Locks and Readers and Writers
CORRECTION: Ok, I simplified my test case a little... I removed the UNC share which is a remote file share that Windows supports. I have two processes running on the same Windows XP machine. 1) The first process uses an IndexWriter to writes to the index and then closes the IndexWriter. 2) The first process sends a multicast event to the other process. 3) The other process receives the multicast event opens an IndexReader. 4) Meanwhile, the first process opens an IndexReader and performs a delete. The IndexReader is then closed. 5) The first process opens an IndexWriter to add more documents. At this point the IndexWriter often gets an error: Lock obtain timed out Any advice? Is this only a problem on Windows? Here's the stack trace: java.io.IOException: Lock obtain timed out: [EMAIL PROTECTED]:\WINDOWS\TEMP\lucene-9906f192fb41cc8775cf839213997b7e-commit.lock at org.apache.lucene.store.Lock.obtain(Lock.java:58) at org.apache.lucene.store.Lock$With.run(Lock.java:108) at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:501) at org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:458) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:310) --- Lucene Users List" <[EMAIL PROTECTED] wrote: Ok, I simplified my test case a little... I removed the UNC share which is > a remote file share that Windows supports. > > I have two processes running > on the same Windows XP machine. > 1) The first process uses an IndexWriter > to writes to the index and then closes the IndexWriter. > 2) The first process > sends a multicast event to the other process. > 3) The other process opens > an IndexReader. > 4) Meanwhile, the first process tries to open the index again > to do a write. > 5) Often the IndexWriter gets an error: Lock obtain timed > out > > Any advice? Is this only a problem on Windows? > > Here's the stack > trace: > java.io.IOException: Lock obtain timed out: [EMAIL PROTECTED]:\WINDOWS\TEMP\lucene-9906f192fb41cc8775cf839213997b7e-commit.lock > > at org.apache.lucene.store.Lock.obtain(Lock.java:58) > at org.apache.lucene.store.Lock$With.run(Lock.java:108) > > at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:501) > > at org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:458) > > at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:310) > > > --- Lucene Users List" <[EMAIL PROTECTED] wrote: > [EMAIL PROTECTED] > writes: > > > Hi Christoph, > > > > > > Thats what I thought. But what I'm > seeing is this: > > > - open > > > reader for searching > > > (the reader is > opening an index on a remote machine > > > (via UNC) which takes a couple seconds) > > > > - meanwhile the other service opens > > > an IndexWriter and adds a document > > > > (the index writer determines that it needs > > > to merge so it tries > to get a lock. since the reader is still opening, the > > > IO exception is > thrown) > > > > > > I believe that increasing the merge factor will > > > reduce > the opportunity for this to occur. But it will still occur at some > > > point. > > > > > > I'm not sure what you mean by `opening an index on a remote machine > (via > > UNC)' but have you made sure that lock files are put in the same > directory > > for both processes (see the mailing list archive for details)? > > > Also note, that lucene's locking is known not to work on NFS (also see > the > > list archive). I don't know if it works on SMB mounts. > > > > Morus > > > > > - > > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For > additional commands, e-mail: [EMAIL PROTECTED] > > > > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Locks and Readers and Writers
Ok, I simplified my test case a little... I removed the UNC share which is a remote file share that Windows supports. I have two processes running on the same Windows XP machine. 1) The first process uses an IndexWriter to writes to the index and then closes the IndexWriter. 2) The first process sends a multicast event to the other process. 3) The other process opens an IndexReader. 4) Meanwhile, the first process tries to open the index again to do a write. 5) Often the IndexWriter gets an error: Lock obtain timed out Any advice? Is this only a problem on Windows? Here's the stack trace: java.io.IOException: Lock obtain timed out: [EMAIL PROTECTED]:\WINDOWS\TEMP\lucene-9906f192fb41cc8775cf839213997b7e-commit.lock at org.apache.lucene.store.Lock.obtain(Lock.java:58) at org.apache.lucene.store.Lock$With.run(Lock.java:108) at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java:501) at org.apache.lucene.index.IndexWriter.maybeMergeSegments(IndexWriter.java:458) at org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:310) --- Lucene Users List" <[EMAIL PROTECTED] wrote: [EMAIL PROTECTED] writes: > > Hi Christoph, > > > > Thats what I thought. But what I'm seeing is this: > > - open > > reader for searching > > (the reader is opening an index on a remote machine > > (via UNC) which takes a couple seconds) > > - meanwhile the other service opens > > an IndexWriter and adds a document > > (the index writer determines that it needs > > to merge so it tries to get a lock. since the reader is still opening, the > > IO exception is thrown) > > > > I believe that increasing the merge factor will > > reduce the opportunity for this to occur. But it will still occur at some > > point. > > > I'm not sure what you mean by `opening an index on a remote machine (via > UNC)' but have you made sure that lock files are put in the same directory > for both processes (see the mailing list archive for details)? > Also note, that lucene's locking is known not to work on NFS (also see the > list archive). I don't know if it works on SMB mounts. > > Morus > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Locks and Readers and Writers
[EMAIL PROTECTED] writes: > Hi Christoph, > > Thats what I thought. But what I'm seeing is this: > - open > reader for searching > (the reader is opening an index on a remote machine > (via UNC) which takes a couple seconds) > - meanwhile the other service opens > an IndexWriter and adds a document > (the index writer determines that it needs > to merge so it tries to get a lock. since the reader is still opening, the > IO exception is thrown) > > I believe that increasing the merge factor will > reduce the opportunity for this to occur. But it will still occur at some > point. > I'm not sure what you mean by `opening an index on a remote machine (via UNC)' but have you made sure that lock files are put in the same directory for both processes (see the mailing list archive for details)? Also note, that lucene's locking is known not to work on NFS (also see the list archive). I don't know if it works on SMB mounts. Morus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Locks and Readers and Writers
Hi Christoph, Thats what I thought. But what I'm seeing is this: - open reader for searching (the reader is opening an index on a remote machine (via UNC) which takes a couple seconds) - meanwhile the other service opens an IndexWriter and adds a document (the index writer determines that it needs to merge so it tries to get a lock. since the reader is still opening, the IO exception is thrown) I believe that increasing the merge factor will reduce the opportunity for this to occur. But it will still occur at some point. Thanks. --- Lucene Users List" <[EMAIL PROTECTED] wrote: Morus Walter wrote: > > >>AFAIK you should never open an IndexWriter and an IndexReader at the > >>same time. You should use only one of them at a time but you may open as > >>many IndexSearchers as you like for searching. > > > > You cannot open an IndexSearcher without opening an IndexReader (explicitly > > or implicitly). > > I should have been more specific. The IndexReader which gets opened by > an IndexSearcher will never have any changes to commit. So there will > never be a problem with locks. As soon as you use an IndexReader to > modify your index (delete doc for example) you should not have another > IndexReader or IndexWriter open or you will probably get problems with > the locks. > > Christoph > > > - > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Locks and Readers and Writers
Morus Walter wrote: AFAIK you should never open an IndexWriter and an IndexReader at the same time. You should use only one of them at a time but you may open as many IndexSearchers as you like for searching. You cannot open an IndexSearcher without opening an IndexReader (explicitly or implicitly). I should have been more specific. The IndexReader which gets opened by an IndexSearcher will never have any changes to commit. So there will never be a problem with locks. As soon as you use an IndexReader to modify your index (delete doc for example) you should not have another IndexReader or IndexWriter open or you will probably get problems with the locks. Christoph - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Locks and Readers and Writers
Christoph Kiehl writes: > > AFAIK you should never open an IndexWriter and an IndexReader at the > same time. You should use only one of them at a time but you may open as > many IndexSearchers as you like for searching. > You cannot open an IndexSearcher without opening an IndexReader (explicitly or implicitly). Morus - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: Locks and Readers and Writers
[EMAIL PROTECTED] wrote: I'm getting: java.io.IOException: Lock obtain timed out I have a writer service that opens the index to delete and add docs. I have a reader service that opens the index for searching only. AFAIK you should never open an IndexWriter and an IndexReader at the same time. You should use only one of them at a time but you may open as many IndexSearchers as you like for searching. Regards, Christoph - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Locks and Readers and Writers
Hi, I'm getting: java.io.IOException: Lock obtain timed out I have a writer service that opens the index to delete and add docs. I have a reader service that opens the index for searching only. This error occurs when the reader service opens the index (this takes about 500ms). Meanwhile the writer service tries to open it a couple milliseconds later. The reader service hasn't fully opened the index yet and this exception gets thrown. What are my options? Should I just set the timeout to a higher value? Thanks. - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
