RE: Index Locked For Write
I am not a Lucene expert but I would like to understand the threading issues
also, and I'm wondering if the following is true when using Lucene in a
multithreaded application.
I understand there are three modes for using IndexReader and IndexWriter:
A- IndexReader for reading only, not deleting
B- IndexReader for deleting (and reading)
C- IndexWriter (for adding and optimizing)
Any number of readers may be used concurrently in mode A. But for B and C
the reader or writer may not be kept open for long periods. Write
operations create a lock, and closing the reader or writer is the only way
to release the lock. In theory a single writer could be kept open, but its
lock will prevent deletions (which are performed with a separate reader).
Therefore for B and C each set of changes should be made inside a
synchronized block where the reader or writer is opened and closed. This
prevents multiple writers (or readers used for deleting) from being open at
once. The synchronization should be done on an object that identifies a
particular index, e.g., on a global object if there is only one index. For
example:
class myindex {
static final Object INDEX_LOCK = new Object();
void delete(int[] docs) {
synchronized (INDEX_LOCK) {
IndexReader reader = IndexReader.open(...);
try {
for (int i = 0; i docs.length; i++) {
reader.delete(docs[i]);
}
} finally {
reader.close();
}
}
}
void add(Document[] docs) {
synchronized (INDEX_LOCK) {
IndexWriter writer = new IndexWriter(...);
try {
for (int i = 0; i docs.length; i++) {
writer.add(docs[i]);
}
writer.optimize();
} finally {
writer.close();
}
}
}
}
Of course there are other techniques for global locking such as 'static
synchronized' methods. Locking on a separate object per index is the
general case (where multiple indexes are present).
Is this correct? Or should Lucene be waiting on the write lock instead of
throwing an exception?
mark
-Original Message-
From: Otis Gospodnetic [mailto:[EMAIL PROTECTED]]
Sent: Sunday, February 24, 2002 9:22 PM
To: Lucene Users List
Subject: RE: Index Locked For Write
--- Howk, Michael [EMAIL PROTECTED] wrote:
Out of curiosity, why didn't we need to close the writer in rc2 or
rc3?
When you suggest a synchronized keyword, are you suggesting that
the
writer is not inherently thread-safe? Do we need to write our own
thread
management on top of Lucene?
Sorry, that might have been a wrong suggestion, IndexWriter (at least
the add method) is supposed to be thread safe.
Otis
-Original Message-
From: Otis Gospodnetic [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 21, 2002 4:07 PM
To: Lucene Users List
Subject: RE: Index Locked For Write
You could use synchronized keyword and use IndexReader.isLocked() or
something like that, no?
Otis
--- Howk, Michael [EMAIL PROTECTED] wrote:
Thank you for your quick responses. But in our application, we're
working in
a transactional environment where multiple threads are accessing a
single
writer using the recommended singleton pattern. Since no
thread has
exclusive access to the writer, how can we have one thread
arbitrarily
decide to close the writer?
Michael
-Original Message-
From: Mark Tucker [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 21, 2002 3:51 PM
To: Lucene Users List
Subject: RE: Index Locked For Write
You forgot to close your writer after the call to optimize.
-Original Message-
From: Howk, Michael [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 21, 2002 2:49 PM
To: Lucene Mailing List (E-mail)
Subject: Index Locked For Write
We just got the newest daily build (to try to fix some NullPointer
errors
with ? and _ characters), and we're getting the same problem
that
Daniel
Calvo mentioned: Index Locked for Write. Here's basically what our
code is
doing:
IndexWriter writer = new IndexWriter(path,
analyzer, create);
try {
Document doc = new Document();
doc.add(Field.Keyword(DOC_ID, 14));
doc.add(Field.UnStored(ANY, mushu));
writer.addDocument(doc);
writer.optimize();
// Search the document for our keyword
{
IndexReader reader = IndexReader.open(path);
IndexSearcher searcher = new IndexSearcher(reader);
Vector returnStuff = searcher.search(mushu);
}
// Verify that we got one record back
assertNotNull(returnStuff);
assertEquals(1, returnStuff.size());
}
finally {
// Clean up after ourselves
IndexReader reader = IndexReader.open(path
RE: Index Locked For Write
--- Howk, Michael [EMAIL PROTECTED] wrote:
Out of curiosity, why didn't we need to close the writer in rc2 or
rc3?
When you suggest a synchronized keyword, are you suggesting that
the
writer is not inherently thread-safe? Do we need to write our own
thread
management on top of Lucene?
Sorry, that might have been a wrong suggestion, IndexWriter (at least
the add method) is supposed to be thread safe.
Otis
-Original Message-
From: Otis Gospodnetic [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 21, 2002 4:07 PM
To: Lucene Users List
Subject: RE: Index Locked For Write
You could use synchronized keyword and use IndexReader.isLocked() or
something like that, no?
Otis
--- Howk, Michael [EMAIL PROTECTED] wrote:
Thank you for your quick responses. But in our application, we're
working in
a transactional environment where multiple threads are accessing a
single
writer using the recommended singleton pattern. Since no thread has
exclusive access to the writer, how can we have one thread
arbitrarily
decide to close the writer?
Michael
-Original Message-
From: Mark Tucker [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 21, 2002 3:51 PM
To: Lucene Users List
Subject: RE: Index Locked For Write
You forgot to close your writer after the call to optimize.
-Original Message-
From: Howk, Michael [mailto:[EMAIL PROTECTED]]
Sent: Thursday, February 21, 2002 2:49 PM
To: Lucene Mailing List (E-mail)
Subject: Index Locked For Write
We just got the newest daily build (to try to fix some NullPointer
errors
with ? and _ characters), and we're getting the same problem
that
Daniel
Calvo mentioned: Index Locked for Write. Here's basically what our
code is
doing:
IndexWriter writer = new IndexWriter(path, analyzer, create);
try {
Document doc = new Document();
doc.add(Field.Keyword(DOC_ID, 14));
doc.add(Field.UnStored(ANY, mushu));
writer.addDocument(doc);
writer.optimize();
// Search the document for our keyword
{
IndexReader reader = IndexReader.open(path);
IndexSearcher searcher = new IndexSearcher(reader);
Vector returnStuff = searcher.search(mushu);
}
// Verify that we got one record back
assertNotNull(returnStuff);
assertEquals(1, returnStuff.size());
}
finally {
// Clean up after ourselves
IndexReader reader = IndexReader.open(path);
reader.delete(new Term(DOC_ID, 14));
reader.close();
}
And the exception we're getting on the reader.delete line in the
finally
clause:
java.io.IOException: Index locked for write:
Lock@C:\devtools\JBossTomcat\jboss\indexes\marc\write.lock at
sun.rmi.transport.StreamRemoteCall.exceptionReceivedFromServer(StreamRemoteC
all.java:245) at
sun.rmi.transport.StreamRemoteCall.executeCall(StreamRemoteCall.java:220)
at
sun.rmi.server.UnicastRef.invoke(UnicastRef.java:122) at
org.jboss.ejb.plugins.jrmp.server.JRMPContainerInvoker_Stub.invoke(Unknown
Source) at
org.jboss.ejb.plugins.jrmp.interfaces.GenericProxy.invokeContainer(GenericPr
oxy.java:357) at
org.jboss.ejb.plugins.jrmp.interfaces.StatelessSessionProxy.invoke(Stateless
SessionProxy.java:123) at
$Proxy5.deleteDocument(Unknown Source)
Are we using the right approach? Any suggestions? Thank you.
Michael Howk
--
To unsubscribe, e-mail:
mailto:[EMAIL PROTECTED]
For additional commands, e-mail:
mailto:[EMAIL PROTECTED]
--
To unsubscribe, e-mail:
mailto:[EMAIL PROTECTED]
For additional commands, e-mail:
mailto:[EMAIL PROTECTED]
--
To unsubscribe, e-mail:
mailto:[EMAIL PROTECTED]
For additional commands, e-mail:
mailto:[EMAIL PROTECTED]
__
Do You Yahoo!?
Yahoo! Sports - Coverage of the 2002 Olympic Games
http://sports.yahoo.com
--
To unsubscribe, e-mail:
mailto:[EMAIL PROTECTED]
For additional commands, e-mail:
mailto:[EMAIL PROTECTED]
--
To unsubscribe, e-mail:
mailto:[EMAIL PROTECTED]
For additional commands, e-mail:
mailto:[EMAIL PROTECTED]
__
Do You Yahoo!?
Yahoo! Sports - Coverage of the 2002 Olympic Games
http://sports.yahoo.com
--
To unsubscribe, e-mail: mailto:[EMAIL PROTECTED]
For additional commands, e-mail: mailto:[EMAIL PROTECTED]
