Looks like, I can never run into that situation. Another doc-id would not even be assigned before flushing out the current doc.
________________________________ From: Jagadesh Nomula [mailto:[EMAIL PROTECTED] Sent: Tuesday, July 29, 2008 2:43 PM To: java-dev@lucene.apache.org Subject: RE: Deadlock when multi-threading DocumentsWriter Hi Mike, Thanks for the comments. Diagnosing, the stack trace the following statements, might run into a nested lock. org.apache.lucene.index.DocumentsWriter.pauseAllThreads(DocumentsWriter.java:507) org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:2670) The doFlush method is synchronized on IndexWriter, and DocumentsWriter method is synchronized on DocumentsWriter. This thread is giving away the lock on DocumentsWriter but still holds a lock on IndexWriter. If documents come to flush state out of order(eg: doc3 getting flushed before doc2) then don't we run into deadlock ?. This is not happening, so I should be missing something. Any Comments ? Thanks, Jagdish -----Original Message----- From: Michael McCandless [mailto:[EMAIL PROTECTED] Sent: Monday, July 28, 2008 4:33 PM To: java-dev@lucene.apache.org Subject: Re: Deadlock when multi-threading DocumentsWriter Can you post a patch with your full changes to DocumentsWriter and IndexWriter? That first thread is trying to flush, but is waiting for all threads to leave DocumentsWriter (finish adding docs). The 2nd thread looks like it's waiting for the flush to finish before proceeding. Are there any other threads? Are you calling DocumentsWriter.finishDocument? That method frees the thread state, which is what that first thread is waiting on... Mike Jagadesh Nomula wrote: > Would anyone be having any insight into deadlock issues, when > running DocumentsWriter.java from multiple threads ?. I am trying to > port ParallelWriter.java code to new codebase of > DocumentsWriter.java and IndexWriter. I am doing this by splitting, > DocumentsWriter.addDocument call into two methods unsynchronized > methods, doGetThreadState and finishDocWithThreadState. > doGetThreadState just calls the synchronized getThreadState method > and returns a thread state to be used by finishDocWithThreadState, > which inverts the document and flushes it. The code base is > semantically equivalent to addDocument method in DocumentsWriter, > the only variation being, call to doGetThreadState executed from a > synched block in ParallelWriter to maintain the consistency of same > doc-ids in parallelWriter. > > You would imagine that, this code would work without any issues, but > it runs into a deadlock. The excerpt of suspicious calls is: > > == Thread ConnectionThreadGroup-26491.pool-8-thread-1 ===> > java.lang.Object.wait(Native Method) > java.lang.Object.wait(Object.java:485) > > org > .apache > .lucene.index.DocumentsWriter.pauseAllThreads(DocumentsWriter.java: > 507) > org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java: > 2670) > org.apache.lucene.index.IndexWriter.flush(IndexWriter.java: > 2660) > > org.apache.lucene.index.IndexWriter.finishDoc(IndexWriter.java:1601) > org.apache.lucene.index.ParallelWriter > $ProcessWorker.run(ParallelWriter.java:464) > java.util.concurrent.ThreadPoolExecutor > $Worker.runTask(ThreadPoolExecutor.java:885) > java.util.concurrent.ThreadPoolExecutor > $Worker.run(ThreadPoolExecutor.java:907) > java.lang.Thread.run(Thread.java:619) > > > ======================================= > == Thread ConnectionThreadGroup-26491.pool-3-thread-6 ===> > java.lang.Object.wait(Native Method) > java.lang.Object.wait(Object.java:485) > > org > .apache > .lucene.index.DocumentsWriter.getThreadState(DocumentsWriter.java: > 2420) > > org > .apache > .lucene.index.DocumentsWriter.doGetThreadState(DocumentsWriter.java: > 2532) > > org.apache.lucene.index.IndexWriter.getThreadState(IndexWriter.java: > 1564) > org.apache.lucene.index.ParallelWriter > $ThreadStateWorker.call(ParallelWriter.java:425) > org.apache.lucene.index.ParallelWriter > $ThreadStateWorker.call(ParallelWriter.java:405) > java.util.concurrent.FutureTask > $Sync.innerRun(FutureTask.java:303) > java.util.concurrent.FutureTask.run(FutureTask.java:138) > java.util.concurrent.ThreadPoolExecutor > $Worker.runTask(ThreadPoolExecutor.java:885) > java.util.concurrent.ThreadPoolExecutor > $Worker.run(ThreadPoolExecutor.java:907) > java.lang.Thread.run(Thread.java:619) > > Any info, that I might be overlooking or any comments would be of > great help to me in resolving this. Thanks in advance for your help. > > Jagdish > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
