Re: Index.close() infinite TIME_WAITING

2009-10-18 Thread Michael McCandless
Jamie did you ever get to the bottom of this? Can you reduce this code down to a smaller example that shows the hang? Also, can you post a thread stack dump when you hit the hang? Is it possible you are adding documents from one thread while calling IndexWriter.close in another? I see you have

Re: Index.close() infinite TIME_WAITING (repost)

2009-10-09 Thread Michael McCandless
Were there any exceptions inside Lucene, before the hang? The fact that you're hitting AlreadyClosedException is a spooky sign -- that means IW thinks you had in fact closed the writer, but then used it again. For increasing indexing throughput, I'd start here: http://wiki.apache.org/lucene-

Re: Index.close() infinite TIME_WAITING (repost)

2009-10-09 Thread Jamie Band
Hi Mike There are other threads involved but none are simultaneously modifying the index. There is one thread that retrieves the total count every 2 seconds on the index for GUI display: public long getTotalMessageCount(Volume volume) throws MessageSearchException { if (volum

Re: Index.close() infinite TIME_WAITING (repost)

2009-10-09 Thread Jamie Band
Hi Mike There are other threads involved but none are simultaneously modifying the idex. There is one read that retrieves the total count every 2 seconds on the Index for GUI display: public long getTotalMessageCount(Volume volume) throws MessageSearchException { if (volume =

Re: Index.close() infinite TIME_WAITING (repost)

2009-10-09 Thread Michael McCandless
Are there other threads involved, besides the one hung in close? Can you post their stack traces? This stack trace seems to indicate that IW believes another thread is in the process of closing. Can you call IndexWriter.setInfoStream and post the output leading to the hang? Mike On Fri, Oct 9,

Re: Index.close() infinite TIME_WAITING (repost)

2009-10-09 Thread Jamie Band
HI Michael / Uwe / others Sorry for the repost... it just does not look like the earlier message I sent go through. FYI: there are no large Lucene merges taking place. Jamie Band wrote: Hi Michael Thanks for your help. Here are the stacks: index processor [TIME_WAITING] CPU time: 33:01 java

Re: Index.close() infinite TIME_WAITING

2009-10-09 Thread Jamie Band
Incidentally, there are no Lucene merge threads doing any work. See attached. - To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org For additional commands, e-mail: java-user-h...@lucene.apache.org

Re: Index.close() infinite TIME_WAITING

2009-10-09 Thread Jamie Band
Hi Michael Thanks for your help. Here are the stacks: index processor [TIME_WAITING] CPU time: 33:01 java.lang.Object.wait(long) org.apache.lucene.index.IndexWriter.doWait() org.apache.lucene.index.IndexWriter.shouldClose() org.apache.lucene.index.IndexWriter.close(boolean) org.apache.lucene.ind

Re: Index.close() infinite TIME_WAITING

2009-10-08 Thread Michael McCandless
Is it possible a large merge is running? By default IW.close waits for outstanding merges to complete. Can you post the stacktrace? Mike On Thu, Oct 8, 2009 at 5:22 PM, Jamie Band wrote: > Hi All > > I have a long running situation where our indexing thread is getting stuck > indefinitely in I

RE: Index.close() infinite TIME_WAITING

2009-10-08 Thread Uwe Schindler
Did you do some extra locking around IndexWriter using the IndexWriter itsself as mutex (e.g. synchronized(writer) {...}). This is not supported and hangs. IndexWriter itself is thread-safe. Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.d