A common error I see is that people assume the IndexWriter to be not threadsafe, and open several different instances. You should use just one IndexWriter, keep it open and flush periodically (not commit at each add operation), and read the Lucene wiki pages about the IndexWriter settings like ramBufferSize. That why there's only one lock, no contention from different threads.
There's an explanation of the fastest design I could get here: http://in.relation.to/Bloggers/HibernateSearch32FastIndexRebuild It's describing the procedure used by Hibernate Search for rebuilding the Lucene index from an Hibernate mapped database. While I recommend reading for newcomers, I'd also appreciate feedback and comments from Lucene experts and developers :-) Regards, Sanne 2010/1/14 Michael McCandless <luc...@mikemccandless.com>: > Calling commit after every addition will drastically slow down your > indexing throughput, and concurrency (commit is internally > synchronized), but should not create lock timeouts, unless you are > also opening a new IndexWriter for every addition? > > Mike > > On Thu, Jan 14, 2010 at 12:15 PM, jchang <jchangkihat...@gmail.com> wrote: >> >> With only 10 concurrent consumers, I do get lock problems. However, I am >> calling commit() at the end of each addition. Could I expect better >> concurrency without timeouts if I did not commit as often? >> >> -- >> View this message in context: >> http://old.nabble.com/Lucene-2.9.0-Near-Real-Time-Indexing-and-lock-timeouts-tp27136743p27164797.html >> Sent from the Lucene - Java Developer mailing list archive at Nabble.com. >> >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: java-dev-h...@lucene.apache.org >> >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-dev-h...@lucene.apache.org > > --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
