The concurrency problem you describe doesn't occur merely because an IndexReader is open when an IndexWriter modifies the index; rather, an IndexReader that has modified the index (presumably to delete documents) is open and then an IndexWriter attempts to modify the index. That is a violation of Lucene's few concurrency rules, and is easily avoided.
All index modifying operations should be such that for a given index, there should only ever be either an IndexReader or IndexWriter having performed index modifications open. Simply put, open an IndexReader, delete documents, close it; open an IndexWriter, insert documents, close it. This doesn't apply to an IndexReader used for solely searching, since searches do not involve modifying the index. When caching an instance of IndexReader for searching, consider using IndexReader.Reopen to ensure searches are in sync with the dynamic index. -----Original Message----- From: Jeff Pennal [mailto:[email protected]] Sent: Friday, August 07, 2009 6:17 PM To: [email protected] Subject: Thread safe lucene index writing We are currently using Lucene.Net 2.3.1 to power the search engine of our web application. Our architecture works like this: - The web application queries the index for searches using the IndexReader/IndexSearcher. - Whenever content is updated by a user of the web app, the database is updated and a message is put into a queue indicating that the content needs to be updated in the lucene index - A windows service is running, listening to the queue and when a message comes in, will update the lucene index using an IndexWriter. The main objective for us here is to keep the database and the index in sync. We need to be able to trust that when content is updated and saved to the database, the document in the index that represents that content will be updated asap. We have run into issues in the past where the IndexWriter cannot update the index because there is a lock on the file. The usual cause of this is that the IndexReader has the index open when the writer tries to write to the index. Our current workaround for this is when we do updates to the index, we do the updates in a temporary folder so we wont have to worry about locks during the updating process. Once the update is done, we replace the existing index with the newly updated one. While this does work now, it is not the most robust solution and will cause us trouble scaling the web app since we want to be able to use the lucene index in other areas than search. Does anyone run Lucene.Net in a similar setup? Have you any ideas on how this configuration can be made to be more reliable when it comes to writing to the index? Thanks in advance, Jeff
