Multiple threads in Lucene

[Nikhil Goel]

Hi Lucene Developers,

According to Lucene Documentation, IndexWriter can exist with multiple
IndexSearcher and its thread safe. To verify that: I wrote a simple program
to simulate that condition but unfortunately I get an exception. Please let
me know if anyone has ever tested the Lucene claim that IndexWriter and
IndexSearcher are thread safe.


I have a program in which I have 4 threads.
1) One IndexWriter Thread
2) 3 IndexSearcher Thread.

Everytime when we need to index a file. We run the following code in
IndexWriter Thread:-
function IndexFile(Document doc)
{
   writer = new IndexWriter(directory, new StandardAnalyzer(), false);
   writer.addDocument(doc);
   writer.close();
}

Our IndexSearcherThread looks like this:-
function IndexSearch(String termToBeSearched)
{
     IndexSearchersearcher = new IndexSearcher(directory);
//Note: This directory is the same reference as used to initiate
IndexWriter  in Indexfile function. Hence this directory //reference is used
across all the threads.

     Query query = QueryParser.parse(termToBeSearched, "contents", new
StandardAnalyzer());
     Hits hits = searcher.search(query);
}

If I execute these 4 threads above together, then whenever a search routine
gets executed and IndexWriter is also in use, then I get an error at the
following line:- writer.close();

Stack Strace looks like this:-
unable to close the writer stream
java.io.IOException: read past EOF
    at org.apache.lucene.store.InputStream.refill(InputStream.java:192)
    at org.apache.lucene.store.InputStream.readByte(InputStream.java:81)
    at org.apache.lucene.store.InputStream.readBytes(InputStream.java:95)
    at org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:375)
    at org.apache.lucene.index.SegmentReader.norms(SegmentReader.java:342)
    at org.apache.lucene.index.SegmentMerger.mergeNorms(SegmentMerger.java
:306)
    at org.apache.lucene.index.SegmentMerger.merge(SegmentMerger.java:99)
    at org.apache.lucene.index.IndexWriter.mergeSegments(IndexWriter.java
:430)
    at org.apache.lucene.index.IndexWriter.flushRamSegments(IndexWriter.java
:383)
    at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:193)


Thanks in advance
-Nikhil