Hi,
I am encountering the following issue with Lucene 3.6.1 if you could let me
know if I am doing something wrong / there is a mistake I am making it
would be great.
In order to improve the performance of the application that I am working at
I went to the approach of reusing the IndexReader and reopening it every 8
hours in order to get the latest changes. (IndexReader is declared as a
global static variable). The search method is called from multiple threads
in parallel so the index reader is shared between threads. Now if I don't
close the old index reader I am noticing increases of virtual memory with
every new reindex reopen (this should not be an issue on 64 bit Linux
correct - this is the configuration I am using and the indexes are on a
shared mount NTFS file system ).
Also from time to times I noticed JVM crasches with the following stack:
Thread Stack Trace:
at memcpy+160()@0x381aa7b060
-- Java stack --
at java/nio/DirectByteBuffer.get(DirectByteBuffer.java:294)[optimized]
at
org/apache/lucene/store/MMapDirectory$MMapIndexInput.readBytes(MMapDirectory.java:298)[optimized]
at org/apache/lucene/store/DataInput.readBytes(DataInput.java:72)
at
org/apache/lucene/index/CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:275)[optimized]
at
org/apache/lucene/store/BufferedIndexInput.refill(BufferedIndexInput.java:270)[optimized]
at
org/apache/lucene/store/BufferedIndexInput.readByte(BufferedIndexInput.java:40)[inlined]
at
org/apache/lucene/store/DataInput.readVInt(DataInput.java:107)[inlined]
at
org/apache/lucene/store/BufferedIndexInput.readVInt(BufferedIndexInput.java:217)[optimized]
at org/apache/lucene/index/FieldsReader.doc(FieldsReader.java:235)
at
org/apache/lucene/index/SegmentReader.document(SegmentReader.java:492)
at
org/apache/lucene/index/DirectoryReader.document(DirectoryReader.java:568)
at org/apache/lucene/index/MultiReader.document(MultiReader.java:252)
at org/apache/lucene/index/IndexReader.document(IndexReader.java:1138)
at
org/apache/lucene/search/IndexSearcher.doc(IndexSearcher.java:258)[inlined]
Can you please tell me if all this corruption is caused by the fact that I
am not closing the old IndexReader. But if I close if given that it is
share by multiple threads I will need to check each time before doing the
search if IndexReader is still open correct? Let's say in a thread I am
reopening the IndexReader and in another thread I am afterwards reusing the
old one in that case I should do the check correct? Or is there a smarter
mechanism in place.
Any help with this would be more than welcome.
Thank you very much and best regards,
Liviu
