Hello,
I've been running some tests w/approx 400 threads reading various indexed
property values. I'm running on 64 bit Linux. I was frequently seeing the
ClosedChannelException below. The javadoc on Lucene's NIOFSDirectory states
that "Accessing this class either directly or indirectly from a thread while
it's interrupted can close the underlying file descriptor immediately if at the
same time the thread is blocked on IO. The file descriptor will remain closed
and subsequent access to {@link NIOFSDirectory} will throw a {@link
ClosedChannelException}. If your application uses either {@link
Thread#interrupt()} or {@link Future#cancel(boolean)} you should use {@link
SimpleFSDirectory} in favor of {@link NIOFSDirectory}."
A bit of debugging revealed that the Thread.interrupts were coming from Neo4j,
specifically in RWLock and MappedPersistenceWindow. So it seems like this
would be a common problem, though perhaps I am missing something?
SimpleFSDirectory seems a bit of a performance bottleneck, so I switched to
MMapDirectory and the problem did go away. I didn't see a way to switch
implementations w/out modifying neo4j code, so I changed LuceneDataSource as
follows:
static Directory getDirectory( String storeDir,
IndexIdentifier identifier ) throws IOException
{
MMapDirectory dir=new MMapDirectory(getFileDirectory( storeDir,
identifier), null);
if(MMapDirectory.UNMAP_SUPPORTED) {
dir.setUseUnmap(true);
}
return dir;
}
So I'm wondering if others have seen this problem and/or if there is a
recommended solution? Our product runs on quite a few different operating
systems, so I have some reservations about using MMapDirectory as well (javadoc
speaks of a few caveats on Windows, 64 vs 32, etc). Also, I'd rather not
maintain a patched version of the neo4j code if avoidable.
Thanks!
Jennifer
Exception:
Caused by: java.nio.channels.ClosedChannelException
at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:613)
at
org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:161)
at
org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:139)
at
org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:285)
at
org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:160)
at
org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39)
at org.apache.lucene.store.DataInput.readVInt(DataInput.java:86)
at
org.apache.lucene.index.codecs.DeltaBytesReader.read(DeltaBytesReader.java:40)
at
org.apache.lucene.index.codecs.PrefixCodedTermsReader$FieldReader$SegmentTermsEnum.next(PrefixCodedTermsReader.java:469)
at
org.apache.lucene.index.codecs.PrefixCodedTermsReader$FieldReader$SegmentTermsEnum.seek(PrefixCodedTermsReader.java:385)
at org.apache.lucene.index.TermsEnum.seek(TermsEnum.java:68)
at org.apache.lucene.index.Terms.docFreq(Terms.java:53)
at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:898)
at org.apache.lucene.index.IndexReader.docFreq(IndexReader.java:882)
at org.apache.lucene.index.DirectoryReader.docFreq(DirectoryReader.java:687)
_______________________________________________
Neo4j mailing list
[email protected]
https://lists.neo4j.org/mailman/listinfo/user
