Hi Tobias, Unfortunately I don't have an isolated test case, as I was doing a fairly involved system test at the time. I may be able to have a colleague work on reproducing it at a later date (I've been diverted to something else for the moment).
I was remote debugging with Eclipse, so I toggled a method breakpoint on Thread.interrupt() and then inspected the stack once the breakpoint was hit. Sorry I don't have more information at the moment. I agree that eliminating the interrupts sounds like the best approach, if possible. Thanks, Jennifer ________________________________________ From: [email protected] [[email protected]] On Behalf Of Tobias Ivarsson [[email protected]] Sent: Thursday, April 28, 2011 6:23 AM To: Neo4j user discussions Subject: Re: [Neo4j] ClosedChannelExceptions in highly concurrent environment Hi Jennifer, I'd first like to thank you for the testing and analysis you've done. Very useful stuff. Do you think you could send some test code our way that reproduces this issue? This is actually the first time this issue has been reported, so I wouldn't say it is a common issue. My guess is that your thread volume triggered a rare condition that wouldn't be encountered otherwise. I'm also curious to know how you found the source of the interruptions. When I debug thread interruptions I've never been able to find out where the thread got interrupted from without doing tedious procedures of breakpoint + logging + trying to match thread ids. If you have a better method for doing that I'd very much like to know. I think we should focus the effort on fixing the interruption issue if we can. And I believe we would be able to do that if the interruptions do in fact originate from where you say they do. But the suggestion of being able to switch the lucene directory implementation is still interesting, but as you point out since it has issues on some platforms it would be better if we could be rid of the interruption issue. Cheers, Tobias On Thu, Apr 28, 2011 at 12:41 AM, Jennifer Hickey <[email protected]>wrote: > Hello, > I've been running some tests w/approx 400 threads reading various indexed > property values. I'm running on 64 bit Linux. I was frequently seeing the > ClosedChannelException below. The javadoc on Lucene's NIOFSDirectory states > that "Accessing this class either directly or indirectly from a thread while > it's interrupted can close the underlying file descriptor immediately if at > the same time the thread is blocked on IO. The file descriptor will remain > closed and subsequent access to {@link NIOFSDirectory} will throw a {@link > ClosedChannelException}. If your application uses either {@link > Thread#interrupt()} or {@link Future#cancel(boolean)} you should use {@link > SimpleFSDirectory} in favor of {@link NIOFSDirectory}." > > A bit of debugging revealed that the Thread.interrupts were coming from > Neo4j, specifically in RWLock and MappedPersistenceWindow. So it seems like > this would be a common problem, though perhaps I am missing something? > > SimpleFSDirectory seems a bit of a performance bottleneck, so I switched to > MMapDirectory and the problem did go away. I didn't see a way to switch > implementations w/out modifying neo4j code, so I changed LuceneDataSource as > follows: > > static Directory getDirectory( String storeDir, > IndexIdentifier identifier ) throws IOException > { > MMapDirectory dir=new MMapDirectory(getFileDirectory( storeDir, > identifier), null); > if(MMapDirectory.UNMAP_SUPPORTED) { > dir.setUseUnmap(true); > } > return dir; > } > > So I'm wondering if others have seen this problem and/or if there is a > recommended solution? Our product runs on quite a few different operating > systems, so I have some reservations about using MMapDirectory as well > (javadoc speaks of a few caveats on Windows, 64 vs 32, etc). Also, I'd > rather not maintain a patched version of the neo4j code if avoidable. > > Thanks! > Jennifer > > Exception: > Caused by: java.nio.channels.ClosedChannelException > at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:88) > at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:613) > at > org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.readInternal(NIOFSDirectory.java:161) > at > org.apache.lucene.store.BufferedIndexInput.readBytes(BufferedIndexInput.java:139) > at > org.apache.lucene.index.CompoundFileReader$CSIndexInput.readInternal(CompoundFileReader.java:285) > at > org.apache.lucene.store.BufferedIndexInput.refill(BufferedIndexInput.java:160) > at > org.apache.lucene.store.BufferedIndexInput.readByte(BufferedIndexInput.java:39) > at org.apache.lucene.store.DataInput.readVInt(DataInput.java:86) > at > org.apache.lucene.index.codecs.DeltaBytesReader.read(DeltaBytesReader.java:40) > at > org.apache.lucene.index.codecs.PrefixCodedTermsReader$FieldReader$SegmentTermsEnum.next(PrefixCodedTermsReader.java:469) > at > org.apache.lucene.index.codecs.PrefixCodedTermsReader$FieldReader$SegmentTermsEnum.seek(PrefixCodedTermsReader.java:385) > at org.apache.lucene.index.TermsEnum.seek(TermsEnum.java:68) > at org.apache.lucene.index.Terms.docFreq(Terms.java:53) > at org.apache.lucene.index.SegmentReader.docFreq(SegmentReader.java:898) > at org.apache.lucene.index.IndexReader.docFreq(IndexReader.java:882) > at > org.apache.lucene.index.DirectoryReader.docFreq(DirectoryReader.java:687) > > > _______________________________________________ > Neo4j mailing list > [email protected] > https://lists.neo4j.org/mailman/listinfo/user > -- Tobias Ivarsson <[email protected]> Hacker, Neo Technology www.neotechnology.com Cellphone: +46 706 534857 _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
