Mattias Persson wrote: > Ah, do you make sure you call IndexHits#close() on your results from > getNodes(), alternatively iterating over the whole result (in which > case it's automatically closed)? > Actually I'm calling the getSingleNode, thus I have no access on the IndexHits object.
> See more information at > http://components.neo4j.org/neo4j-index/apidocs/index.html > > Are you using the "index" component btw (artifactId: neo4j-index)? > Yes. Is it possible that the problem is caused due to the fact that we perform several writes in both the graph and the index and we don't commit and that we read in between the transaction commits? > 2010/1/26 Symeon (Akis) Papadopoulos <[email protected]>: > >> Hi all >> >> While populating a Neo graph, I got the attached exception >> (org.apache.lucene.index.MergePolicy$MergeException. See file for details: I >> replaced a local path with [some-local-path]). >> My setup is like this: I want to benchmark Neo4j for some operations, >> amongst which is graph loading. >> So, I try to load graphs of various sizes to Neo4j. Up to size 1M edges the >> graphs are loaded without any problem, but then this exception is thrown. I >> suspect this has to do with the transaction management in my program, which >> is handled by a BatchTxManager (a class written by me, thus highly likely to >> be the source of trouble). Does the exception ring any bells? What could I >> try out in order to identify the problem? >> >> Best regards, >> Akis >> >> >> >> Exception in thread "Lucene Merge Thread #0" >> org.apache.lucene.index.MergePolicy$MergeException: >> java.io.FileNotFoundException: /[some-local-path]/_4k.cfs (Too many open >> files) >> at >> org.apache.lucene.index.ConcurrentMergeScheduler.handleMergeException(ConcurrentMergeScheduler.java:351) >> at >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:315) >> Caused by: java.io.FileNotFoundException: /[some-local-path]/_4k.cfs (Too >> many open files) >> at java.io.RandomAccessFile.open(Native Method) >> at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233) >> at >> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput$Descriptor.<init>(SimpleFSDirectory.java:78) >> at >> org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexInput.<init>(SimpleFSDirectory.java:108) >> at >> org.apache.lucene.store.NIOFSDirectory$NIOFSIndexInput.<init>(NIOFSDirectory.java:94) >> at >> org.apache.lucene.store.NIOFSDirectory.openInput(NIOFSDirectory.java:70) >> at >> org.apache.lucene.index.CompoundFileReader.<init>(CompoundFileReader.java:70) >> at >> org.apache.lucene.index.SegmentReader$CoreReaders.<init>(SegmentReader.java:111) >> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:638) >> at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:608) >> at >> org.apache.lucene.index.IndexWriter$ReaderPool.get(IndexWriter.java:686) >> at >> org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4979) >> at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4614) >> at >> org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:235) >> at >> org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:291) >> Exception in thread "main" >> org.neo4j.kernel.impl.transaction.TransactionFailureException: Unable to >> commit transaction >> at >> org.neo4j.kernel.EmbeddedGraphDbImpl$TransactionImpl.finish(EmbeddedGraphDbImpl.java:331) >> at >> org.neo4j.util.SimpleBatchTxManager.beginReadOperation(SimpleBatchTxManager.java:94) >> at >> graph.implementation.neo.STSGraphNeoImpl.getUserNeo(STSGraphNeoImpl.java:463) >> at >> graph.implementation.neo.STSGraphNeoImpl.increaseUserTagFreq(STSGraphNeoImpl.java:824) >> at >> graph.benchmark.LoadGraphBenchmark.runRealBenchmark(LoadGraphBenchmark.java:433) >> at >> graph.benchmark.LoadGraphBenchmark.main(LoadGraphBenchmark.java:489) >> Caused by: java.lang.RuntimeException: Unable to close lucene writer >> org.apache.lucene.index.indexwri...@32a4eb93 >> at >> org.neo4j.index.lucene.LuceneDataSource.removeWriter(LuceneDataSource.java:413) >> at >> org.neo4j.index.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:197) >> at >> org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:316) >> at >> org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:399) >> at >> org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) >> at >> org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:514) >> at >> org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:573) >> at >> org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:543) >> at >> org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:102) >> at >> org.neo4j.kernel.EmbeddedGraphDbImpl$TransactionImpl.finish(EmbeddedGraphDbImpl.java:316) >> ... 5 more >> Caused by: java.io.IOException: directory '/[some-local-path]/resource_id' >> exists and is a directory, but cannot be listed: list() returned null >> at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:516) >> at org.apache.lucene.store.FSDirectory.listAll(FSDirectory.java:531) >> at >> org.apache.lucene.index.IndexFileDeleter.refresh(IndexFileDeleter.java:307) >> at >> org.apache.lucene.index.IndexWriter.doFlushInternal(IndexWriter.java:4300) >> at org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:4192) >> at org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:4183) >> at >> org.apache.lucene.index.IndexWriter.closeInternal(IndexWriter.java:2190) >> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2153) >> at org.apache.lucene.index.IndexWriter.close(IndexWriter.java:2117) >> at >> org.neo4j.index.lucene.LuceneDataSource.removeWriter(LuceneDataSource.java:409) >> ... 14 more >> >> >> _______________________________________________ >> Neo mailing list >> [email protected] >> https://lists.neo4j.org/mailman/listinfo/user >> >> >> > > > > _______________________________________________ Neo mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
