There are two things here: The error you're getting is because you probably forget to close some IndexHits somewhere, IndexHits that you haven't iterated all the way through... could you check your code for such mistakes?
The other one that doCommit always seems to be invoked I just tried and cannot reproduce, so may be something on your end? could you provide a code snippet to reproduce this? Best, Mattias 2011/3/30 Axel Morgner <a...@morgner.de> > Hi, > > during stress tests, I got a "Too many open files" message (see detailed > stack trace below). Server is a Debian Linux 2.6.26 x86_64 w/ 24 GB RAM, > Core i7 which handles the load with ease. > > cat /proc/sys/fs/file-max > 2399285 > > The parameter fs.file-max was already set very high, but I'll increase > it further and see if it will happen again. > > fs.file-max = 6815744 > fs.aio-max-nr = 1048576 > > Are there recommendations of linux kernel parameters? Or/and would you > recommend other things to circumvent that, f.e. using multiple indexes > instead of one? > > And it seems to me that finishing a Neo4j transaction will always > trigger a LuceneTransaction.doCommit, even if there was no update on > index (I try to avoid for the log nodes). Can this be switched off? > > Thanks and greetings > > Axel > > > [1] Stack trace > > java.lang.RuntimeException: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) > at > org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) > at > > org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) > at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) > at org.structr.core.log.LogService.flushQueue(LogService.java:101) > at org.structr.core.log.LogService.run(LogService.java:73) > Caused by: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233) > at > > org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.<init>(SimpleFSDirectory.java:180) > at > org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) > at org.apache.lucene.index.FieldsWriter.<init>(FieldsWriter.java:86) > at > > org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66) > at > > org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144) > at > > org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:193) > at > > org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1460) > at > > org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1479) > at > > org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1099) > at > > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:777) > at > > org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:752) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1932) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1906) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.applyDocuments(LuceneTransaction.java:299) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:260) > ... 12 more > javax.transaction.xa.XAException: Unknown > xid[GlobalId[NEOKERNL|5131995998687892543|61442], BranchId[ 52 49 52 49 > 52 49 ]] > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.rollback(XaResourceManager.java:470) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.rollback(XaResourceHelpImpl.java:111) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.doRollback(TransactionImpl.java:533) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:648) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) > at > org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) > at > > org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) > at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) > at org.structr.core.log.LogService.flushQueue(LogService.java:101) > at org.structr.core.log.LogService.run(LogService.java:73) > Mar 30, 2011 6:31:53 PM org.neo4j.kernel.impl.transaction.TxManager commit > SEVERE: Unable to rollback transaction. Some resources may be commited > others not. Neo4j kernel should be SHUTDOWN for resource maintance and > transaction recovery ----> > java.lang.RuntimeException: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:272) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaTransaction.commit(XaTransaction.java:318) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceManager.commit(XaResourceManager.java:446) > at > > org.neo4j.kernel.impl.transaction.xaframework.XaResourceHelpImpl.commit(XaResourceHelpImpl.java:64) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.doCommit(TransactionImpl.java:516) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:621) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) > at > org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) > at > > org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) > at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) > at org.structr.core.log.LogService.flushQueue(LogService.java:101) > at org.structr.core.log.LogService.run(LogService.java:73) > Caused by: java.io.FileNotFoundException: > /opt/structr/t5s/db/index/lucene/node/fulltextAllNodes/_ngl.fdx (Too > many open files) > at java.io.RandomAccessFile.open(Native Method) > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:233) > at > > org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutput.<init>(SimpleFSDirectory.java:180) > at > org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDirectory.java:74) > at org.apache.lucene.index.FieldsWriter.<init>(FieldsWriter.java:86) > at > > org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter(StoredFieldsWriter.java:66) > at > > org.apache.lucene.index.StoredFieldsWriter.finishDocument(StoredFieldsWriter.java:144) > at > > org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish(StoredFieldsWriter.java:193) > at > > org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocument(DocumentsWriter.java:1460) > at > > org.apache.lucene.index.DocumentsWriter$WaitQueue.add(DocumentsWriter.java:1479) > at > > org.apache.lucene.index.DocumentsWriter.finishDocument(DocumentsWriter.java:1099) > at > > org.apache.lucene.index.DocumentsWriter.updateDocument(DocumentsWriter.java:777) > at > > org.apache.lucene.index.DocumentsWriter.addDocument(DocumentsWriter.java:752) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1932) > at > org.apache.lucene.index.IndexWriter.addDocument(IndexWriter.java:1906) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.applyDocuments(LuceneTransaction.java:299) > at > > org.neo4j.index.impl.lucene.LuceneTransaction.doCommit(LuceneTransaction.java:260) > ... 12 more > org.neo4j.graphdb.TransactionFailureException: Unable to commit transaction > at > org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:104) > at > > org.structr.core.node.TransactionCommand.execute(TransactionCommand.java:37) > at org.structr.core.entity.AbstractNode.commit(AbstractNode.java:1048) > at org.structr.core.log.LogService.flushQueue(LogService.java:101) > at org.structr.core.log.LogService.run(LogService.java:73) > Caused by: javax.transaction.HeuristicMixedException: Unable to rollback > ---> error code in commit: -1 ---> error code for rollback: 0 > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:662) > at > org.neo4j.kernel.impl.transaction.TxManager.commit(TxManager.java:584) > at > > org.neo4j.kernel.impl.transaction.TransactionImpl.commit(TransactionImpl.java:104) > at > org.neo4j.kernel.TopLevelTransaction.finish(TopLevelTransaction.java:85) > ... 4 more > > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
