What does lsof tell - which handles are open and which are unexpected or suspicious?
Am 22.02.2017 5:27 nachm. schrieb "Leonid Bolshinsky" <leonid...@gmail.com>: > No code changes between running in Java 6 and Java 8. The same code still > runs successfully with Java 6 while is failing with Java 8. > > On Feb 22, 2017 5:52 PM, "Frederik Van Hoyweghen" < > frederik.vanhoyweg...@chapoo.com> wrote: > > Did you make any changes to your code between running on java 6 and 8? > The java file API has changed considerably since java 8. > > If you -have- rewritten some of the file handling code in your indexing > process, make sure to explicitly close the streams you create, or use the > (since java 7) try-with-resources construct. > > > On 22/02/2017 16:18, Leonid Bolshinsky wrote: > > > I have a search engine based on Lucene 3.0.3 and I can't change the > Lucene > > version for reasons that are out of scope of this question. Now I have a > > requirement to move from Java 6 to Java 8, however when I run the > indexing > > using Java 8 JVM, I hit "Too many open files issue" as below: > > > > java.io.FileNotFoundException: /myIndex/_27c.fdx (Too many open files) > > at java.io.RandomAccessFile.open0(Native Method) > > at java.io.RandomAccessFile.open(RandomAccessFile.java:333) > > at java.io.RandomAccessFile.<init>(RandomAccessFile.java:257) > > at org.apache.lucene.store.SimpleFSDirectory$SimpleFSIndexOutpu > > t.<init>(SimpleFSDirectory.java:180) > > at org.apache.lucene.store.NIOFSDirectory.createOutput(NIOFSDir > > ectory.java:85) > > at org.apache.lucene.index.FieldsWriter.<init>( > FieldsWriter.java:86) > > at org.apache.lucene.index.StoredFieldsWriter.initFieldsWriter( > > StoredFieldsWriter.java:66) > > at org.apache.lucene.index.StoredFieldsWriter.finishDocument(St > > oredFieldsWriter.java:144) > > at org.apache.lucene.index.StoredFieldsWriter$PerDoc.finish( > > StoredFieldsWriter.java:192) > > at org.apache.lucene.index.DocumentsWriter$WaitQueue.writeDocum > > ent(DocumentsWriter.java:1559) > > at org.apache.lucene.index.DocumentsWriter$WaitQueue.add(Docume > > ntsWriter.java:1578) > > at org.apache.lucene.index.DocumentsWriter.finishDocument(Docum > > entsWriter.java:1155) > > at org.apache.lucene.index.DocumentsWriter.updateDocument(Docum > > entsWriter.java:832) > > at org.apache.lucene.index.DocumentsWriter.updateDocument(Docum > > entsWriter.java:807) > > at org.apache.lucene.index.IndexWriter.updateDocument(IndexWrit > > er.java:2155) > > > > The ulimit on my system is 80.000 files and when I run lsof, I get that > the > > open files belong to the index and they are marked as (deleted) > segments. I > > am aware that there are tons of questions, answers and blogs about the > > Lucene open files issue, however: > > > > - I've already tried whatever is recommended (review the code looking > > > > for open IndexWriters, increase ulimit, enable compound mode, > decrease > > the > > number of segments from 10 to 4, review the code for open files) > > without > > success and > > - The issue didn't appear with Java 6 so I am guessing that the > problem > > > > is in how Java 8 handles files. > > > > Anybody faced this issue with Java 8 before? Any additional idea about > how > > to further troubleshoot this issue or what could be the cause? > > > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org > For additional commands, e-mail: java-user-h...@lucene.apache.org >
