Re: Lucene 2.9.0 leaves too many .cfs files open, causing too many files open java error.

Sun, 18 Oct 2009 11:12:19 -0700

I actually have seen the problem in both, with our app. Not sure if it's how we are using lucene or if the problem is internal. We actually only saw the problem recently because we starting using a new linux machine that had it's open file handle settings too low. And with our volume it only took about 10 hours for the machine to start having problems. Hence the need for the hack patch. 

Aaron

On Oct 18, 2009, at 1:49 PM, GlenAbbeyDrive wrote:



Are you using 2.9.0 or 2.4?



Aaron McCurry wrote:


I also have seen this problem recently.  I had to make a patch to our
production system to at least relief the system of the deleted files
handles.  I did it by capturing all of the Descriptors that the
FileInputIndex object creates and monitoring if the file it references 
still exists or not.  If it doesn't I then try and close the
Descriptors.  I know that this is a hack, but it was needed ASAP, and
I couldn't find in my code anywhere where I wasn't closing the opened
indexreaders or indexwriters.

Aaron


On Oct 18, 2009, at 11:14 AM, GlenAbbeyDrive wrote:



I switched from Lucene 2.4.0 to the latest 2.9.0 version and got too
many
files open within a few hours from my indexing process.  Our
indexing Java
process adds about 2000 documents/minute.
The IndexWriter (iw) has the following settings:

          iw.setMaxFieldLength(1024*1024*1024); // 1G as maximum
terms.
          iw.setMergeFactor(10);
          iw.setMaxMergeDocs(100000);
          iw.setMaxBufferedDocs(1000);
          iw.setRAMBufferSizeMB(128);
          iw.setUseCompoundFile(true);



The 'lsof' result shows rapidly growing number of open files in
'deleted'
state. Counting the number of these:

[server-1]#date
Sun Oct 18 11:02:37 EDT 2009
[server-1]# lsof -p 19606 |grep deleted  |wc -l
3187
[server-1]# date
Sun Oct 18 11:07:29 EDT 2009
[server-1]# lsof -p 19606 |grep deleted  |wc -l
3912

Here is a sample of those open files left behind in 'deleted' state:

java    19606 root 1861r   REG       8,17    273561 368673054
/archive/index/_5fq.cfs (deleted)
java    19606 root 1862r   REG       8,17      9428 368673053
/archive/index/_5fp.cfs (deleted)
java    19606 root 1863r   REG       8,17    363690 368673052
/archive/index/_5fo.cfs (deleted)
java    19606 root 1864r   REG       8,17     47732 368673051
/archive/index/_5fn.cfs (deleted)
java    19606 root 1865r   REG       8,17    417453 368673050
/archive/index/_5fm.cfs (deleted)
java    19606 root 1866r   REG       8,17   1518126 368673049
/archive/index/_5fl.cfs (deleted)
java    19606 root 1871r   REG       8,17   1236023 368673057
/archive/index/_5ft.cfs (deleted)
java    19606 root 1872r   REG       8,17     29743 368673055
/archive/index/_5fr.cfs (deleted)
java    19606 root 1873r   REG       8,17    273561 368673054
/archive/index/_5fq.cfs (deleted)
java    19606 root 1874r   REG       8,17      9428 368673053
/archive/index/_5fp.cfs (deleted)
java    19606 root 1875r   REG       8,17    363690 368673052
/archive/index/_5fo.cfs (deleted)
java    19606 root 1876r   REG       8,17     47732 368673051
/archive/index/_5fn.cfs (deleted)
java    19606 root 1877r   REG       8,17    417453 368673050
/archive/index/_5fm.cfs (deleted)
java    19606 root 1878r   REG       8,17   1518126 368673049
/archive/index/_5fl.cfs (deleted)
java    19606 root 1883r   REG       8,17   1389856 368673056
/archive/index/_5fs.cfs (deleted)
java    19606 root 1884r   REG       8,17     29743 368673055
/archive/index/_5fr.cfs (deleted)
java    19606 root 1885r   REG       8,17    273561 368673054
/archive/index/_5fq.cfs (deleted)
java    19606 root 1886r   REG       8,17      9428 368673053
/archive/index/_5fp.cfs (deleted)
java    19606 root 1887r   REG       8,17    363690 368673052
/archive/index/_5fo.cfs (deleted)
java    19606 root 1888r   REG       8,17     47732 368673051
/archive/index/_5fn.cfs (deleted)
java    19606 root 1889r   REG       8,17    417453 368673050
/archive/index/_5fm.cfs (deleted)
java    19606 root 1890r   REG       8,17   1518126 368673049
/archive/index/_5fl.cfs (deleted)
java    19606 root 1895r   REG       8,17   1389856 368673056
/archive/index/_5fs.cfs (deleted)
java    19606 root 1896r   REG       8,17     29743 368673055
/archive/index/_5fr.cfs (deleted)
java    19606 root 1897r   REG       8,17    273561 368673054
/archive/index/_5fq.cfs (deleted)
java    19606 root 1898r   REG       8,17      9428 368673053
/archive/index/_5fp.cfs (deleted)
java    19606 root 1899r   REG       8,17    363690 368673052
/archive/index/_5fo.cfs (deleted)
java    19606 root 1900r   REG       8,17     47732 368673051
/archive/index/_5fn.cfs (deleted)
java    19606 root 1901r   REG       8,17    417453 368673050
/archive/index/_5fm.cfs (deleted)
java    19606 root 1902r   REG       8,17   1518126 368673049
/archive/index/_5fl.cfs (deleted)
java    19606 root 1910r   REG       8,17    403610 368673058
/archive/index/_5fu.cfs (deleted)
java    19606 root 1911r   REG       8,17   1236023 368673057
/archive/index/_5ft.cfs (deleted)
java    19606 root 1912r   REG       8,17   1389856 368673056
/archive/index/_5fs.cfs (deleted)
java    19606 root 1913r   REG       8,17     29743 368673055
/archive/index/_5fr.cfs (deleted)
java    19606 root 1914r   REG       8,17    273561 368673054
/archive/index/_5fq.cfs (deleted)
java    19606 root 1915r   REG       8,17      9428 368673053
/archive/index/_5fp.cfs (deleted)
java    19606 root 1916r   REG       8,17    363690 368673052
/archive/index/_5fo.cfs (deleted)
java    19606 root 1917r   REG       8,17     47732 368673051
/archive/index/_5fn.cfs (deleted)
java    19606 root 1918r   REG       8,17    417453 368673050
/archive/index/_5fm.cfs (deleted)
java    19606 root 1919r   REG       8,17   1518126 368673049
/archive/index/_5fl.cfs (deleted)


--
View this message in context:
http://www.nabble.com/Lucene-2.9.0-leaves-too-many-.cfs-files-open%2C-causing-too-many-files-open-java-error.-tp25947462p25947462.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com. 


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org





--
View this message in context: 
http://www.nabble.com/Lucene-2.9.0-leaves-too-many-.cfs-files-open%2C-causing-too-many-files-open-java-error.-tp25947462p25948828.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org




---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org

Reply via email to