Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-11-10 Thread Thomas Rewig
Hello, please excuse that I hijack this old thread but I have the same problem with the deleted file handles, so I think this is the right place for. I also integrated the searchManager in our Code and see the file handles fluctuate up and down. At first glance the situation seems stable but

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-10-01 Thread Michael McCandless
On Fri, Oct 1, 2010 at 1:41 AM, Jamie wrote: >  I managed to get hold of a copy of your book through Safari Books. Quite an > impressive online reading system they have there! Neat! I had never heard of Safari Books until now, but it looks cool. > I integrated your > SearchManager class into o

RE: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Uwe Schindler
-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Jamie [mailto:ja...@stimulussoft.com] > Sent: Friday, October 01, 2010 7:41 AM > To: java-user@lucene.apache.org > Subject: Re: File Handle Leaks During Lucene 3.0.2 Merge > > Hi

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Jamie
Hi Mike I managed to get hold of a copy of your book through Safari Books. Quite an impressive online reading system they have there! I integrated your SearchManager class into our code, but I am still seeing file handles marked deleted in the index directory. I am running the following com

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Michael McCandless
On Thu, Sep 30, 2010 at 5:59 AM, Jamie wrote: >  Hi Michael / Uwe > >>It's good to cache the reader, but, finalize would worry me too since >>you have no control over when GC gets around to calling it... you risk >>tying up resources for longer than necessary. > > I did it this way, as I didn't wa

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Michael McCandless
You can also use the IndexReader's incRef/decRef methods. Mike On Thu, Sep 30, 2010 at 6:12 AM, Uwe Schindler wrote: > Hi Jamie, >>  >It's good to cache the reader, but, finalize would worry me too since >>you >> have no control over when GC gets around to calling it... you risk  >tying > up >>

RE: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Uwe Schindler
Hi Jamie, > >It's good to cache the reader, but, finalize would worry me too since >you > have no control over when GC gets around to calling it... you risk >tying up > resources for longer than necessary. > > I did it this way, as I didn't want to over complicate the code by introducing > mecha

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Jamie
Hi Michael / Uwe >It's good to cache the reader, but, finalize would worry me too since >you have no control over when GC gets around to calling it... you risk >tying up resources for longer than necessary. I did it this way, as I didn't want to over complicate the code by introducing mechanis

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Michael McCandless
Comments inline... On Thu, Sep 30, 2010 at 5:26 AM, Jamie wrote: >  Uwe > > If I recall correctly when you call writer.getReader(), the returned > IndexReader can consume alot of memory with large indexes The reopened reader shares sub-readers with the previous one, so, if all that's changed sin

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Michael McCandless
t; From: Jamie [mailto:ja...@stimulussoft.com] >> Sent: Wednesday, September 29, 2010 11:50 AM >> To: java-user@lucene.apache.org >> Subject: Re: File Handle Leaks During Lucene 3.0.2 Merge >> >>   Hi Uwe >> >> Thanks in advance for your help. Well, I just tried se

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Jamie
Uwe If I recall correctly when you call writer.getReader(), the returned IndexReader can consume alot of memory with large indexes. To ensure that the same index reader is reused across multiple search threads, I keep a cached copy of the reader and return it. If a search thread closes the r

RE: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-30 Thread Uwe Schindler
- Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de > -Original Message- > From: Jamie [mailto:ja...@stimulussoft.com] > Sent: Wednesday, September 29, 2010 11:50 AM > To: java-user@lucene.apache.org > Subject: Re: File Handle

Re: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-29 Thread Jamie
Hi Uwe Thanks in advance for your help. Well, I just tried searching again and it made no difference. My LuceneIndex getReader() function will call writer.getReader() on occasion or return a cached copy. To make sure that IndexReader's are closed when they are no longer needed, I wrap the Ind

RE: File Handle Leaks During Lucene 3.0.2 Merge

2010-09-29 Thread Uwe Schindler
ginal Message- > From: Jamie [mailto:ja...@stimulussoft.com] > Sent: Wednesday, September 29, 2010 10:48 AM > To: java-user@lucene.apache.org > Subject: File Handle Leaks During Lucene 3.0.2 Merge > > Hi There > > I am noticing file handle leaks appearing on Index files.

File Handle Leaks During Lucene 3.0.2 Merge

2010-09-29 Thread Jamie
Hi There I am noticing file handle leaks appearing on Index files. I think the leaks occur during the Lucene merge operation. Lsof reports the following: java 28604 root 213r REG 8,33 1098681 57409621 /var/index/vol201009/_a4w.cfs (deleted) java 28604 roo