An update on this, the error was on my side doubly incrementing the searcher reference. no problem on trunk!
simon On Fri, Sep 10, 2010 at 10:04 PM, Simon Rosenthal <simon.rosent...@yahoo.com> wrote: > Karl: > > I reported something very similar a few months back and opened a Jira issue > - see https://issues.apache.org/jira/browse/SOLR-1911. After I changed to a > newer nightly build the leak went away. The issue is still open, so you mayw > ant to update it. > > -Simon > > ________________________________ > > ---------- Forwarded message ---------- > From: <karl.wri...@nokia.com> > Date: Fri, Sep 10, 2010 at 3:24 PM > Subject: RE: Trunk file handle leak? > To: dev@lucene.apache.org, yo...@lucidimagination.com > > > Hi Yonik, > > Be that as it may, I'm seeing a steady increase in file handles used by that > process over an extended period of time (now 20+ minutes): > > r...@duck6:~# lsof -p 22379 | wc > 786 7714 108339 > r...@duck6:~# lsof -p 22379 | wc > 787 7723 108469 > r...@duck6:~# lsof -p 22379 | wc > 787 7723 108469 > r...@duck6:~# lsof -p 22379 | wc > 812 7948 111719 > r...@duck6:~# lsof -p 22379 | wc > 816 7984 112239 > r...@duck6:~# lsof -p 22379 | wc > 817 7993 112369 > r...@duck6:~# lsof -p 22379 | wc > 822 8038 113019 > r...@duck6:~# lsof -p 22379 | wc > 847 8308 116719 > r...@duck6:~# lsof -p 22379 | wc > 852 8353 117369 > r...@duck6:~# lsof -p 22379 | wc > 897 8803 123669 > r...@duck6:~# lsof -p 22379 | wc > 1022 10018 140819 > r...@duck6:~# > > This doesn't smell like spiky resource usage to me. It smells like a leak. > ;-) > > Karl > > -----Original Message----- > From: ysee...@gmail.com [mailto:ysee...@gmail.com] On Behalf Of ext Yonik > Seeley > Sent: Friday, September 10, 2010 2:08 PM > To: dev@lucene.apache.org > Subject: Re: Trunk file handle leak? > > On Fri, Sep 10, 2010 at 1:51 PM, <karl.wri...@nokia.com> wrote: >> (1) There are periodic commits, every 10,000 records. >> (2) I have no searcher/reader open at the same time, that I am aware of. >> This is a straight indexing task. (You ought to know, you wrote some of >> the code!) > > A commit currently opens a new searcher in Solr. > It's not too hard to go past 1024 descriptors - either raise the limit > to 10240 or something, use the compound file format, or lower the > merge factor. > > >> (3) I *do* see auto warming being called, but it seems not to happen at >> the same time as a commit, but rather afterwards. >> >> Once it starts happening, this happens repeatedly on every commit. > > This would also be expected - it's at a point where there are too many > files for your descriptors. > > -Yonik > http://lucenerevolution.org Lucene/Solr Conference, Boston Oct 7-8 > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > > --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
