Re: 4820 searchers opened?

2011-08-06 Thread Paul Libbrecht
Le 6 août 2011 à 02:09, Yonik Seeley a écrit : On Fri, Aug 5, 2011 at 7:30 PM, Paul Libbrecht p...@hoplahup.net wrote: my solr is coming to slowly reach its memory limits (8Gb) and the stats displays me a reasonable fieldCache (1800) but 4820 searchers. That sounds a bit much to me, each

Re: 4820 searchers opened?

2011-08-06 Thread Yonik Seeley
On Sat, Aug 6, 2011 at 11:31 AM, Paul Libbrecht p...@hoplahup.net wrote: Le 6 août 2011 à 02:09, Yonik Seeley a écrit : On Fri, Aug 5, 2011 at 7:30 PM, Paul Libbrecht p...@hoplahup.net wrote: my solr is coming to slowly reach its memory limits (8Gb) and the stats displays me a reasonable

Re: 4820 searchers opened?

2011-08-06 Thread Paul Libbrecht
Le 6 août 2011 à 17:37, Yonik Seeley a écrit : I have a custom query-handler and a custom response writer. Do you always retrieve the searcher via SolrQueryRequest.getSearcher()? If so, there should be no problem... but if you call SolrCore.getSearcher(), that is where leaks can happen if

Re: 4820 searchers opened?

2011-08-06 Thread Yonik Seeley
On Sat, Aug 6, 2011 at 1:35 PM, Paul Libbrecht p...@hoplahup.net wrote: Le 6 août 2011 à 17:37, Yonik Seeley a écrit : I have a custom query-handler and a custom response writer. Do you always retrieve the searcher via SolrQueryRequest.getSearcher()?  If so, there should be no problem...

Re: 4820 searchers opened?

2011-08-06 Thread Paul Libbrecht
Le 6 août 2011 à 19:52, Yonik Seeley a écrit : I've been using the following: rb.req.getCore().getSearcher().get().getReader() Bingo! Code should never do core.getSearcher().get() since core.getSearcher returns a reference that must be decremented when you are done. Using

Re: 4820 searchers opened?

2011-08-06 Thread Paul Libbrecht
This is convincing me... I'd like to experiment and close. So, how can I be sure this is the right thing? I would have thought adding a document and committing would have created a Searcher in my current usage but I do not see the reference list actually being enlarged on my development

Re: 4820 searchers opened?

2011-08-06 Thread Yonik Seeley
On Sat, Aug 6, 2011 at 2:17 PM, Paul Libbrecht p...@hoplahup.net wrote: This is convincing me... I'd like to experiment and close. So, how can I be sure this is the right thing? I would have thought adding a document and committing would have created a Searcher in my current usage but I do

Re: 4820 searchers opened?

2011-08-06 Thread Paul Libbrecht
Le 6 août 2011 à 20:21, Yonik Seeley a écrit : It is creating a new searcher, but then closing the old searcher after all currently running requests are done using it (that's what the reference counting is for). After the searcher is closed, it's removed from the list. Not if using:

Re: 4820 searchers opened?

2011-08-06 Thread Paul Libbrecht
PS: why is RefCounted not using SoftReference?? I think I would not see my bug then.

Re: 4820 searchers opened?

2011-08-06 Thread Yonik Seeley
On Sat, Aug 6, 2011 at 2:30 PM, Paul Libbrecht p...@hoplahup.net wrote: Le 6 août 2011 à 20:21, Yonik Seeley a écrit : It is creating a new searcher, but then closing the old searcher after all currently running requests are done using it (that's what the reference counting is for). After

Re: 4820 searchers opened?

2011-08-06 Thread Paul Libbrecht
Le 7 août 2011 à 01:10, Yonik Seeley a écrit : Oh, I see... you want to re-create the bug so you can see when it is fixed? To trigger the bug, you need to hit a code path that uses the getCore().getSearcher().get() code. So first send a request that hits that buggy code, then add a doc and

4820 searchers opened?

2011-08-05 Thread Paul Libbrecht
Hello list, my solr is coming to slowly reach its memory limits (8Gb) and the stats displays me a reasonable fieldCache (1800) but 4820 searchers. That sounds a bit much to me, each has been opened in its own time since the last restart about two weeks ago. What could be wrong in my

Re: 4820 searchers opened?

2011-08-05 Thread Yonik Seeley
On Fri, Aug 5, 2011 at 7:30 PM, Paul Libbrecht p...@hoplahup.net wrote: my solr is coming to slowly reach its memory limits (8Gb) and the stats displays me a reasonable fieldCache (1800) but 4820 searchers. That sounds a bit much to me, each has been opened in its own time since the last