If you create SolrQueryRequest make sure you close it then, since it's
necessary to release a searcher.
On Wed, Apr 19, 2017 at 12:35 PM, Elodie Sannier <elodie.sann...@kelkoo.fr>
wrote:
> Hello,
>
> We have found how to fix the problem.
> When we update the original SolrQueryResponse object, we need to create
> a new BasicResultContext object with the extra response.
>
> Simplified code :
>
> public class CustomSearchHandler extends SearchHandler {
>
> public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse
> rsp) throws Exception {
>
> SolrQueryRequest extraReq = createExtraRequest();
> SolrQueryResponse extraRsp = new SolrQueryResponse();
>
> super.handleRequestBody(extraReq, extraRsp);
>
> ResultContext extraRc = (ResultContext) extraRsp.getResponse();
>
> // code with memory leak !!
> rsp.addResponse(extraRc);
>
> // code without memory leak
> ResultContext extraRcClone = new BasicResultContext(extraRc.get
> DocList(),
> rsp.getReturnFields(), req.getSearcher(),
> extraRc.getQuery(), req);
> rsp.addResponse(extraRcClone);
>
> }
>
> }
>
> We don't know why we need to create a new BasicResultContext to properly
> manage searchers. Do you know why ?
>
> Elodie
>
>
> On 04/07/2017 04:14 PM, Rick Leir wrote:
>
>> Hi Gerald
>> The best solution in my mind is to look at the custom code and try to
>> find a way to remove it from your system. Solr queries can be complex, and
>> I hope there is a way to get the results you need. Would you like to say
>> what results you want to get, and what Solr queries you have tried?
>> I realize that in large organizations it is difficult to suggest change.
>> Cheers -- Rick
>>
>> On April 7, 2017 9:08:19 AM EDT, Shawn Heisey <apa...@elyograg.org>
>> wrote:
>>
>>> On 4/7/2017 3:09 AM, Gerald Reinhart wrote:
>>>
>>>> We have some custom code that extends SearchHandler to be able to
>>>>
>>> :
>>>
>>>> - do an extra request
>>>> - merge/combine the original request and the extra request
>>>> results
>>>>
>>>> On Solr 5.x, our code was working very well, now with Solr 6.x we
>>>> have the following issue: the number of SolrIndexSearcher are
>>>> increasing (we can see them in the admin view > Plugins/ Stats > Core
>>>>
>>> ).
>>>
>>>> As SolrIndexSearcher are accumulating, we have the following issues :
>>>> - the memory used by Solr is increasing => OOM after a long
>>>> period of time in production
>>>> - some files in the index has been deleted from the system but
>>>> the Solr JVM still hold them => ("fake") Full disk after a long
>>>>
>>> period
>>>
>>>> of time in production
>>>>
>>>> We are wondering,
>>>> - what has changed between Solr 5.x and Solr 6.x in the
>>>> management of the SolrIndexSearcher ?
>>>> - what would be the best way, in a Solr plugin, to perform 2
>>>> queries and merge the results to a single SolrQueryResponse ?
>>>>
>>> I hesitated to send a reply because when it comes right down to it, I
>>> do
>>> not know a whole lot about deep Solr internals. I tend to do my work
>>> with the code at a higher level, and don't dive down in the depths all
>>> that often. I am slowly learning, though. You may need to wait for a
>>> reply from someone who really knows those internals.
>>>
>>> It looks like you and I participated in a discussion last month where
>>> you were facing a similar problem with searchers -- deleted index files
>>> being held open. How did that turn out? Seems like if that problem
>>> were solved, it would also solve this problem.
>>>
>>> Very likely, the fact that the plugin worked correctly in 5.x was
>>> actually a bug in Solr related to reference counting, one that has been
>>> fixed in later versions.
>>>
>>> You may need to use a paste website or a file-sharing website to share
>>> all your plugin code so that people can get a look at it. The list has
>>> a habit of deleting attachments.
>>>
>>> Thanks,
>>> Shawn
>>>
>>
>
> Kelkoo SAS
> Société par Actions Simplifiée
> Au capital de € 4.168.964,30
> Siège social : 158 Ter Rue du Temple 75003 Paris
> 425 093 069 RCS Paris
>
> Ce message et les pièces jointes sont confidentiels et établis à
> l'attention exclusive de leurs destinataires. Si vous n'êtes pas le
> destinataire de ce message, merci de le détruire et d'en avertir
> l'expéditeur.
>
--
Sincerely yours
Mikhail Khludnev
