Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-03-02 Thread Webster Homer
essage- > From: Webster Homer [mailto:webster.ho...@sial.com] > Sent: Friday, March 2, 2018 10:29 AM > To: solr-user@lucene.apache.org > Subject: Re: NRT replicas miss hits and return duplicate hits when paging > solrcloud searches > > I am trying to test if enabling stats

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-03-02 Thread Webster Homer
Thanks Shawn. Commenting it out works to remove it. If I change the values e.g. change the 512 to 0, it does require a restart to take effect. Tested using statsCache set to org.apache.solr.search.stats.ExactSharedStatsCache, with the queryResultCache disabled, and I still see the problem with NR

RE: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-03-02 Thread Becky Bonner
pplication they belong to. It wasn’t a problem until we had 2 collections for one of the apps. -Original Message- From: Webster Homer [mailto:webster.ho...@sial.com] Sent: Friday, March 2, 2018 10:29 AM To: solr-user@lucene.apache.org Subject: Re: NRT replicas miss hits and return

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-03-02 Thread Shawn Heisey
On 3/2/2018 9:28 AM, Webster Homer wrote: I've never disabled this before. I edited the solrconfig.xml setting the sizes to 0. I'm not sure if this is how to disable the cache or not. To completely disable a cache, either comment it out or remove it from the config.  I do not know wheth

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-03-02 Thread Webster Homer
I am trying to test if enabling stats cache as suggested by Eric would also address this issue. I added this to my solrconfig.xml I executed queries and saw no differences. Then I re-indexed the data, again I saw no differences in behavior. Then I found this, SOLR-10952. It seems we need to di

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-02-27 Thread Webster Homer
Emir, Using tlog replica types addresses my immediate problem. The secondary issue is that all of our searches show inconsistent results. These are all normal paging use cases. We regularly test our relevancy, and these differences creates confusion in the testers. Moreover, we are migrating from

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-02-27 Thread Emir Arnautović
Hi Webster, Since you are returning all hits, returning the last page is almost as heavy for Solr as returning all documents. Maybe you should consider just returning one large page and completely avoid this issue. I agree with you that this should be handled by Solr. ES solved this issue with “

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-02-26 Thread Webster Homer
Erick, No we didn't look at that. I will add it to the list. We have not seen performance issues with solr. We have much slower technologies in our stack. This project was to replace a system that was too slow. Thank you, I will look into it Webster On Mon, Feb 26, 2018 at 1:13 PM, Erick Erick

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-02-26 Thread Erick Erickson
Did you try enabling distributed IDF (statsCache)? See: https://lucene.apache.org/solr/guide/6_6/distributed-requests.html It's may not totally fix the issue, but it's worth trying. It does come with a performance penalty of course. Best, Erick On Mon, Feb 26, 2018 at 11:00 AM, Webster Homer wr

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-02-26 Thread Webster Homer
Thanks Shawn, I had settled on this as a solution. All our use cases for Solr is to return results in order of relevancy to the query, so having a deterministic sort would defeat that purpose. Since we wanted to be able to return all the results for a query, I originally looked at using the Stream

Re: NRT replicas miss hits and return duplicate hits when paging solrcloud searches

2018-02-26 Thread Shawn Heisey
On 2/26/2018 10:26 AM, Webster Homer wrote: > We need the results by relevancy so the application sorts the results by > score desc, and the unique id ascending as the tie breaker This is the reason for the discrepancy, and why the different replica types don't have the same issue. Each NRT repli