Re: Inconsistent numFound in SC when querying core directly

2017-03-14 Thread Erick Erickson
bq: If I changed the routing strategy back to composite (which it should be). is it ok? I sincerely doubt it. The docs have already been routed to the wrong place (actually, I'm not sure how it worked at all). You can't get them redistributed simply by changing the definition in ZooKeeper,

Re: Inconsistent numFound in SC when querying core directly

2017-03-14 Thread vbindal
I think I dint explain properly. I have 3 data centers each with its own SOLR cloud. My original strategy was composite routing but when one data center went down and we brought it back, somehow the routing strategy on this changed to implicit (Other 2 DC still have composit and they are

Re: Inconsistent numFound in SC when querying core directly

2017-03-14 Thread Erick Erickson
That would make the problem even worse. If you created the collection with implicit routing, there are no hash ranges for each shard. CompositeId requires hash ranges to be defined for each shard. Don't even try. Best, Erick On Tue, Mar 14, 2017 at 11:13 AM, vbindal

Re: Inconsistent numFound in SC when querying core directly

2017-03-14 Thread vbindal
Compared it against the other 2 datacenters and they both have `compositeId `. This started happening after 1 of our zookeeper died due to hardware issue and we had to setup a new zookeeper machine. update the config in all the solr machine and restart the cloud. My guess is something went wrong

Re: Inconsistent numFound in SC when querying core directly

2017-03-14 Thread Erick Erickson
The default router has always been compositeId, but when you created your collection you may have created it with implicit. Looking at the clusterstate.json and/or state.json in the individual collection should show you (admin UI>>cloud>>tree). But we need to be very clear about what a

Re: Inconsistent numFound in SC when querying core directly

2017-03-14 Thread vbindal
Hi Shawn, We are on 4.10.0 version. Is that the default router in this version? Also, we dont see all the documents duplicated, only some of them. I have a indexer job to index data in SOLR. After I delete all the records and run this job, the count is correct but when I run the job again, we

Re: Inconsistent numFound in SC when querying core directly

2017-03-13 Thread Shawn Heisey
On 3/13/2017 3:16 AM, vbindal wrote: > I am facing the same issue where my query *:* returns inconsistent number > (almost 3) time the actual number in millions. > > When I try disturb=false on every machine, the results are correct. but > without `disturb=false` results are incorrect. This most

Re: Inconsistent numFound in SC when querying core directly

2017-03-13 Thread vbindal
Hi, I am facing the same issue where my query *:* returns inconsistent number (almost 3 ) time the actual number in millions. When I try disturb=false on every machine, the results are correct. but without `disturb=false` results are incorrect. Can you guys suggest something? -- View this

Re: Inconsistent numFound in SC when querying core directly

2013-12-05 Thread Tim Vaillancourt
Very good point. I've seen this issue occur once before when I was playing with 4.3.1 and don't remember it happening since 4.5.0+, so that is good news - we are just behind. For anyone that is curious, on my earlier mention that Zookeeper/clusterstate.json was not taking updates: this was NOT

Re: Inconsistent numFound in SC when querying core directly

2013-12-05 Thread Tim Vaillancourt
I spoke too soon, my plan for fixing this didn't quite work. I've moved this issue into a new thread/topic: No /clusterstate.json updates on Solrcloud 4.3.1 Cores API UNLOAD/CREATE. Thanks all for the help on this one! Tim On 5 December 2013 11:37, Tim Vaillancourt t...@elementspace.com

Re: Inconsistent numFound in SC when querying core directly

2013-12-04 Thread Tim Vaillancourt
To add two more pieces of data: 1) This occurs with real, conditional queries as well (eg: q=key:timvaillancourt), not just the q=*:* I provided in my email. 2) I've noticed when I bring a node of the SolrCloud down it is remaining state: active in my /clusterstate.json - something is really

RE: Inconsistent numFound in SC when querying core directly

2013-12-04 Thread Markus Jelsma
commits now (since tuesday) and are still waiting for it to happen again. -Original message- From:Tim Vaillancourt t...@elementspace.com Sent: Wednesday 4th December 2013 23:38 To: solr-user@lucene.apache.org Subject: Re: Inconsistent numFound in SC when querying core directly

Re: Inconsistent numFound in SC when querying core directly

2013-12-04 Thread Tim Vaillancourt
: Inconsistent numFound in SC when querying core directly To add two more pieces of data: 1) This occurs with real, conditional queries as well (eg: q=key:timvaillancourt), not just the q=*:* I provided in my email. 2) I've noticed when I bring a node of the SolrCloud down it is remaining state

Re: Inconsistent numFound in SC when querying core directly

2013-12-04 Thread Chris Hostetter
: : I may be incorrect here, but I assumed when querying a single core of a : SolrCloud collection, the SolrCloud routing is bypassed and I am talking : directly to a plain/non-SolrCloud core. No ... every query received from a client by solr is handled by a single core -- if that core knows

Re: Inconsistent numFound in SC when querying core directly

2013-12-04 Thread Tim Vaillancourt
Chris, this is extremely helpful and it's silly I didn't think of this sooner! Thanks a lot, this makes the situation make much more sense. I will gather some proper data with your suggestion and get back to the thread shortly. Thanks!! Tim On 04/12/13 02:57 PM, Chris Hostetter wrote: : :

Re: Inconsistent numFound in SC when querying core directly

2013-12-04 Thread Tim Vaillancourt
Hey all, Now that I am getting correct results with distrib=false, I've identified that 1 of my nodes has just 1/3rd of the total data set and totally explains the flapping in results. The fix for this is obvious (rebuild replica) but the cause is less obvious. There is definately more than

Re: Inconsistent numFound in SC when querying core directly

2013-12-04 Thread Mark Miller
Keep in mind, there have been a *lot* of bug fixes since 4.3.1. - Mark On Dec 4, 2013, at 7:07 PM, Tim Vaillancourt t...@elementspace.com wrote: Hey all, Now that I am getting correct results with distrib=false, I've identified that 1 of my nodes has just 1/3rd of the total data set and