Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-28 Thread Michael Della Bitta
eory - Zookeeper logs. From: S.L [mailto:simpleliving...@gmail.com] Sent: Monday, October 27, 2014 1:42 PM To: solr-user@lucene.apache.org Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch. Please find the clusterstate.json attached. Also in this

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-28 Thread S.L
he said you've >>> saturated your environment. In AWS people just auto-scale up and don't >>> worry about where the load comes from; its dumb if it happens more than 2 >>> times. Capacity planning is tough, let's hope it doesn't disappear >>> a

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-28 Thread S.L
AWS people just auto-scale up and don't >>> worry about where the load comes from; its dumb if it happens more than 2 >>> times. Capacity planning is tough, let's hope it doesn't disappear >>> altogether. >>> >>> G'luck >>>

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-28 Thread Michael Della Bitta
nt: Monday, October 27, 2014 9:25 PM To: solr-user@lucene.apache.org Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch. Good point about ZK logs , I do see the following exceptions intermittently in the ZK log. 2014-10-27 06:54:14,621 [myid:1] - INFO [NIOServerCx

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-28 Thread S.L
hope it doesn't disappear > altogether. > > G'luck > > > -Original Message- > From: S.L [mailto:simpleliving...@gmail.com] > Sent: Monday, October 27, 2014 9:25 PM > To: solr-user@lucene.apache.org > Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1

RE: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Will Martin
, let's hope it doesn't disappear altogether. G'luck -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: Monday, October 27, 2014 9:25 PM To: solr-user@lucene.apache.org Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of sy

RE: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Will Martin
line. Fwiw. -Original Message- From: S.L [mailto:simpleliving...@gmail.com] Sent: Monday, October 27, 2014 9:25 PM To: solr-user@lucene.apache.org Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch. Good point about ZK logs , I do see the following exceptions inte

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread S.L
onday, October 27, 2014 1:42 PM > To: solr-user@lucene.apache.org > Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas > out of synch. > > > > Please find the clusterstate.json attached. > > Also in this case atleast the Shard1 replicas are out of

RE: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Will Martin
2 naïve comments, of course. - Queuing theory - Zookeeper logs. From: S.L [mailto:simpleliving...@gmail.com] Sent: Monday, October 27, 2014 1:42 PM To: solr-user@lucene.apache.org Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread S.L
gt; markus.jel...@openindex.io> > > wrote: > > > > > Hi - if there is a very large discrepancy, you could consider to purge > > the > > > smallest replica, it will then resync from the leader. > > > > > > > > > -Original message

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Shalin Shekhar Mangar
eader. > > > > > > -Original message- > > > From:S.L > > > Sent: Monday 27th October 2014 16:41 > > > To: solr-user@lucene.apache.org > > > Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 > replicas > > out of synch.

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread S.L
iginal message- > > From:S.L > > Sent: Monday 27th October 2014 16:41 > > To: solr-user@lucene.apache.org > > Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas > out of synch. > > > > Markus, > > > > I would like to ignor

RE: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Markus Jelsma
t; > > From:S.L > > > Sent: Monday 27th October 2014 16:25 > > > To: solr-user@lucene.apache.org > > > Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas > > out of synch. > > > > > > Thank Otis, > > > > >

RE: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Markus Jelsma
olved some versions ago but we are still seeing it sometimes too, there > > is nothing to see in the logs. We ignore it and just reindex. > > > > -Original message- > >> From:S.L > >> Sent: Monday 27th October 2014 16:25 > >> To: solr-user@

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread S.L
are still seeing it sometimes too, there > is nothing to see in the logs. We ignore it and just reindex. > > -Original message- > > From:S.L > > Sent: Monday 27th October 2014 16:25 > > To: solr-user@lucene.apache.org > > Subject: Re: Heavy Multi-threaded i

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Michael Della Bitta
logs. We ignore it and just reindex. -Original message- From:S.L Sent: Monday 27th October 2014 16:25 To: solr-user@lucene.apache.org Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch. Thank Otis, I have checked the logs , in my case the de

RE: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Markus Jelsma
5 > To: solr-user@lucene.apache.org > Subject: Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out > of synch. > > Thank Otis, > > I have checked the logs , in my case the default catalina.out and I dont > see any OOMs or , any other exceptions. > >

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread S.L
Thank Otis, I have checked the logs , in my case the default catalina.out and I dont see any OOMs or , any other exceptions. What others metrics do you suggest ? On Mon, Oct 27, 2014 at 9:26 AM, Otis Gospodnetic < otis.gospodne...@gmail.com> wrote: > Hi, > > You may simply be overwhelming your

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Otis Gospodnetic
Hi, You may simply be overwhelming your cluster-nodes. Have you checked various metrics to see if that is the case? Otis -- Monitoring * Alerting * Anomaly Detection * Centralized Log Management Solr & Elasticsearch Support * http://sematext.com/ > On Oct 26, 2014, at 9:59 PM, S.L wrote: > >

Re: Heavy Multi-threaded indexing and SolrCloud 4.10.1 replicas out of synch.

2014-10-27 Thread Erick Erickson
OK, clarify a bit more what you're doing with Hadoop. Are you using the MapReduceIndexerTool? Or are your Hadoop jobs writing directly to SolrCloud? How are you measuring "out of sync"? Are you sure that you've committed? Does "out of synch" mean reporting different result counts? Different order?