Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-19 Thread John Bickerstaff
When combining a load balancer with SolrCloud, the handler definitions in solrconfig.xml should set preferLocalShards to true (which Tom mentioned) Thanks Shawn! I was wondering where to set this... Yup - my IT guy is sharp, sharp, sharp -- nice to get this confirmation from the list... On

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-19 Thread John Bickerstaff
@Charlie It's easy to do and wow does it save time and database resources... I've built a Spring Boot Micro-services architecture that also registers in Zookeeper. One micro-service pulls from the original data source and pushes to Kafka. The second micro-service pulls from Kafka into SOLR.

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-19 Thread Shawn Heisey
On 4/18/2016 11:22 AM, John Bickerstaff wrote: > So - my IT guy makes the case that we don't really need Zookeeper / Solr > Cloud... > I'm biased in terms of using the most recent functionality, but I'm aware > that bias is not necessarily based on facts and want to do my due > diligence... > >

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-19 Thread Charlie Hull
On 18/04/2016 18:22, John Bickerstaff wrote: So - my IT guy makes the case that we don't really need Zookeeper / Solr Cloud... He may be right - we're serving static data (changes to the collection occur only 2 or 3 times a year and are minor) We probably could have 3 or 4 Solr nodes running

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread John Bickerstaff
inging up a whole new solr on a whole new server - > > > SolrCloud already virtualizes this, and so I can make up a random > > > collection name that doesn't conflict, and create the thing, and smoke > test > > > with it. I know that standard practice is to bring up all new no

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread Erick Erickson
erstaff [mailto:j...@johnbickerstaff.com] > > Sent: Monday, April 18, 2016 1:23 PM > > To: solr-user@lucene.apache.org > > Subject: Re: Verifying - SOLR Cloud replaces load balancer? > > > > So - my IT guy makes the case that we don't really need Zookeeper / Solr > >

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread John Bickerstaff
om] > Sent: Monday, April 18, 2016 1:23 PM > To: solr-user@lucene.apache.org > Subject: Re: Verifying - SOLR Cloud replaces load balancer? > > So - my IT guy makes the case that we don't really need Zookeeper / Solr > Cloud... > > He may be right - we're serving static data (changes

RE: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread Davis, Daniel (NIH/NLM) [C]
't see why this is needed. -Original Message- From: John Bickerstaff [mailto:j...@johnbickerstaff.com] Sent: Monday, April 18, 2016 1:23 PM To: solr-user@lucene.apache.org Subject: Re: Verifying - SOLR Cloud replaces load balancer? So - my IT guy makes the case that we don't really need

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread John Bickerstaff
So - my IT guy makes the case that we don't really need Zookeeper / Solr Cloud... He may be right - we're serving static data (changes to the collection occur only 2 or 3 times a year and are minor) We probably could have 3 or 4 Solr nodes running in non-Cloud mode -- each configured the same

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread Tom Evans
On Mon, Apr 18, 2016 at 3:52 PM, John Bickerstaff wrote: > Thanks all - very helpful. > > @Shawn - your reply implies that even if I'm hitting the URL for a single > endpoint via HTTP - the "balancing" will still occur across the Solr Cloud > (I understand the caveat

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread John Bickerstaff
Excellent - thanks! On Mon, Apr 18, 2016 at 9:16 AM, Erick Erickson wrote: > Your summary pretty much nails it. > > For (b) note that CloudSolrClient uses an internal software load > balancer to distribute queries, FWIW. > > > > On Mon, Apr 18, 2016 at 7:52 AM, John

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread Erick Erickson
Your summary pretty much nails it. For (b) note that CloudSolrClient uses an internal software load balancer to distribute queries, FWIW. On Mon, Apr 18, 2016 at 7:52 AM, John Bickerstaff wrote: > Thanks all - very helpful. > > @Shawn - your reply implies that even

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread John Bickerstaff
Thanks all - very helpful. @Shawn - your reply implies that even if I'm hitting the URL for a single endpoint via HTTP - the "balancing" will still occur across the Solr Cloud (I understand the caveat about that single endpoint being a potential point of failure). I just want to verify that I'm

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread Shawn Heisey
On 4/17/2016 10:35 PM, John Bickerstaff wrote: > My prior use of SOLR in production was pre SOLR cloud. We put a > round-robin load balancer in front of replicas for searching. > > Do I understand correctly that a load balancer is unnecessary with SOLR > Cloud? I. E. -- SOLR and Zookeeper will

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread Jack Krupansky
SolrJ does indeed provide load balancing via CloudSolrClient which uses LBHttpSolrClient: https://lucene.apache.org/solr/5_5_0/solr-solrj/org/apache/solr/client/solrj/impl/CloudSolrClient.html https://lucene.apache.org/solr/5_5_0/solr-solrj/org/apache/solr/client/solrj/impl/LBHttpSolrClient.html

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-18 Thread Jaroslaw Rozanski
Hi, How are you executing searches? I am asking because if you search using Solr client, for example SolrJ - ie. create instance of CloudSolrClient, and not directly via HTTP endpoint, it will provided load-balancing (last time I checked it picks random non-stale node). Thanks, Jarek On Mon,

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-17 Thread John Bickerstaff
Thanks, so on the matter of indexing -- while I could isolate a cloud replica from queries by not including it in the load balancer's list... ... I cannot isolate any of the replicas from an indexing perspective by a similar strategy because the SOLR leader decides who does indexing? Or do all

Re: Verifying - SOLR Cloud replaces load balancer?

2016-04-17 Thread Walter Underwood
No, Zookeeper is used for managing the locations of replicas and the leader for indexing. Queries should still be distributed with a load balancer. Queries do NOT go through Zookeeper. wunder Walter Underwood wun...@wunderwood.org http://observer.wunderwood.org/ (my blog) > On Apr 17, 2016,

Verifying - SOLR Cloud replaces load balancer?

2016-04-17 Thread John Bickerstaff
My prior use of SOLR in production was pre SOLR cloud. We put a round-robin load balancer in front of replicas for searching. Do I understand correctly that a load balancer is unnecessary with SOLR Cloud? I. E. -- SOLR and Zookeeper will balance the load, regardless of which replica's URL is