Re: pySolr and other Python client options for SolrCloud.
That makes perfect sense , thanks again! On Wed, Oct 1, 2014 at 10:09 PM, Shawn Heisey wrote: > On 10/1/2014 7:08 PM, S.L wrote: > > Thanks ,load balancer seems to be the preferred solution here , I have a > > topology where I have 6 Solr nodes that support 3 shards with a > replication > > factor of 2. > > > > Looks like it woul dbe better to use the load balancers for querying > > only.The question, that I have is if I go the load balancer route should > I > > be listing all the six nodes in the load balancer or only the leaders as > > identified by SolrCloud admin console?Would the load balancing solution > > also incur any additional routing of requests between the individual > nodes > > of SolrCloud that would have not happened had the python Solr client been > > zookeeper aware? > > > > Also for indexing ,which is not done from a Python client but is done > using > > Solrj, I will avoid the load balancers and do the indexing it via the > > Zookeeper route. > > If you were to send all your queries to just one server, it's my > understanding that SolrCloud will load balance the actual work across > the cloud. I have not verified this. > > For a load balancer, the minimum requirement would be to list two of the > servers, but it's probably better to list them all. Leader designations > can change, and I'm pretty sure you don't want to change your load > balancer config just because the leader changed. > > If your 3 shards are using automatic document routing, then you can send > updates to any machine in the cluster and they'll end up in the right > place. Since you're using SolrJ for updates, this is probably not > something you need to worry about. > > Thanks, > Shawn > >
Re: pySolr and other Python client options for SolrCloud.
On 10/1/2014 7:08 PM, S.L wrote: > Thanks ,load balancer seems to be the preferred solution here , I have a > topology where I have 6 Solr nodes that support 3 shards with a replication > factor of 2. > > Looks like it woul dbe better to use the load balancers for querying > only.The question, that I have is if I go the load balancer route should I > be listing all the six nodes in the load balancer or only the leaders as > identified by SolrCloud admin console?Would the load balancing solution > also incur any additional routing of requests between the individual nodes > of SolrCloud that would have not happened had the python Solr client been > zookeeper aware? > > Also for indexing ,which is not done from a Python client but is done using > Solrj, I will avoid the load balancers and do the indexing it via the > Zookeeper route. If you were to send all your queries to just one server, it's my understanding that SolrCloud will load balance the actual work across the cloud. I have not verified this. For a load balancer, the minimum requirement would be to list two of the servers, but it's probably better to list them all. Leader designations can change, and I'm pretty sure you don't want to change your load balancer config just because the leader changed. If your 3 shards are using automatic document routing, then you can send updates to any machine in the cluster and they'll end up in the right place. Since you're using SolrJ for updates, this is probably not something you need to worry about. Thanks, Shawn
Re: pySolr and other Python client options for SolrCloud.
Shawn, Thanks ,load balancer seems to be the preferred solution here , I have a topology where I have 6 Solr nodes that support 3 shards with a replication factor of 2. Looks like it woul dbe better to use the load balancers for querying only.The question, that I have is if I go the load balancer route should I be listing all the six nodes in the load balancer or only the leaders as identified by SolrCloud admin console?Would the load balancing solution also incur any additional routing of requests between the individual nodes of SolrCloud that would have not happened had the python Solr client been zookeeper aware? Also for indexing ,which is not done from a Python client but is done using Solrj, I will avoid the load balancers and do the indexing it via the Zookeeper route. Thanks. On Wed, Oct 1, 2014 at 8:42 PM, Shawn Heisey wrote: > On 10/1/2014 2:29 PM, S.L wrote: > > Right , but my query was to know if there are any Python clients which > > achieve the same thing as SolrJ , or the approach one should take when > > using Python based clients. > > If the python client can support multiple hosts and failing over between > them, then you would simply list multiple URLs. If not, then you'll > need a load balancer. I use haproxy with Solr (not in Cloud mode) for > automatic failover, and it should work equally well for SolrCloud and a > non-java client. > > It looks like Alexandre knows a lot more about it than I do ... I know > very little about python. > > Thanks, > Shawn > >
Re: pySolr and other Python client options for SolrCloud.
On 10/1/2014 2:29 PM, S.L wrote: > Right , but my query was to know if there are any Python clients which > achieve the same thing as SolrJ , or the approach one should take when > using Python based clients. If the python client can support multiple hosts and failing over between them, then you would simply list multiple URLs. If not, then you'll need a load balancer. I use haproxy with Solr (not in Cloud mode) for automatic failover, and it should work equally well for SolrCloud and a non-java client. It looks like Alexandre knows a lot more about it than I do ... I know very little about python. Thanks, Shawn
Re: pySolr and other Python client options for SolrCloud.
I know of 10 Python Solr clients. you can find the list at: https://leanpub.com/solr-clients/read#leanpub-auto-clients I don't think any of them are as complete as SolrJ, but I would look at SolrCloudPy (it has a nice console too). Regards, Alex. Personal: http://www.outerthoughts.com/ and @arafalov Solr resources and newsletter: http://www.solr-start.com/ and @solrstart Solr popularizers community: https://www.linkedin.com/groups?gid=6713853 On 1 October 2014 16:29, S.L wrote: > Right , but my query was to know if there are any Python clients which > achieve the same thing as SolrJ , or the approach one should take when > using Python based clients. > > On Wed, Oct 1, 2014 at 3:57 PM, Upayavira wrote: > >> >> >> On Wed, Oct 1, 2014, at 08:47 PM, S.L wrote: >> > Hi All, >> > >> > We recently moved from a single Solr instance to SolrCloud and we are >> > using >> > pysolr , I am wondering what options (clients) we have from Python to >> > take advantage of Zookeeper and load balancing capabilities that >> > SolrCloud >> > provides if I were to use a smart client like Solrj? >> >> Obviously SolrJ is Java, not Python. SolrJ has integration with >> Zookeeper, so when you instantiate a CloudSolrServer instance, you tell >> it where Zookeeper is, not Solr. Your app then consults Zookeeper to >> find out which Solr instance to talk to. >> >> This means you can move stuff around within your infrastructure without >> needing to tell your app, and without needing to mess with load >> balancers as that is all handled for you by the SolrJ client deciding >> which node to forward your request. >> >> Upayavira >>
Re: pySolr and other Python client options for SolrCloud.
Right , but my query was to know if there are any Python clients which achieve the same thing as SolrJ , or the approach one should take when using Python based clients. On Wed, Oct 1, 2014 at 3:57 PM, Upayavira wrote: > > > On Wed, Oct 1, 2014, at 08:47 PM, S.L wrote: > > Hi All, > > > > We recently moved from a single Solr instance to SolrCloud and we are > > using > > pysolr , I am wondering what options (clients) we have from Python to > > take advantage of Zookeeper and load balancing capabilities that > > SolrCloud > > provides if I were to use a smart client like Solrj? > > Obviously SolrJ is Java, not Python. SolrJ has integration with > Zookeeper, so when you instantiate a CloudSolrServer instance, you tell > it where Zookeeper is, not Solr. Your app then consults Zookeeper to > find out which Solr instance to talk to. > > This means you can move stuff around within your infrastructure without > needing to tell your app, and without needing to mess with load > balancers as that is all handled for you by the SolrJ client deciding > which node to forward your request. > > Upayavira >
Re: pySolr and other Python client options for SolrCloud.
On Wed, Oct 1, 2014, at 08:47 PM, S.L wrote: > Hi All, > > We recently moved from a single Solr instance to SolrCloud and we are > using > pysolr , I am wondering what options (clients) we have from Python to > take advantage of Zookeeper and load balancing capabilities that > SolrCloud > provides if I were to use a smart client like Solrj? Obviously SolrJ is Java, not Python. SolrJ has integration with Zookeeper, so when you instantiate a CloudSolrServer instance, you tell it where Zookeeper is, not Solr. Your app then consults Zookeeper to find out which Solr instance to talk to. This means you can move stuff around within your infrastructure without needing to tell your app, and without needing to mess with load balancers as that is all handled for you by the SolrJ client deciding which node to forward your request. Upayavira
pySolr and other Python client options for SolrCloud.
Hi All, We recently moved from a single Solr instance to SolrCloud and we are using pysolr , I am wondering what options (clients) we have from Python to take advantage of Zookeeper and load balancing capabilities that SolrCloud provides if I were to use a smart client like Solrj? Thanks.
