Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-08 Thread Jason Gerlowski
Hey Shawn, A few answers, where I can give them. 1. It's easy to miss in the thread, but the user mentioned that they're creating their CloudSolrClient via solr URLs. 2. When you create a CloudSolrClient with a Solr URL, it's not just used to fetch the ZK connString so that it can use ZK from the

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-07 Thread Shawn Heisey
On 11/6/2018 10:06 PM, Gus Heck wrote: One thing that causes a clusterstatus call is alias resolution if the HttpClusterStateProvider is in use instead of the ZkClusterStateProvider. I've just been fixing spurious error messages generated by this in SOLR-12938. Gus, If CloudSolrClient is creat

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Gus Heck
Tomáš, One thing that causes a clusterstatus call is alias resolution if the HttpClusterStateProvider is in use instead of the ZkClusterStateProvider. I've just been fixing spurious error messages generated by this in SOLR-12938. -Gus On Tue, Nov 6, 2018 at 1:08 PM Zimmermann, Thomas < tzimmerm.

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Zimmermann, Thomas
Hi Shawn, We¹re equally impressed by how well the server is handling it. We¹re using Sematext for monitoring and the load on the box has been steady under 1 and not entering a swap state memory wise. We are 100% certain the traffic is coming from the 3 web hosts running this code. We have put som

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Tomáš Hampl
This error comes every request, in solr client or if i call url in chrome browser or curl from console. I have no replicas actually for this test but it is NRT type. There is no writes or another reads on this server (solr cloud) completely isolated. (version 7.5 single docker container) I have 6

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Shawn Heisey
On 11/6/2018 10:12 AM, Zimmermann, Thomas wrote: Shawn - Server performance is fine and request time are great. We are tolerating the level of traffic, but the server that is taking all the hits is obviously performing a bit slower than the others. Response times are under 5MS avg for queries on

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Zimmermann, Thomas
I should mention I¹m also hanging out in the Solr IRC Channel today under the nick ³apatheticnow² if anyone wants to follow up in real time during business hours EST. On 11/6/18, 11:39 AM, "Shawn Heisey" wrote: >On 11/6/2018 9:06 AM, Zimmermann, Thomas wrote: >> For example - 75k request per min

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Zimmermann, Thomas
Erik - This box did have all the leaders for the dozen or so collections we have when the cloud spun up. We were able to force the leaders for other cores onto other nodes using the apis, but did not see this traffic load migrate to the new hosts when leadership changed. All nodes are NRT. The re

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Shawn Heisey
On 11/6/2018 9:06 AM, Zimmermann, Thomas wrote: For example - 75k request per minute going to this one box, and 3.5k RPM to all other nodes in the cloud. All of those extra requests on the one box are "/solr/admin/collections?collection=collectionName&action=CLUSTERSTATUS&wt=javabin&version=2"

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Jason Gerlowski
My understanding was that we always tried to use the cached version of this information until either (a) Solr responds in a way that indicates our cache is out of date, or (b) the TTL on the cache entry expires. Though there might very well be a code path that behaves differently as Erick suggests

Re: CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Erick Erickson
Is the box you're seeing this on the Overseer? Or is it in any other way "special", like has all the leaders? And I'm assuming all these are NRT replicas, not TLOG or PULL. What are you doing when these occur? Queries? Updates? If you're doing updates, are these coincident with each request? Each

CloudSolrClient produces tons of CLUSTERSTATUS commands against single server in Cloud

2018-11-06 Thread Zimmermann, Thomas
Question about CloudSolrClient and CLUSTERSTATUS. We just deployed a 3 server ZK cluster and a 5 node solr cluster using the CloudSolrClient in Solr 7.4. We're seeing a TON of traffic going to one server with just cluster status commands. Every single query seems to be hitting this box for statu