Following up: This epic has mostly completed.CoreContainer & ZkController have a default HttpSolrClient & CloudSolrClient respectively. HttpSolrClient can be used to make request to specific URLs now. SolrCloudManager is only used from "cloud" classes, except for one little ugly exception in IndexSchemaFactory. SolrCloudManager is used less... good or bad is a matter of opinion but the creator of it is no longer very active. SolrClientCache is now only used when solrj-streaming is used. I'd like to see the getter gone from CoreContainer; it's deprecated at least. HttpShardHandler: no change but it's a fairly separate matter.
On Thu, Sep 5, 2024 at 3:09 PM David Smiley <[email protected]> wrote: > There are too many ways to submit an intra-cluster request within Solr. > It’s been gnawing at me so I did an audit and made recommendations: > > There are too many ways to submit an intra-cluster request within Solr. > It’s been gnawing at me so I did an audit and made recommendations: > > A. Creating new Http SolrClient (with or without Cloud) from > org.apache.solr.update.UpdateShardHandler#getDefaultHttpClient — very > popular, 14 usages. > Recommendation: This is a poor place for such a client; will be moved > shortly in SOLR-16503 to CoreContainer and return an Http2SolrClient > instead of HttpClient. > > B. org.apache.solr.client.solrj.cloud.SolrCloudManager#request (default > impl is essentially a CloudLegacySolrClient). 3 usages. > I’m suspicious we need this; either embrace or abandon IMO. > Recommendation: embrace but migrate to getCloudSolrClient and add > getHttpSolrClient too, the latter sourced from CoreContainer. Then allows > requestAsync usage and hopefully will also have a URL-specific method for > SOLR-17256 > right next to it, there’s an httpRequest method that is now unused. > Recommendation: remove > > C. org.apache.solr.update.UpdateShardHandler#getUpdateOnlyHttpClient for > “updates” only. 2 usages > > D. org.apache.solr.update.UpdateShardHandler#getRecoveryOnlyHttpClient for > “recovery” only. 4 usages. > > E. org.apache.solr.client.solrj.io.SolrClientCache wow, 86 usages!! 68 > are in solrj-streaming since this is where this thing originated from, thus > leaving 18 usages elsewhere but I’m over-counting. The unique classes > using (excluding streaming, excluding a benchmark that tests streaming too) > are 6 usages (classes). > > * Some usages are generic to a configurable SolrCloud cluster > (solrj-streaming definitely): > ReindexCollection and SolrReporter (metrics). > This is the primary purpose/value of SolrClientCache; should continue. > * Some of the other cases just need a cluster-local CloudSolrClient: > SchemaDesigner (2 classes), CollectionsRepairEventListener. > Recommendation: Use SolrCloudManager. > * The other 4 usages are to get an Http SolrClient for a URL. > Recommendation: Use CoreContainer.getDefaultHttpSolrClient moving from > UpdateShardHandler (or similar recommended in SolrCloudManager). Requires > solution to SOLR-17256 like an Http2SolrClient specific request method > specifying an additional URL parameter. > > F. org.apache.solr.handler.component.HttpShardHandler / ShardHandler. > Originally built for distributed-search, which requires supporting multiple > replicas that are equivalent to serve a request for a shard. Later a > number of Overseer/cluster commands started using/abusing it. There are > some API tech-debt / issues I see here as a result of this. Now that > Http2SolrClient has async support (it didn’t when HttpShardHandler was > first created), I wonder if some Overseer usages should use that instead. > I think it could be simpler. > Recommendation: Add javadocs warning users to avoid using it outside of > distributed search. Point out Http2SolrClient.requestAsync. > > As an aside, I want to mention that EmbeddedSolrServer is pretty cool; I > could imagine expanding its use within Solr in some cases to avoid an HTTP > layer and disconnected caller chain. Like to issue an admin call, even a > SolrCloud one (e.g. to move a shard. But it doesn't do collection > routing. It could be neat if the node-local CloudSolrClient proposed above > could somehow detect such an admin call and route through locally to the > node. > > ~ David Smiley > Apache Lucene/Solr Search Developer > http://www.linkedin.com/in/davidwsmiley >
