[
https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17815298#comment-17815298
]
Sanjay Dutt commented on SOLR-16505:
------------------------------------
Adding more info to this ticket. Here discussing history of
UpdateShardHandler.java.
Initially we had only one client for all propose -- default, update and
recovery.
{code:java}
private final CloseableHttpClient client;
private final InstrumentedPoolingHttpClientConnectionManager
clientConnectionManager;
private final InstrumentedHttpRequestExecutor httpRequestExecutor;
{code}
h3. SOLR-12293: Updates need to use their own connection pool to maintain
connection reuse and prevent spurious recoveries
Additional http client for update added because of the pool being shared too
broadly. For more info https://issues.apache.org/jira/browse/SOLR-12293
New code Added:
{code:java}
private final CloseableHttpClient updateOnlyClient;
private final InstrumentedPoolingHttpClientConnectionManager
updateOnlyConnectionManager;
{code}
And I believe, we have initialized this connectionManager with same
configuration as what we were using for clientConnectionManager
{code:java}
updateOnlyConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
updateOnlyConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());
defaultConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
defaultConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());{code}
Note :- And the clientConnectionManager named changed to
defaultConnectionManager
*So far no new client introduced for recovery*
SOLR-12314: Use http timeout's defined in solr.xml for creating
ConcurrentUpdateSolrClient during indexing requests between leader and replica
Introduced two new parameters that being used in configuring Http clients --
default and update.
{code:java}
clientParams.set(HttpClientUtil.PROP_SO_TIMEOUT,
cfg.getDistributedSocketTimeout());
clientParams.set(HttpClientUtil.PROP_CONNECTION_TIMEOUT,
cfg.getDistributedConnectionTimeout());
...
updateOnlyClient = HttpClientUtil.createClient(clientParams,
updateOnlyConnectionManager, false, httpRequestExecutor);
defaultClient = HttpClientUtil.createClient(clientParams,
defaultConnectionManager, false, httpRequestExecutor);
{code}
h3. SOLR-12801: Make massive improvements to the tests
*Now here the new http client and connection manager introduced for the
recoveryOps.*
Till this point, there are three separate connections for default, update and
recovery. And I would like to point out that all of them {*}using similar
configuration{*}. For more info
https://github.com/apache/solr/blob/75b183196798232aa6f2dcaaaab117f309119053/solr/core/src/java/org/apache/solr/update/UpdateShardHandler.java
{code:java}
updateOnlyConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
updateOnlyConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());
recoveryOnlyConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
recoveryOnlyConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());
defaultConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
defaultConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());{code}
{code:java}
httpRequestExecutor = new InstrumentedHttpRequestExecutor(metricNameStrategy);
updateOnlyClient = HttpClientUtil.createClient(clientParams,
updateOnlyConnectionManager, false, httpRequestExecutor);
recoveryOnlyClient = HttpClientUtil.createClient(clientParams,
recoveryOnlyConnectionManager, false, httpRequestExecutor);
defaultClient = HttpClientUtil.createClient(clientParams,
defaultConnectionManager, false, httpRequestExecutor);{code}
There is one more method created which would return
*recoveryOnlyConnectionManager* that is being used by IndexFetcher.java. Before
this change IndexFetcher using defaultConnectionManager from UpdateShardHandler.
{code:java}
- return HttpClientUtil.createClient(httpClientParams,
core.getCoreContainer().getUpdateShardHandler().getDefaultConnectionManager(),
true);
+ return HttpClientUtil.createClient(httpClientParams,
core.getCoreContainer().getUpdateShardHandler().getRecoveryOnlyConnectionManager(),
true);{code}
h3. Merge jira/http2 branch to
master(https://github.com/apache/solr/commit/f80e8e11672d31c6e12069d2bd12a28b92e5a336)
Replace the update old http client with Http2SolrClient and we do not need the
PoolingManager anymore, instead we used Http2SolrClient.Builder#withHttpClient
> Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
> ------------------------------------------------------------------
>
> Key: SOLR-16505
> URL: https://issues.apache.org/jira/browse/SOLR-16505
> Project: Solr
> Issue Type: Sub-task
> Reporter: David Smiley
> Priority: Major
>
> This method and its callers (only RecoveryStrategy) should be converted to a
> Jetty HTTP2 client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
