[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849166#comment-17849166 ] Sanjay Dutt commented on SOLR-16505: Done. > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Fix For: 9.7 > > Time Spent: 9h 10m > Remaining Estimate: 0h > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17849109#comment-17849109 ] David Smiley commented on SOLR-16505: - Can this be resolved again? If so, please do it Sanjay. > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Assignee: David Smiley >Priority: Major > Fix For: 9.7 > > Time Spent: 9h 10m > Remaining Estimate: 0h > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846185#comment-17846185 ] Sanjay Dutt commented on SOLR-16505: Without your support, this PR wouldn't have seen the light of day. Thank you immensely for your consistent reviews and ongoing support! > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Fix For: 9.7 > > Time Spent: 8.5h > Remaining Estimate: 0h > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17846150#comment-17846150 ] ASF subversion and git services commented on SOLR-16505: Commit 0da3d34be61c153414df9301835ae0ca8b6da0c2 in solr's branch refs/heads/branch_9x from Sanjay Dutt [ https://gitbox.apache.org/repos/asf?p=solr.git;h=0da3d34be61 ] SOLR-16505: Switch Recovery/Replication to Jetty HTTP2 (#2276) UpdateShardHandler * switch "recoveryOnlyHttpClient" to Http2SolrClient RecoveryStrategy: * Use Http2SolrClient * Simplify cancelation of prep recovery command IndexFetcher: * Use Http2SolrClient * Ensure the entire stream is consumed to avoid a connection reset Http2SolrClient: * make HttpListenerFactory configurable (used for internal purposes) - Co-authored-by: iamsanjay Co-authored-by: David Smiley (cherry picked from commit e62cb1b0066132347589b5e5ca38f38fd5e668d0) fix buildUrl > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Time Spent: 8.5h > Remaining Estimate: 0h > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17845181#comment-17845181 ] Mark Robert Miller commented on SOLR-16505: --- Yeah, probably. The idea was to isolate connection pools so that recovery requests don’t have to queue up behind updates and queries and issues in one type of pool don’t affect another - eg if connections are not fully read and take a while to timeout frequently in the query pool, that won’t affect recovery. > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Time Spent: 8.5h > Remaining Estimate: 0h > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844985#comment-17844985 ] David Smiley commented on SOLR-16505: - [~markrmiller] , based on the callers of org.apache.solr.update.UpdateShardHandler#getRecoveryOnlyHttpClient, "recovery" refers to IndexFetcher and RecoveryStrategy. Should this be expanded slightly to include PeerSync (see PeerSyncWithLeader using the "default" client), or not if there's a distinct between index replication and peer-sync with respect to possible configuration needs like timeouts and threads. RecoveryStrategy can initiate peer sync, after all. If you agree, shouldn't this include SyncStrategy, which is related to PeerSync? At a glance, it's not clear why SyncStrategy and RecoveryStrategy aren't the same thing. Javadocs could be added to elaborate on what the scope of this recovery client is for, and also why it exists in the first place. There's a sad "bus factor" in this part of Solr, I think, so I really appreciate it when you share your insights. BTW, I debate UpdateShardHandler's name [here|https://github.com/apache/solr/pull/2351#discussion_r1595413371]; you may have an opinion. > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Time Spent: 8.5h > Remaining Estimate: 0h > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17844857#comment-17844857 ] ASF subversion and git services commented on SOLR-16505: Commit e62cb1b0066132347589b5e5ca38f38fd5e668d0 in solr's branch refs/heads/main from Sanjay Dutt [ https://gitbox.apache.org/repos/asf?p=solr.git;h=e62cb1b0066 ] SOLR-16505: Switch Recovery/Replication to Jetty HTTP2 (#2276) UpdateShardHandler * switch "recoveryOnlyHttpClient" to Http2SolrClient RecoveryStrategy: * Use Http2SolrClient * Simplify cancelation of prep recovery command IndexFetcher: * Use Http2SolrClient * Ensure the entire stream is consumed to avoid a connection reset Http2SolrClient: * make HttpListenerFactory configurable (used for internal purposes) - Co-authored-by: iamsanjay Co-authored-by: David Smiley > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Time Spent: 8.5h > Remaining Estimate: 0h > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17829251#comment-17829251 ] David Smiley commented on SOLR-16505: - Could be useful to distinguish two changes here – one is the change in HttpClient, but the other is Using a SolrClient vs HttpClient. So you could temporarily have the SolrClients created here based on Apache HttpClient and see if you get the same issues. If the issues persist, then there's probably an issue around sharing an HttpClient that gets closed in one SolrClient but is still referenced in a non-closed other SolrClient. Or simply using a SolrClient after closure was done; maybe it's a field. If the issues don't persist, then maybe this means the Apache HttpClient is tolerant to extra close calls but Jetty HttpClient isn't. > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > Time Spent: 3h 50m > Remaining Estimate: 0h > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[
https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828941#comment-17828941
]
Sanjay Dutt commented on SOLR-16505:
*IndexFetcher Exception:*
IndexFetcher calls fetchFile(), to download the file via fetch(), in case of
failure it retries by calling fetch() again.
However every calls to *fetch* would call cleanup() which close the directory,
and therefore another to call to fetch throws the exception
*AlreadyClosedException.*
o.a.s.h.IndexFetcher Error in fetching file: _1h.fdx (downloaded 0 of 64 bytes)
{code:java}
2> => org.apache.lucene.store.AlreadyClosedException: Already
closed: MockIndexOutputWrapper(ByteBuffersDirectory output (file=_1h.fdx))
2> at
org.apache.lucene.tests.store.MockIndexOutputWrapper.ensureOpen(MockIndexOutputWrapper.java:124)
2> org.apache.lucene.store.AlreadyClosedException: Already closed:
MockIndexOutputWrapper(ByteBuffersDirectory output (file=_1h.fdx))
2> at
org.apache.lucene.tests.store.MockIndexOutputWrapper.ensureOpen(MockIndexOutputWrapper.java:124)
~[lucene-test-framework-9.9.2.jar:9.9.2
a2939784c4ca60bc28bf488b5479c02fc2e5e22c - 2024-01-25 09:51:09]
{code}
> Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
> --
>
> Key: SOLR-16505
> URL: https://issues.apache.org/jira/browse/SOLR-16505
> Project: Solr
> Issue Type: Sub-task
>Reporter: David Smiley
>Priority: Major
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> This method and its callers (only RecoveryStrategy) should be converted to a
> Jetty HTTP2 client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[
https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17828940#comment-17828940
]
Sanjay Dutt commented on SOLR-16505:
Three different exceptions I have seen so far: Two related to Http2SolrClient
and one related to IndexFetcher.
*Http2SolrClient Exceptions:*
{code:java}
Caused by: java.lang.IllegalStateException: session closed
2> at
org.eclipse.jetty.http2.HTTP2Session$StreamsState.reserveSlot(HTTP2Session.java:2318)
~[http2-common-10.0.20.jar:10.0.20]
2> at
org.eclipse.jetty.http2.HTTP2Session$StreamsState.newLocalStream(HTTP2Session.java:2179)
~[http2-common-10.0.20.jar:10.0.20]
2> at
org.eclipse.jetty.http2.HTTP2Session.newStream(HTTP2Session.java:645)
~[http2-common-10.0.20.jar:10.0.20]
2> at
org.eclipse.jetty.http2.client.http.HttpSenderOverHTTP2.sendHeaders(HttpSenderOverHTTP2.java:129)
~[http2-http-client-transport-10.0.20.jar:10.0.20]{code}
{code:java}
2> Caused by: java.nio.channels.AsynchronousCloseException
2> at
org.eclipse.jetty.http2.client.http.HttpConnectionOverHTTP2.close(HttpConnectionOverHTTP2.java:217)
~[http2-http-client-transport-10.0.20.jar:10.0.20]
2> at
org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2.onClose(HttpClientTransportOverHTTP2.java:175)
~[http2-http-client-transport-10.0.20.jar:10.0.20]
2> at
org.eclipse.jetty.http2.client.http.HttpClientTransportOverHTTP2$SessionListenerPromise.onClose(HttpClientTransportOverHTTP2.java:194)
~[http2-http-client-transport-10.0.20.jar:10.0.20]
{code}
For instance, IndexFetcher downloading list of files, then getStream gets
failed for one of the file.
> Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
> --
>
> Key: SOLR-16505
> URL: https://issues.apache.org/jira/browse/SOLR-16505
> Project: Solr
> Issue Type: Sub-task
>Reporter: David Smiley
>Priority: Major
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> This method and its callers (only RecoveryStrategy) should be converted to a
> Jetty HTTP2 client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[
https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17818237#comment-17818237
]
Sanjay Dutt commented on SOLR-16505:
In {{{}RecoveryStrategy.java{}}}, sendPrepRecoveryCmd sending async request
using HttpUriRequest, After replacing it with newer client there is no option
to create HttpUriRequest. However, we can send asyncRequest using
Http2SolrClient which use executor internally. Upon calling, {{withExecutor()}}
to set the executor, it did not work. Further inspection revealed that
Http2SolrClient only set executor while creating new HttpClient, In case, if
builder is initialized with existing client there is no way to set the
executor. I added the new code to tackle that issue.
> Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
> --
>
> Key: SOLR-16505
> URL: https://issues.apache.org/jira/browse/SOLR-16505
> Project: Solr
> Issue Type: Sub-task
>Reporter: David Smiley
>Priority: Major
>
> This method and its callers (only RecoveryStrategy) should be converted to a
> Jetty HTTP2 client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[ https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17816352#comment-17816352 ] David Smiley commented on SOLR-16505: - The word "executor" in this file is an overloaded word. Generally we refer to an ExecutorService. But the HttpClients don't use an ExecutorService (ignoring that the Solr Http2SolrClient using Jetty can use one for async requests; isn't relevant in UpdateShardHandler). The Apache one does use an httpRequestExecutor but this is *not* an ExecutorService; it's a mechanism to decorate the HTTP requests, like for instrumentation. It's not used for multi-threaded execution. There's a Jetty side equivalent you will see in this file: InstrumentedHttpListenerFactory. > Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2 > -- > > Key: SOLR-16505 > URL: https://issues.apache.org/jira/browse/SOLR-16505 > Project: Solr > Issue Type: Sub-task >Reporter: David Smiley >Priority: Major > > This method and its callers (only RecoveryStrategy) should be converted to a > Jetty HTTP2 client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[
https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815302#comment-17815302
]
Sanjay Dutt commented on SOLR-16505:
I came across updateExecutor object in UpdateShardHandler.java.
{code:java}
private ExecutorService updateExecutor =
new ExecutorUtil.MDCAwareThreadPoolExecutor(
0,
Integer.MAX_VALUE,
60L,
TimeUnit.SECONDS,
new SynchronousQueue<>(),
new SolrNamedThreadFactory("updateExecutor"),
// the Runnable added to this executor handles all exceptions so we disable
stack trace
// collection as an optimization
// see SOLR-11880 for more details
false);{code}
*I wonder why we did not used this executor while initializing the
`Http2SolrClient.Builder()` for*
{*}updateOnlyClient{*}. Instead used, the default executor provided by the
builder. But then we have two Executors – one for Update requests and another
one internal use. But earlier, before introducing http2 client for update, we
had one executor for both.
Default Executor from builder
{code:java}
this.executor =
new ExecutorUtil.MDCAwareThreadPoolExecutor(
32, 256, 60, TimeUnit.SECONDS, queue, new SolrNamedThreadFactory("h2sc"));{code}
> Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
> --
>
> Key: SOLR-16505
> URL: https://issues.apache.org/jira/browse/SOLR-16505
> Project: Solr
> Issue Type: Sub-task
>Reporter: David Smiley
>Priority: Major
>
> This method and its callers (only RecoveryStrategy) should be converted to a
> Jetty HTTP2 client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[
https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17815298#comment-17815298
]
Sanjay Dutt commented on SOLR-16505:
Adding more info to this ticket. Here discussing history of
UpdateShardHandler.java.
Initially we had only one client for all propose -- default, update and
recovery.
{code:java}
private final CloseableHttpClient client;
private final InstrumentedPoolingHttpClientConnectionManager
clientConnectionManager;
private final InstrumentedHttpRequestExecutor httpRequestExecutor;
{code}
h3. SOLR-12293: Updates need to use their own connection pool to maintain
connection reuse and prevent spurious recoveries
Additional http client for update added because of the pool being shared too
broadly. For more info https://issues.apache.org/jira/browse/SOLR-12293
New code Added:
{code:java}
private final CloseableHttpClient updateOnlyClient;
private final InstrumentedPoolingHttpClientConnectionManager
updateOnlyConnectionManager;
{code}
And I believe, we have initialized this connectionManager with same
configuration as what we were using for clientConnectionManager
{code:java}
updateOnlyConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
updateOnlyConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());
defaultConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
defaultConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());{code}
Note :- And the clientConnectionManager named changed to
defaultConnectionManager
*So far no new client introduced for recovery*
SOLR-12314: Use http timeout's defined in solr.xml for creating
ConcurrentUpdateSolrClient during indexing requests between leader and replica
Introduced two new parameters that being used in configuring Http clients --
default and update.
{code:java}
clientParams.set(HttpClientUtil.PROP_SO_TIMEOUT,
cfg.getDistributedSocketTimeout());
clientParams.set(HttpClientUtil.PROP_CONNECTION_TIMEOUT,
cfg.getDistributedConnectionTimeout());
...
updateOnlyClient = HttpClientUtil.createClient(clientParams,
updateOnlyConnectionManager, false, httpRequestExecutor);
defaultClient = HttpClientUtil.createClient(clientParams,
defaultConnectionManager, false, httpRequestExecutor);
{code}
h3. SOLR-12801: Make massive improvements to the tests
*Now here the new http client and connection manager introduced for the
recoveryOps.*
Till this point, there are three separate connections for default, update and
recovery. And I would like to point out that all of them {*}using similar
configuration{*}. For more info
https://github.com/apache/solr/blob/75b183196798232aa6f2dcb117f309119053/solr/core/src/java/org/apache/solr/update/UpdateShardHandler.java
{code:java}
updateOnlyConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
updateOnlyConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());
recoveryOnlyConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
recoveryOnlyConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());
defaultConnectionManager.setMaxTotal(cfg.getMaxUpdateConnections());
defaultConnectionManager.setDefaultMaxPerRoute(cfg.getMaxUpdateConnectionsPerHost());{code}
{code:java}
httpRequestExecutor = new InstrumentedHttpRequestExecutor(metricNameStrategy);
updateOnlyClient = HttpClientUtil.createClient(clientParams,
updateOnlyConnectionManager, false, httpRequestExecutor);
recoveryOnlyClient = HttpClientUtil.createClient(clientParams,
recoveryOnlyConnectionManager, false, httpRequestExecutor);
defaultClient = HttpClientUtil.createClient(clientParams,
defaultConnectionManager, false, httpRequestExecutor);{code}
There is one more method created which would return
*recoveryOnlyConnectionManager* that is being used by IndexFetcher.java. Before
this change IndexFetcher using defaultConnectionManager from UpdateShardHandler.
{code:java}
- return HttpClientUtil.createClient(httpClientParams,
core.getCoreContainer().getUpdateShardHandler().getDefaultConnectionManager(),
true);
+ return HttpClientUtil.createClient(httpClientParams,
core.getCoreContainer().getUpdateShardHandler().getRecoveryOnlyConnectionManager(),
true);{code}
h3. Merge jira/http2 branch to
master(https://github.com/apache/solr/commit/f80e8e11672d31c6e12069d2bd12a28b92e5a336)
Replace the update old http client with Http2SolrClient and we do not need the
PoolingManager anymore, instead we used Http2SolrClient.Builder#withHttpClient
> Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
> --
>
> Key: SOLR-16505
> URL: https://issues.apache.org/jira/browse/SOLR-16505
> Project: Solr
> Issue Type: Sub-task
>Reporter: David Smiley
>Priority: Major
>
> This method and
[jira] [Commented] (SOLR-16505) Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
[
https://issues.apache.org/jira/browse/SOLR-16505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17714633#comment-17714633
]
Bence Szabó commented on SOLR-16505:
I was looking at this issue and I think I could easily piggyback on the
solution how we use {{Http2SolrClient}} for the {{updateOnlyClient}} but what
I could not figure out is do you happen to know what is the purpose of having 3
different clients here and if they also should differ in their configuration or
only in their usage? The reason why I am asking is that I was not able to find
any difference between the configuration of the {{recoveryOnlyClient}} and the
{{defaultClient}} and because of this I am not sure how these should differ
from the {{{}updateOnlyClient{}}}.
Maybe [~epugh] if you have the time and knowledge to help me out, any insights
would be appreciated.
> Switch UpdateShardHandler.getRecoveryOnlyHttpClient to Jetty HTTP2
> --
>
> Key: SOLR-16505
> URL: https://issues.apache.org/jira/browse/SOLR-16505
> Project: Solr
> Issue Type: Sub-task
>Reporter: David Smiley
>Priority: Major
>
> This method and its callers (only RecoveryStrategy) should be converted to a
> Jetty HTTP2 client.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
