[
https://issues.apache.org/jira/browse/HTTPCLIENT-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116785#comment-18116785
]
Jacques-Etienne Beaudet commented on HTTPCLIENT-2434:
-----------------------------------------------------
While debugging, I realized that the ConnectionConfig.Builder::copy does not
propagate the idleTimeout set there,
[https://github.com/apache/httpcomponents-client/blob/master/httpclient5/src/main/java/org/apache/hc/client5/http/config/ConnectionConfig.java#L180-L185]
This seems like a bug too, it was introduced in
[https://github.com/apache/httpcomponents-client/pull/732] FWIW.
> Improve Idle connection management customization
> ------------------------------------------------
>
> Key: HTTPCLIENT-2434
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2434
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (async), HttpClient (classic)
> Affects Versions: 5.6.4
> Reporter: Jacques-Etienne Beaudet
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hi
> Follow-up to HTTPCLIENT-2388, as suggested by Oleg on the users list:
> [https://lists.apache.org/thread/hymc9p6srgyx4nbxgqzqz6gn036rgt8g]
> Since HTTPCLIENT-2388, HttpClientBuilder passes sleepTime = null to
> IdleConnectionEvictor, so the sweep interval becomes maxIdleTime/10 with a
> 1 second floor. It used to pass sleepTime = maxIdleTime. The sweep is now 10x
> more frequent at any value of maxIdleTime, and there is no way to set
> sleepTime through the builder.
> The process of removing idle connections is not free: closeExpired() and
> closeIdle() go
> through StrictConnPool.enumAvailable(), which holds the pool lock while
> calling PoolEntry.discardConnection(GRACEFUL), and DefaultDisposalCallback
> clamps the socket timeout to 1 second for that close. So a single slow
> close holds the pool lock for up to a second. With a 1 second
> connectionRequestTimeout we sometimes got:
> RequestFailedException: Request execution failed
> at InternalExecRuntime.acquireEndpoint(InternalExecRuntime.java:133)
> Caused by: DeadlineTimeoutException: Deadline: ..., -116 MILLISECONDS overdue
> at StrictConnPool.lease(StrictConnPool.java:216)
> I've included the stack trace below.
> Two things that could be considered:
> 1. A way to set sleepTime from the builder, for example an
> evictIdleConnections(TimeValue sleepTime, TimeValue maxIdleTime) overload on
> HttpClientBuilder and HttpAsyncClientBuilder.
> Managing the evictor ourselves does work, but tying its lifecycle to the
> client means subclassing the builder, because addCloseable is protected. This
> is the hack we ended up with:
> {code:java}
> public static class CloseableExposedHttpClientBuilder extends
> HttpClientBuilder
> {
> @Override
> public void addCloseable(Closeable closeable)
> {
> super.addCloseable(closeable);
> }
> }
> IdleConnectionEvictor evictor =
> new IdleConnectionEvictor(connectionManager, sleepTime,
> maxIdleTime);
> builder.addCloseable(() -> {
> evictor.shutdown();
> try {
> evictor.awaitTermination(Timeout.ofSeconds(1));
> } catch (InterruptedException ignored) {
> Thread.currentThread().interrupt();
> }
> });
> evictor.start();
> {code}
> 2. Raise the floor of 1 second to something a bit higher like 5 or 10
> seconds, given the contention that this process has.
>
> Thanks!
>
> Stack trace of the errors we've been seeing :
> {code:java}
> org.apache.hc.client5.http.impl.classic.RequestFailedException: Request
> execution failed
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.acquireEndpoint(InternalExecRuntime.java:133)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:127)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:138)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:112)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> ... application frames omitted ...
> Caused by: org.apache.hc.core5.util.DeadlineTimeoutException: Deadline:
> 2026-09-16T21:11:31.165+0000, -116 MILLISECONDS overdue
> at
> org.apache.hc.core5.util.DeadlineTimeoutException.from(DeadlineTimeoutException.java:49)
> at org.apache.hc.core5.pool.StrictConnPool.lease(StrictConnPool.java:216)
> at
> org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.lease(PoolingHttpClientConnectionManager.java:366)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.acquireEndpoint(InternalExecRuntime.java:105)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
