[
https://issues.apache.org/jira/browse/HTTPCLIENT-2434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18116675#comment-18116675
]
Arturo Bernal commented on HTTPCLIENT-2434:
-------------------------------------------
[~loginatnine]
I have a proposed change that adds an {{evictIdleConnections(sleepTime,
maxIdleTime)}} overload to both {{HttpClientBuilder}} and
{{{}HttpAsyncClientBuilder{}}}.
The existing single-argument method keeps the current behavior, while the new
overload allows the eviction sweep interval to be configured independently of
the maximum idle time.
This does not change the default eviction policy; it only exposes the existing
{{IdleConnectionEvictor}} capability through the builders.
Would you be able to test this change with your workload and confirm whether
this is the configuration you were looking for?
https://github.com/apache/httpcomponents-client/pull/885
> Improve Idle connection management customization
> ------------------------------------------------
>
> Key: HTTPCLIENT-2434
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2434
> Project: HttpComponents HttpClient
> Issue Type: Improvement
> Components: HttpClient (async), HttpClient (classic)
> Affects Versions: 5.6.4
> Reporter: Jacques-Etienne Beaudet
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hi
> Follow-up to HTTPCLIENT-2388, as suggested by Oleg on the users list:
> [https://lists.apache.org/thread/hymc9p6srgyx4nbxgqzqz6gn036rgt8g]
> Since HTTPCLIENT-2388, HttpClientBuilder passes sleepTime = null to
> IdleConnectionEvictor, so the sweep interval becomes maxIdleTime/10 with a
> 1 second floor. It used to pass sleepTime = maxIdleTime. The sweep is now 10x
> more frequent at any value of maxIdleTime, and there is no way to set
> sleepTime through the builder.
> The process of removing idle connections is not free: closeExpired() and
> closeIdle() go
> through StrictConnPool.enumAvailable(), which holds the pool lock while
> calling PoolEntry.discardConnection(GRACEFUL), and DefaultDisposalCallback
> clamps the socket timeout to 1 second for that close. So a single slow
> close holds the pool lock for up to a second. With a 1 second
> connectionRequestTimeout we sometimes got:
> RequestFailedException: Request execution failed
> at InternalExecRuntime.acquireEndpoint(InternalExecRuntime.java:133)
> Caused by: DeadlineTimeoutException: Deadline: ..., -116 MILLISECONDS overdue
> at StrictConnPool.lease(StrictConnPool.java:216)
> I've included the stack trace below.
> Two things that could be considered:
> 1. A way to set sleepTime from the builder, for example an
> evictIdleConnections(TimeValue sleepTime, TimeValue maxIdleTime) overload on
> HttpClientBuilder and HttpAsyncClientBuilder.
> Managing the evictor ourselves does work, but tying its lifecycle to the
> client means subclassing the builder, because addCloseable is protected. This
> is the hack we ended up with:
> {code:java}
> public static class CloseableExposedHttpClientBuilder extends
> HttpClientBuilder
> {
> @Override
> public void addCloseable(Closeable closeable)
> {
> super.addCloseable(closeable);
> }
> }
> IdleConnectionEvictor evictor =
> new IdleConnectionEvictor(connectionManager, sleepTime,
> maxIdleTime);
> builder.addCloseable(() -> {
> evictor.shutdown();
> try {
> evictor.awaitTermination(Timeout.ofSeconds(1));
> } catch (InterruptedException ignored) {
> Thread.currentThread().interrupt();
> }
> });
> evictor.start();
> {code}
> 2. Raise the floor of 1 second to something a bit higher like 5 or 10
> seconds, given the contention that this process has.
>
> Thanks!
>
> Stack trace of the errors we've been seeing :
> {code:java}
> org.apache.hc.client5.http.impl.classic.RequestFailedException: Request
> execution failed
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.acquireEndpoint(InternalExecRuntime.java:133)
> at
> org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:127)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.ContentCompressionExec.execute(ContentCompressionExec.java:138)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:112)
> at
> org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
> at
> org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:185)
> at
> org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
> at
> org.apache.hc.client5.http.classic.HttpClient.executeOpen(HttpClient.java:183)
> ... application frames omitted ...
> Caused by: org.apache.hc.core5.util.DeadlineTimeoutException: Deadline:
> 2026-09-16T21:11:31.165+0000, -116 MILLISECONDS overdue
> at
> org.apache.hc.core5.util.DeadlineTimeoutException.from(DeadlineTimeoutException.java:49)
> at org.apache.hc.core5.pool.StrictConnPool.lease(StrictConnPool.java:216)
> at
> org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager.lease(PoolingHttpClientConnectionManager.java:366)
> at
> org.apache.hc.client5.http.impl.classic.InternalExecRuntime.acquireEndpoint(InternalExecRuntime.java:105)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
