Federico Mariani created CAMEL-27053:
----------------------------------------
Summary: camel-http - connectionIdleTimeout and
validateAfterInactivity options, so the pool does not reuse a connection a NAT
dropped or the server closed
Key: CAMEL-27053
URL: https://issues.apache.org/jira/browse/CAMEL-27053
Project: Camel
Issue Type: Improvement
Components: camel-http
Reporter: Federico Mariani
Fix For: 4.24.0
The connection pool that {{HttpComponent}} creates for each endpoint only takes
the age of a connection from the component: {{createConnectionManager}} sets
{{ConnectionConfig.timeToLive}} from {{connectionTimeToLive}} and nothing else
({{components/camel-http/src/main/java/org/apache/camel/component/http/HttpComponent.java}},
around line 713 on main). HttpClient 5.6 has two more settings on that
{{ConnectionConfig}} that cannot be reached from Camel without replacing the
whole {{clientConnectionManager}}:
* {{idleTimeout}} (new in HttpClient 5.6): a connection idle in the pool for
longer is discarded when it is next leased, whatever {{Keep-Alive}} timeout the
server announced. No background thread, works per lease.
* {{validateAfterInactivity}} (HttpClient default 2 seconds): a connection idle
for longer is checked (a 1 ms read) before reuse, so one the server closed is
discarded.
The {{httpClient.}} and {{httpConnection.}} prefixes bind
{{HttpClientBuilder}}, {{RequestConfig.Builder}} and {{SocketConfig.Builder}},
not {{ConnectionConfig}}. The http component page (section "Pooled connections
closed by the server or by a NAT", added by
https://github.com/apache/camel/pull/30064) already says that
{{validateAfterInactivity}} is not exposed.
h3. Why it matters
Found with fault-injection tests (a proxy that drops idle connections silently,
as cloud NATs, firewalls and load balancers do after a few minutes without FIN
or RST):
* *Silent idle drop*: the next request on the dropped pooled connection waits
for the read timeout (3 minutes with HttpClient's defaults) and fails with
{{SocketTimeoutException}}. HttpClient never retries an
{{InterruptedIOException}}, not even for {{GET}}. The documented workaround,
{{httpClient.connectionKeepAlive}}, only applies when the response has no
{{Keep-Alive: timeout=N}} header: {{DefaultConnectionKeepAliveStrategy}} takes
the server's value first, so a server that announces a longer timeout than the
NAT's keeps the problem. {{connectionTimeToLive}} works, but also recycles busy
connections.
* *Server closed the idle connection*: a request sent less than 2 seconds after
the connection's last use, on a connection the server has closed in the
meantime, fails with {{NoHttpResponseException}}. HttpClient retries it only
for idempotent methods, so a {{POST}} fails.
h3. Steps to reproduce
Test HTTP server ({{ServerBootstrap}}) behind a small in-test TCP proxy;
{{responseTimeout=1000}} on the endpoint.
# Silent drop: {{GET}} (succeeds, one pooled connection); the proxy turns the
open connection silent (reads and discards, never answers, never closes); wait
1 s; {{GET}} again. Server responses carry {{Keep-Alive: timeout=300}};
endpoint with {{httpClient.connectionKeepAlive=500}}.
# Server close: {{POST}} (succeeds); the proxy closes the open connection;
{{POST}} again right away.
h3. Actual (main, HttpClient 5.6.5)
# {{java.net.SocketTimeoutException: Read timed out}} (with the defaults it
would wait 3 minutes). Without the {{Keep-Alive}} header,
{{httpClient.connectionKeepAlive=500}} avoids it.
# {{org.apache.hc.core5.http.NoHttpResponseException: localhost:<port> failed
to respond}}
h3. Expected
Component options that set these two values on the pool the component creates,
off / unchanged by default:
* {{connectionIdleTimeout}} (milliseconds, 0 = no limit, the default): maps to
{{ConnectionConfig.setIdleTimeout}}. Set below the idle timeout of the NAT,
firewall or load balancer, the request opens a new connection instead of using
a dead one.
* {{validateAfterInactivity}} (milliseconds, default 2000 = HttpClient's
default): maps to {{ConnectionConfig.setValidateAfterInactivity}}; 0 checks the
connection before each reuse.
Both next to {{connectionTimeToLive}} (component options, like it), documented
in the timeouts section of the http component page.
{{HttpClientBuilder.evictIdleConnections}} is not a good fit: it starts a
thread per HttpClient (Camel builds one per endpoint), has no effect with a
shared connection manager, and still lets a connection be reused between two
runs of the evictor.
_Claude Code on behalf of Croway_
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)