Federico Mariani created CAMEL-27053:
----------------------------------------

             Summary: camel-http - connectionIdleTimeout and 
validateAfterInactivity options, so the pool does not reuse a connection a NAT 
dropped or the server closed
                 Key: CAMEL-27053
                 URL: https://issues.apache.org/jira/browse/CAMEL-27053
             Project: Camel
          Issue Type: Improvement
          Components: camel-http
            Reporter: Federico Mariani
             Fix For: 4.24.0


The connection pool that {{HttpComponent}} creates for each endpoint only takes 
the age of a connection from the component: {{createConnectionManager}} sets 
{{ConnectionConfig.timeToLive}} from {{connectionTimeToLive}} and nothing else 
({{components/camel-http/src/main/java/org/apache/camel/component/http/HttpComponent.java}},
 around line 713 on main). HttpClient 5.6 has two more settings on that 
{{ConnectionConfig}} that cannot be reached from Camel without replacing the 
whole {{clientConnectionManager}}:

* {{idleTimeout}} (new in HttpClient 5.6): a connection idle in the pool for 
longer is discarded when it is next leased, whatever {{Keep-Alive}} timeout the 
server announced. No background thread, works per lease.
* {{validateAfterInactivity}} (HttpClient default 2 seconds): a connection idle 
for longer is checked (a 1 ms read) before reuse, so one the server closed is 
discarded.

The {{httpClient.}} and {{httpConnection.}} prefixes bind 
{{HttpClientBuilder}}, {{RequestConfig.Builder}} and {{SocketConfig.Builder}}, 
not {{ConnectionConfig}}. The http component page (section "Pooled connections 
closed by the server or by a NAT", added by 
https://github.com/apache/camel/pull/30064) already says that 
{{validateAfterInactivity}} is not exposed.

h3. Why it matters

Found with fault-injection tests (a proxy that drops idle connections silently, 
as cloud NATs, firewalls and load balancers do after a few minutes without FIN 
or RST):

* *Silent idle drop*: the next request on the dropped pooled connection waits 
for the read timeout (3 minutes with HttpClient's defaults) and fails with 
{{SocketTimeoutException}}. HttpClient never retries an 
{{InterruptedIOException}}, not even for {{GET}}. The documented workaround, 
{{httpClient.connectionKeepAlive}}, only applies when the response has no 
{{Keep-Alive: timeout=N}} header: {{DefaultConnectionKeepAliveStrategy}} takes 
the server's value first, so a server that announces a longer timeout than the 
NAT's keeps the problem. {{connectionTimeToLive}} works, but also recycles busy 
connections.
* *Server closed the idle connection*: a request sent less than 2 seconds after 
the connection's last use, on a connection the server has closed in the 
meantime, fails with {{NoHttpResponseException}}. HttpClient retries it only 
for idempotent methods, so a {{POST}} fails.

h3. Steps to reproduce

Test HTTP server ({{ServerBootstrap}}) behind a small in-test TCP proxy; 
{{responseTimeout=1000}} on the endpoint.

# Silent drop: {{GET}} (succeeds, one pooled connection); the proxy turns the 
open connection silent (reads and discards, never answers, never closes); wait 
1 s; {{GET}} again. Server responses carry {{Keep-Alive: timeout=300}}; 
endpoint with {{httpClient.connectionKeepAlive=500}}.
# Server close: {{POST}} (succeeds); the proxy closes the open connection; 
{{POST}} again right away.

h3. Actual (main, HttpClient 5.6.5)

# {{java.net.SocketTimeoutException: Read timed out}} (with the defaults it 
would wait 3 minutes). Without the {{Keep-Alive}} header, 
{{httpClient.connectionKeepAlive=500}} avoids it.
# {{org.apache.hc.core5.http.NoHttpResponseException: localhost:<port> failed 
to respond}}

h3. Expected

Component options that set these two values on the pool the component creates, 
off / unchanged by default:

* {{connectionIdleTimeout}} (milliseconds, 0 = no limit, the default): maps to 
{{ConnectionConfig.setIdleTimeout}}. Set below the idle timeout of the NAT, 
firewall or load balancer, the request opens a new connection instead of using 
a dead one.
* {{validateAfterInactivity}} (milliseconds, default 2000 = HttpClient's 
default): maps to {{ConnectionConfig.setValidateAfterInactivity}}; 0 checks the 
connection before each reuse.

Both next to {{connectionTimeToLive}} (component options, like it), documented 
in the timeouts section of the http component page. 
{{HttpClientBuilder.evictIdleConnections}} is not a good fit: it starts a 
thread per HttpClient (Camel builds one per endpoint), has no effect with a 
shared connection manager, and still lets a connection be reused between two 
runs of the evictor.

_Claude Code on behalf of Croway_




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

Reply via email to