Croway opened a new pull request, #30208: URL: https://github.com/apache/camel/pull/30208
JIRA: https://issues.apache.org/jira/browse/CAMEL-27053 ## What was missing `HttpComponent.createConnectionManager` builds the `ConnectionConfig` of the pool that the component creates for each endpoint with the connection age only (`connectionTimeToLive`). Two other settings of that `ConnectionConfig` could not be set without replacing the whole `clientConnectionManager` (the `httpClient.` and `httpConnection.` prefixes bind `HttpClientBuilder`, `RequestConfig.Builder` and `SocketConfig.Builder`, not `ConnectionConfig`): - `idleTimeout` (new in HttpClient 5.6): a connection idle in the pool for longer is discarded when it is next leased. - `validateAfterInactivity` (HttpClient default 2 s): a connection idle for longer is checked for a close by the server before it is reused. Found with fault-injection tests: - A NAT, firewall or load balancer that drops an idle connection silently (no FIN, no RST): the next request on that pooled connection waits for the read timeout (3 minutes with the defaults) and fails with `SocketTimeoutException`, which HttpClient never retries, not even for `GET`. The workaround in the docs, `httpClient.connectionKeepAlive`, does not help when the server announces a longer `Keep-Alive: timeout=N`: `DefaultConnectionKeepAliveStrategy` takes the server's value first. `connectionTimeToLive` works but also recycles busy connections. - A `POST` sent less than 2 s after the last use of a connection that the server has closed in the meantime fails with `NoHttpResponseException`; HttpClient retries it only for idempotent methods. ## The change Two component options next to `connectionTimeToLive`, set on the `ConnectionConfig` of the pool that the component creates: - `connectionIdleTimeout` (millis, default `0` = no limit): `ConnectionConfig.setIdleTimeout`, set only when positive. - `validateAfterInactivity` (millis, default `2000`, which is HttpClient's own default, so nothing changes): `ConnectionConfig.setValidateAfterInactivity`; `0` checks the connection before each reuse (a read of up to 1 ms), a negative value disables the check. Docs: the "Pooled connections closed by the server or by a NAT" section of the http page (added by #30064) describes both options, notes that a server's `Keep-Alive` timeout overrides `httpClient.connectionKeepAlive`, and has a properties example. Generated files (configurer, component JSON, catalog copies, component DSL) are regenerated. ## Design choices for reviewers - `ConnectionConfig.idleTimeout`, not `HttpClientBuilder.evictIdleConnections(...)`. The evictor starts a background thread per `HttpClient` (Camel builds one per endpoint), has no effect when the connection manager is shared, and can still hand out a connection that went past its idle time between two of its runs. `idleTimeout` is checked when a connection is leased: no thread to stop with the endpoint, and an expired connection is never reused. This is the rationale HttpClient gives in the commit that added it (HttpClient 5.6, the version on main). - Component options only, like `connectionTimeToLive`, which is a setting of the same `ConnectionConfig`. Like it, they do not apply to a user-supplied `clientConnectionManager` (the existing NOTE in the docs now lists them). - No upgrade-guide entry: new options, defaults unchanged. ## Tests `HttpPooledConnectionReuseTest`: the module's test HTTP server behind a small in-test TCP proxy that can make the open connections silent (reads and discards, never answers, never closes), as a NAT idle drop looks to the client, or close them. No Docker. - `connectionDroppedSilentlyIsReusedWithoutIdleTimeout` / `connectionIdleTimeoutDiscardsConnectionDroppedSilently`: GET, the connection goes silent, it stays idle for more than 500 ms, GET again (`responseTimeout=1000`, the server answers with `Keep-Alive: timeout=300`). Without the option, even with `httpClient.connectionKeepAlive=500`: `SocketTimeoutException` on the reused connection. With `connectionIdleTimeout=500`: the second GET succeeds on a new connection. - `postOnConnectionClosedByServerFailsWithDefaultValidation` / `validateAfterInactivityDiscardsConnectionClosedByServer`: POST, the connection is closed, POST again right away. With the defaults: `NoHttpResponseException`. With `validateAfterInactivity=0`: the second POST succeeds on a new connection. Fail before: on main, the same scenarios (with only the settings main has) fail with `java.net.SocketTimeoutException: Read timed out` and `org.apache.hc.core5.http.NoHttpResponseException: localhost:<port> failed to respond`. With `httpClient.connectionKeepAlive=500` and no `Keep-Alive` header from the server, the silent-drop scenario already passes on main, as the docs say. camel-http: all unit tests pass (317 + 9 in the system-properties execution, 0 failures); the new test class also passed 3 more times in a row with reruns disabled. _Claude Code on behalf of Croway_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
