[
https://issues.apache.org/jira/browse/HTTPCORE-750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17728608#comment-17728608
]
Oleg Kalnichevski commented on HTTPCORE-750:
--------------------------------------------
[~Gaojie_Liu] There is a race condition between the connection request thread
and the I/O reactor thread that creates the connection and assigns it to the
pool entry. The request can be marked as done but its completion callback has
not been fully executed yet. Presently the pool immediately creates another
connection request if the previous request has been marked as done but there is
no connection assigned to the pool entry, which potentially leads to multiple
connections getting created instead of one. This needs to be fixed.
Oleg
> H2ConnPool creates too many conns under high load at initialization time
> ------------------------------------------------------------------------
>
> Key: HTTPCORE-750
> URL: https://issues.apache.org/jira/browse/HTTPCORE-750
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Components: HttpCore
> Affects Versions: 5.2.1
> Environment: Linux / Ubuntu / openjdk version "1.8.0_292"
> Reporter: Gaojie Liu
> Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> We are using httpcore5 5.2.1 and we observed that at application startup
> time, httpclient5 could create more than 1 conn per endpoint and in some
> cases, too many connections caused OOM issue since httpclient5 creates a pair
> of input/output buffer per conn.
> The regression is introduced by HTTPCORE-750, which made this change:
> [https://github.com/apache/httpcomponents-core/commit/14caf43eae407c544161c7f92329e8beb42a3534]
>
> {code:java}
> if (poolEntry.session != null) {
> callback.completed(poolEntry.session);
> } else {
> poolEntry.requestQueue.add(callback);
> if (poolEntry.sessionFuture != null &&
> poolEntry.sessionFuture.isDone()) {
> poolEntry.sessionFuture = null;
> } {code}
> When poolEntry.sessionFuture.isDone() is true, the connection is already
> ready, but the existing logic will abandon it and create a new one, and under
> high load, this logic could create a lot of conns per endpoint, which
> consumes a lot of memory.
>
> The proposed fix:
> {code:java}
> if (poolEntry.session != null) {
> callback.completed(poolEntry.session);
> } else {
> poolEntry.requestQueue.add(callback);
> if (poolEntry.sessionFuture != null &&
> poolEntry.sessionFuture.isDone()) {
> // Check whether we should recreate a new conn or not
> try {
> poolEntry.session = poolEntry.sessionFuture.get();
> while (true) {
> final FutureCallback<IOSession> pendingCallback =
> poolEntry.requestQueue.poll();
> if (pendingCallback != null) {
> pendingCallback.completed(poolEntry.session);
> } else {
> break;
> }
> }
> } catch (final Exception e) {
> poolEntry.sessionFuture = null;
> }
> } {code}
> I am planning to cut a PR for this.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@hc.apache.org
For additional commands, e-mail: dev-h...@hc.apache.org
