[
https://issues.apache.org/jira/browse/HTTPCLIENT-2155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17331248#comment-17331248
]
Saksham edited comment on HTTPCLIENT-2155 at 4/24/21, 3:02 PM:
---------------------------------------------------------------
Hi Oleg,
Thanks for the reply.
Can you please help me understand that why just after adding
'*EntityUtils.toByteArray(response.getEntity()) in the else block'* the
connection reuse starts happening properly even though os properties remain
unchanged in both the scenarios ?
(Our remote service returns 404 if the key requested is not found and thus in
case there were lot of 404 we are observing this issue, since in 2xx we were
already consuming the stream hence we concluded that we were not facing the
issue in 2xx case.)
To put in some numbers:
The first Snippet -> creates 25K TIME_WAIT connections with just 400 RPS
while the second snippet can do 500-600 RPS with just 120-150 connections in
TIME_WAIT.
And the only difference between both the scenarios is just putting the above
line in else block, rest all things remain unchanged.
was (Author: saksham2405):
Hi Oleg,
Thanks for the reply.
Can you please help me understand that why just after adding
'*EntityUtils.toByteArray(response.getEntity()) in the else block'* the
connection reuse starts happening properly even though os properties remain
unchanged in both the scenarios ?
To put in some numbers:
The first Snippet -> creates 25K TIME_WAIT connections with just 400 RPS
while the second snippet can do 500-600 RPS with just 120-150 connections in
TIME_WAIT.
And the only difference between both the scenarios is just putting the above
line in else block, rest all things remain unchanged.
> CloseableHttpClient leaving too many TIME_WAIT connections
> ----------------------------------------------------------
>
> Key: HTTPCLIENT-2155
> URL: https://issues.apache.org/jira/browse/HTTPCLIENT-2155
> Project: HttpComponents HttpClient
> Issue Type: Bug
> Reporter: Saksham
> Priority: Major
>
> We are using *CloseableHttpClient* with *PoolingHttpClientConnectionManager,*
> following is the configuration:
> {code:java}
> RequestConfig requestConfig = RequestConfig.custom()
> .setConnectTimeout(1000)
> .setConnectionRequestTimeout(1000)
> .setSocketTimeout(1000)
> .build();
> PoolingHttpClientConnectionManager connectionManager = new
> PoolingHttpClientConnectionManager();
> connectionManager.setMaxTotal(100);
> connectionManager.setDefaultMaxPerRoute(100);
> this.httpClient = HttpClients.custom()
> .setDefaultRequestConfig(requestConfig)
> .setConnectionManager(connectionManager)
> .build();
> {code}
>
> Now when we are not consuming the response entity and just relying on
> closeableHttpResponse to release the resources, we end up with 25K
> connections in CLOSE_WAIT state with just over 400RPS to the system. Due this
> system runs out of resources very quickly and requests start failing.
> Problem Snippet:
> {code:java}
> try (CloseableHttpResponse response = this.httpClient.execute(httpGet)){
> if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
> return EntityUtils.toByteArray(response.getEntity());
> } else {
> if (response.getStatusLine().getStatusCode() >= 500) {
> throw new RemoteCallFailException();
> }
> }
> } catch (IOException e) {
> logger.error("Remote db read request failed [{}]", e.getMessage(), e);
> throw new RemoteCallFailException();
> }
> {code}
> After using *EntityUtils.toByteArray(response.getEntity()) in the else
> block* of above code connections are properly getting released and reused in
> subsequent requests.
> Working Snippet:
> {code:java}
> try (CloseableHttpResponse response = this.httpClient.execute(httpGet)) {
> if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
> return EntityUtils.toByteArray(response.getEntity());
> } else {
> // Consuming stream to release connection
> EntityUtils.toByteArray(response.getEntity());
> if (response.getStatusLine().getStatusCode() >= 500) {
> throw new RemoteCallFailException();
> }
> }
> } catch (IOException e) {
> logger.error("Remote db read request failed [{}]", e.getMessage(), e);
> throw new RemoteCallFailException();
> }
> {code}
> Can someone please point why releasing of connection is not automatically
> done by closeablehttpResponse.close() ? ( Or maybe point to something wrong
> in the code)
> MVN dependency used for same:
> <dependency>
> <groupId>org.apache.httpcomponents</groupId>
> <artifactId>httpclient</artifactId>
> <version>4.3.3</version>
> </dependency>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
