HttpClient 4.1 MaxTotalConnections and MaxPerRoute questions
--------------------------------------------------------------
Key: HTTPCLIENT-1150
URL: https://issues.apache.org/jira/browse/HTTPCLIENT-1150
Project: HttpComponents HttpClient
Issue Type: Bug
Components: HttpConn
Affects Versions: 4.1.2, 4.1.1
Environment: LAMP stack
Reporter: mohan pindyala
We have a java web application which makes calls to 2 of our third party
Services using httpclient.
Following are the settings for
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManagerr. the numbers are
ridiculously high
httpclient.defaultMaxPerRoute = 10000
httpclient.maxTotalConnections = 20000
Socket and connection timeout on org.apache.http.params.HttpConnectionParams
are set as
httpclient.socketTimeout = 120000ms
httpclient.connectionTimeout = 20000ms
we have disabled the
org.apache.http.params.HttpConnectionParams#setStaleCheckingEnabled = false.
we have a cron process running which checks for idle connections using
ClientConnectionManager object every 10 seconds and closed expired & idle
connections.
We also have retry handler set on
org.apache.http.impl.client.DefaultHttpClient. It retries once with a 40ms
delay.
So questions are as follows
1. Does above setting look ok for you ? if not what are the optimal numbers.
Lets say there are 4 third party services we have to call , how should set
these numbers to support that.
2. We sometimes get following errors, which are not able to figure out if it
is the third party server or httpclient issue. These errors show up during
peak volume hitting our servers.
a. java.net.SocketException: Connection reset
b. :java.net.SocketTimeoutException: Read timed out
c. org.apache.http.conn.ConnectionPoolTimeoutException:
Timeout waiting for connection
we checked with our third party services and everything was normal on their
side when these things happened.
Following are the stack traces for each exception type
org.apache.http.conn.ConnectionPoolTimeoutException: Timeout waiting for
connection
at
org.apache.http.impl.conn.tsccm.ConnPoolByRoute.getEntryBlocking(ConnPoolByRoute.java:417)
at
org.apache.http.impl.conn.tsccm.ConnPoolByRoute$1.getPoolEntry(ConnPoolByRoute.java:300)
at
org.apache.http.impl.conn.tsccm.ThreadSafeClientConnManager$1.getConnection(ThreadSafeClientConnManager.java:224)
at
gwy.core.io.TrackingClientConnectionRequest.getConnection(TrackingClientConnectionRequest.java:68)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:391)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:168)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:110)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:260)
at
org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
at
gwy.core.io.TrackingClientConnection.receiveResponseHeader(TrackingClientConnection.java:140)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:622)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:454)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:732)
java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.fillBuffer(AbstractSessionInputBuffer.java:149)
at
org.apache.http.impl.io.SocketInputBuffer.fillBuffer(SocketInputBuffer.java:110)
at
org.apache.http.impl.io.AbstractSessionInputBuffer.readLine(AbstractSessionInputBuffer.java:260)
at
org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:98)
at
org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
at
org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
at
org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
at
org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
at
gwy.core.io.TrackingClientConnection.receiveResponseHeader(TrackingClientConnection.java:140)
at
org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at
org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:622)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:454)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:754)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
