Hi Oleg,
Thanks for the fast and excellent response, am implementing your
suggested changes now.
Incidentally, the problem I observed where it sometimes didn't appear to
retry, was in fact due to problem 2 retrying, and then immediately being
followed by problem 1. The stack trace for problem 1 was on the console
stderr stream, so wasn't appearing the logs, which is why I thought it
was mysteriously exiting before retrying the full 5 times. So case
closed on that one, it was my misinterpretation.
Regarding the stale connection check, I am sure you have already
considered this, but what is the problem with doing the following:
- removing the stale connection check (or at least disabling by default)
- making the default http client use a retry handler that silently
retries the IO exceptions
- making the default http client have an IdleConnectionMonitorThread as
per your reference [2], that cleans up the stale connections
Wouldn't that cover all the bases and be just as effective at managing
connections as with the stale connection check enabled, and no one would
report any problems, and get a free performance benefit? Or is there
something else that the stale connection check is doing that makes it a
better choice?
Thanks again,
Tony
Oleg Kalnichevski wrote:
Tony Poppleton wrote:
Hi,
I am eager to squeeze the most performance out of my usage of
HttpClient 4.0 and have been reading the archives where it was
suggested to disable the stale connection check. I have done this,
and it does indeed significantly improve performance, however I am
now occasionally getting the following two exceptions:
1. Exception in thread "main" java.net.SocketException: Connection
reset by peer: socket write error
at java.net.SocketOutputStream.socketWrite0(Native Method)
at
java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:92)
at java.net.SocketOutputStream.write(SocketOutputStream.java:136)
at
org.apache.http.impl.io.AbstractSessionOutputBuffer.flushBuffer(AbstractSessionOutputBuffer.java:106)
at
org.apache.http.impl.io.AbstractSessionOutputBuffer.flush(AbstractSessionOutputBuffer.java:113)
at
org.apache.http.impl.AbstractHttpClientConnection.doFlush(AbstractHttpClientConnection.java:260)
at
org.apache.http.impl.SocketHttpClientConnection.close(SocketHttpClientConnection.java:248)
at
org.apache.http.impl.conn.DefaultClientConnection.close(DefaultClientConnection.java:154)
at
org.apache.http.impl.conn.AbstractPooledConnAdapter.close(AbstractPooledConnAdapter.java:131)
at
org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:130)
at
org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:447)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:641)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:730)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:708)
at
org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:699)
...
2. DefaultRequestDirector:455 - I/O exception
(org.apache.http.NoHttpResponseException) caught when processing
request: The target server failed to respond
DefaultRequestDirector:462 - Retrying request
The first one is problematic because the program terminates (I don't
handle the exception anywhere). The second one usually retries as I
am using the DefaultHttpRequestRetryHandler, but on occasion the
program has just exited too (without doing the 5 retries that I have
requested - I am trying to capture this case in a debugger so I can
get a clue as to what is happening, will post an update if I find
anything). The second error is far more frequent than the first, for
example today I have seen the second error about 50 times, but the
1st error only once.
Anyway, I presume that both these errors are only now apparent
because I have disabled the stale connection check.
Tony,
The best strategy to tackle the stale connection issue is very simple:
do not let connections go stale in the first place by evicting
connections from the connection pool that have been idle for too long.
For details see [1] and [2].
What advice can you give me on how to recover from the first
problem? It is important that I don't re-post my requests if they
did manage to get through before the error occurred.
As long as the method being executed can be considered idempotent [3]
it is safe simply to retry the request using a custom retry handler
[4] and be done with it. However, if your application uses HTTP
transport to execute transactional business logic (like placing an
order for an item) you may have a problem. Actually you do have a
problem anyways because HTTP transport is not transaction safe and
HTTP connections can and do fail in a middle of a transaction (the
request gets executed, an order gets placed but the response is never
delivered back to the client). To sum things up this is an application
design issue, and not an issue of the HTTP transport.
For the second one, is it 100% safe to resume from that error?
Yes, it is.
Is it
fully equivalent in terms of network communications as having the
stale connection check enabled?
Pretty much.
Out of interest, what is the history of the stale connection check?
From reading the archives it appears it is a relic of an older
HttpClient, and its usage is not recommended anymore. Is this correct?
Yes, it is
If so, why isn't it just removed from the latest version, and
substituted with a robust retry handler that can deal with all the
consequences?
Firstly, there is no such thing as a robust retry handler for HTTP
transport for the reason given above. HTTP request retrial logic is
always application specific. Secondly, there are tons of people out
there who do not really understand what persistent connections are all
about and start complaining about HttpClient nor working if they
occasionally see a request failing with an I/O exception. It is easier
for the sake of everyone's sanity to have the damn stale check on per
default.
Cheers
Oleg
[1]
http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e638
[2]
http://hc.apache.org/httpcomponents-client/tutorial/html/connmgmt.html#d4e645
[3]
http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d4e255
[4]
http://hc.apache.org/httpcomponents-client/tutorial/html/fundamentals.html#d4e280
Many thanks,
Tony
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
---------------------------------------------------------------------------------------
Orange vous informe que cet e-mail a ete controle par l'anti-virus
mail. Aucun virus connu a ce jour par nos services n'a ete detecte.
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
