But how can I catch this exception? Is there anywhere I can register an exceptionhandler that catches this?
On Thu, Feb 23, 2012 at 8:20 PM, Oleg Kalnichevski <[email protected]> wrote: > On Thu, 2012-02-23 at 16:22 +0100, Alexander Hjalmarsson wrote: > > I've created a simple crawling framework with the help from > HttpAsyncClient > > and sometimes I get the exception that an existing connection was > forcibly > > closed by the remote host. I'm doing the requests rather polite (one > > request every 15 seconds) so I'm not really overwhelming the server. What > > could be the cause for this? I'd also like to know how/where I can catch > > this exception myself, so I don't continue to send requests to a server > > that closes the connections. > > > > 2012-feb-23 16:04:59 > > org.apache.http.impl.nio.client.LoggingClientProtocolHandler exception > > SEVERE: http-outgoing-259 [ACTIVE] I/O error: An existing connection was > > forcibly closed by the remote host > > java.io.IOException: An existing connection was forcibly closed by the > > remote host > > Things like that can happen. HTTP servers can and do drop connections > for all sorts of reasons. Your crawler should be prepared to deal with > such situations. I also think there is absolutely no reason to blacklist > the server just because of one failed request. The server should respond > with status 503 (Service Unavailable) if it is unable to process the > request due to overload or a similar condition. > > Oleg > > > > at sun.nio.ch.SocketDispatcher.read0(Native Method) > > at sun.nio.ch.SocketDispatcher.read(Unknown Source) > > at sun.nio.ch.IOUtil.readIntoNativeBuffer(Unknown Source) > > at sun.nio.ch.IOUtil.read(Unknown Source) > > at sun.nio.ch.SocketChannelImpl.read(Unknown Source) > > at > > > org.apache.http.impl.nio.reactor.SessionInputBufferImpl.fill(SessionInputBufferImpl.java:95) > > at > > > org.apache.http.impl.nio.codecs.AbstractMessageParser.fillBuffer(AbstractMessageParser.java:115) > > at > > > org.apache.http.impl.nio.conn.DefaultAsyncClientConnection$LoggingNHttpMessageParser.fillBuffer(DefaultAsyncClientConnection.java:164) > > at > > > org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:153) > > at > > > org.apache.http.impl.nio.DefaultClientIODispatch.onInputReady(DefaultClientIODispatch.java:118) > > at > > > org.apache.http.impl.nio.DefaultClientIODispatch.onInputReady(DefaultClientIODispatch.java:49) > > at > > > org.apache.http.impl.nio.AbstractIODispatch.inputReady(AbstractIODispatch.java:113) > > at > > > org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:160) > > at > > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:342) > > at > > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:320) > > at > > > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:280) > > at > > > org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:106) > > at > > > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:599) > > at java.lang.Thread.run(Unknown Source) > > > > The requests I make are just basic GET-requests, executed like following: > > > > httpclient.execute(request, new FutureCallback<HttpResponse>() { > > public void failed(Exception arg0) { > > //Do nothing > > } > > public void completed(HttpResponse arg0) { > > EntityUtils.consume(arg0.getEntity()); > > } > > > > public void cancelled() { > > //Do nothing > > } > > }); > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
