[
https://issues.apache.org/jira/browse/HTTPCORE-629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Kalnichevski resolved HTTPCORE-629.
----------------------------------------
Resolution: Duplicate
> IOReactor shutdowns on ConnectionClosedException
> ------------------------------------------------
>
> Key: HTTPCORE-629
> URL: https://issues.apache.org/jira/browse/HTTPCORE-629
> Project: HttpComponents HttpCore
> Issue Type: Bug
> Affects Versions: 4.4.12
> Reporter: Kristof Jozsa
> Priority: Major
>
> We run a Camel/CXFRS project with httpcore-nio 4.4.12 and we experience
> IOReactor shutdowns which seem to happen on ConnectionClosedExceptions in our
> production environment.
> We found this spamming in our logs:
> {{Caused by: java.lang.IllegalStateException: Request cannot be executed; I/O
> reactor status: STOPPED}}
> {{ at org.apache.http.util.Asserts.check(Asserts.java:46)}}
> {{ at
> org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase.ensureRunning(CloseableHttpAsyncClientBase.java:90)}}
> {{ at
> org.apache.http.impl.nio.client.InternalHttpAsyncClient.execute(InternalHttpAsyncClient.java:123)}}
> {{ at
> org.apache.cxf.transport.http.asyncclient.AsyncHTTPConduit$AsyncWrappedOutputStream.connect(AsyncHTTPConduit.java:596)}}
> {{ at
> org.apache.cxf.transport.http.asyncclient.AsyncHTTPConduit$AsyncWrappedOutputStream.setupWrappedStream(AsyncHTTPConduit.java:442)}}
> {{ at
> org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleHeadersTrustCaching(HTTPConduit.java:1343)}}
> {{ at
> org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.onFirstWrite(HTTPConduit.java:1304)}}
> {{ at
> org.apache.cxf.transport.http.asyncclient.AsyncHTTPConduit$AsyncWrappedOutputStream.onFirstWrite(AsyncHTTPConduit.java:435)}}
>
> Hunting for reasons we first found `ConnectionClosedException`s - these are
> completely fine:
> {{}}{{Caused by: org.apache.http.ConnectionClosedException:
> ConnectionClosedException invoking
> [https://target-host-masked.com/api/rest/v1/products/21110363:] Connection
> closed unexpectedly at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method) at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)
> at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:481)
> at
> org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1400)
> at
> org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1384)
> at
> org.apache.cxf.transport.http.asyncclient.AsyncHTTPConduit$AsyncWrappedOutputStream.close(AsyncHTTPConduit.java:429)
> at
> org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77)
> at
> org.apache.cxf.io.CacheAndWriteOutputStream.postClose(CacheAndWriteOutputStream.java:56)
> at org.apache.cxf.io.CachedOutputStream.close(CachedOutputStream.java:228)
> at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) at
> org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:671) at
> org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63)}}
>
> but these following traces:
> {{Caused by: org.apache.http.ConnectionClosedException: Connection closed
> unexpectedlyCaused by: org.apache.http.ConnectionClosedException: Connection
> closed unexpectedly at
> org.apache.http.nio.protocol.HttpAsyncRequestExecutor.closed(HttpAsyncRequestExecutor.java:146)
> at
> org.apache.http.impl.nio.client.InternalIODispatch.onClosed(InternalIODispatch.java:71)
> at
> org.apache.http.impl.nio.client.InternalIODispatch.onClosed(InternalIODispatch.java:39)
> at
> org.apache.http.impl.nio.reactor.AbstractIODispatch.disconnected(AbstractIODispatch.java:100)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.sessionClosed(BaseIOReactor.java:277)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.processClosedSessions(AbstractIOReactor.java:449)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.hardShutdown(AbstractIOReactor.java:590)
> at
> org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:305)
> at
> org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
> at
> org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)}}
>
> seem to show that the IOReactor went to hardShutdown upon the
> ConnectionClosedException?
>
> We tried digging into the source to understand how this can happen and we
> believe that if the {{processClosedSessions()}} method in
> {{AbstractIOReactor}} class shows any uncaught exceptions (like a
> ConnectionClosedException?), it will indeed cause the reactor to stop outside
> its infinite loop in the finally block.
>
> Can you please double check and confirm this behaviour? How can we avoid
> getting this problem? I would assume if a certain connection gets broken on
> the server side, it's handled gracefully. The fact that the whole reactor
> shuts down leaves us no other option then to restart our service - clearly we
> need a proper way to manage this somehow.
>
> I hope I'm not horribly mistaken with the drawn consequences above - if so,
> please point out my errors. Thanks really much in advance!
> EDIT: fixed formatting of stacktraces
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
