On Wed, 2007-03-21 at 12:19 +0100, Oleg Kalnichevski wrote: > On Wed, 2007-03-21 at 11:06 +0000, ant elder wrote: > > I've been trying to use JMeter to test Synapse but getting > > intermittent failures, see below for the stack trace output. Happens > > all the time, maybe a few hundred requests are successful but then it > > gets the failure and hangs. I'm not sure whats the easiest way to > > debug this is, any ideas? > > > > ...ant > > >
Folks, It turned out the Synapse transport lockup was unlikely to be have been caused by HttpCore components. I overlooked the fact ClosedChannelException was a checked I/O exception and it could not have caused the I/O dispatch thread to terminate. So, Synapse must have locked up due to some other reason. It appears now something went screwy in Sun's PipeImpl. I do not know NIO pipes well enough to comment as to what may be the cause. > $Worker.run(AbstractMultiworkerIOReactor.java :153) > > at java.lang.Thread.run(Thread.java:595) > > Caused by: java.net.BindException: Address already in use: connect > > at sun.nio.ch.Net.connect(Native Method) > > at sun.nio.ch.SocketChannelImpl.connect > > (SocketChannelImpl.java:464) > > at > > java.nio.channels.SocketChannel.open(SocketChannel.java:146) > > at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:78) > > ... 14 more I tweaked HttpCore code a little just in case and made it catch ClosedChannelException-s in I/O operations. I re-published the latest snapshots to the Maven repository. I am standing by just in case you want me to do more checks on HttpCore code. Oleg > Anthony, > > The transport appears to be hanging because an unchecked exception > caused the I/O dispatch thread to terminate. I believe there are several > different types of problems (at least two) that we are seeing here. > > [I/O reactor worker thread 5] ERROR ServerHandler - I/O Error : null > > java.nio.channels.ClosedChannelException > > at > > sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:112) > > at > > sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:139) > > This one is definitely a bug in HttpCore. Could you please raise a Jira > issue in HttpComponents Core and I'll get this problem fixed tonight or > tomorrow? > > Oleg > > > > [I/O reactor worker thread 5] ERROR ServerHandler - Error processing > > request received for : /axis2/services/SimpleStockQuoteService > > java.io.IOException: Unable to establish loopback connection > > at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:106) > > at java.security.AccessController.doPrivileged(Native Method) > > at sun.nio.ch.PipeImpl.<init>(PipeImpl.java:122) > > at sun.nio.ch.SelectorProviderImpl.openPipe > > (SelectorProviderImpl.java:27) > > at java.nio.channels.Pipe.open(Pipe.java:133) > > at > > org.apache.axis2.transport.nhttp.ServerHandler.requestReceived(ServerHandler.java:108) > > at > > org.apache.axis2.transport.nhttp.LoggingNHttpServiceHandler.requestReceived > > (LoggingNHttpServiceHandler.java:79) > > at > > org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput(DefaultNHttpServerConnection.java:96) > > at > > org.apache.axis2.transport.nhttp.PlainServerIOEventDispatch.inputReady > > (PlainServerIOEventDispatch.java:67) > > at > > org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:68) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java > > :160) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:145) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:127) > > at > > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor > > $Worker.run(AbstractMultiworkerIOReactor.java:153) > > at java.lang.Thread.run(Thread.java:595) > > Caused by: java.net.BindException: Address already in use: connect > > at sun.nio.ch.Net.connect(Native Method) > > at > > sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:464) > > at > > java.nio.channels.SocketChannel.open(SocketChannel.java:146) > > at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:78) > > ... 14 more > > [I/O reactor worker thread 5] ERROR ServerHandler - I/O Error : null > > java.nio.channels.ClosedChannelException > > at sun.nio.ch.SocketChannelImpl.ensureReadOpen > > (SocketChannelImpl.java:112) > > at > > sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:139) > > at org.apache.axis2.transport.nhttp.LoggingIOSession > > $LoggingByteChannel.read(LoggingIOSession.java:153) > > at > > org.apache.http.impl.nio.reactor.SessionInputBuffer.fill(SessionInputBuffer.java:75) > > at > > org.apache.http.impl.nio.codecs.ChunkDecoder.read(ChunkDecoder.java:157) > > at org.apache.axis2.transport.nhttp.ServerHandler.inputReady > > (ServerHandler.java:150) > > at > > org.apache.axis2.transport.nhttp.LoggingNHttpServiceHandler.inputReady(LoggingNHttpServiceHandler.java:103) > > at > > org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput > > (DefaultNHttpServerConnection.java:108) > > at > > org.apache.axis2.transport.nhttp.PlainServerIOEventDispatch.inputReady(PlainServerIOEventDispatch.java:67) > > at org.apache.http.impl.nio.reactor.BaseIOReactor.readable > > (BaseIOReactor.java:68) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:160) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java > > :145) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:127) > > at > > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor > > $Worker.run(AbstractMultiworkerIOReactor.java :153) > > at java.lang.Thread.run(Thread.java:595) > > [I/O reactor worker thread 5] WARN ServerHandler - Connection Timeout > > for request to : /axis2/services/SimpleStockQuoteService > > [I/O reactor worker thread 5] ERROR ServerHandler - Error processing > > request received for : /axis2/services/SimpleStockQuoteService > > java.io.IOException: Unable to establish loopback connection > > at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:106) > > at java.security.AccessController.doPrivileged(Native Method) > > at sun.nio.ch.PipeImpl.<init>(PipeImpl.java:122) > > at > > sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:27) > > at java.nio.channels.Pipe.open(Pipe.java:133) > > at > > org.apache.axis2.transport.nhttp.ServerHandler.requestReceived > > (ServerHandler.java:108) > > at > > org.apache.axis2.transport.nhttp.LoggingNHttpServiceHandler.requestReceived(LoggingNHttpServiceHandler.java:79) > > at > > org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput > > (DefaultNHttpServerConnection.java:96) > > at > > org.apache.axis2.transport.nhttp.PlainServerIOEventDispatch.inputReady(PlainServerIOEventDispatch.java:67) > > at org.apache.http.impl.nio.reactor.BaseIOReactor.readable > > (BaseIOReactor.java:68) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:160) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java > > :145) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:127) > > at > > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor > > $Worker.run(AbstractMultiworkerIOReactor.java :153) > > at java.lang.Thread.run(Thread.java:595) > > Caused by: java.net.BindException: Address already in use: connect > > at sun.nio.ch.Net.connect(Native Method) > > at sun.nio.ch.SocketChannelImpl.connect > > (SocketChannelImpl.java:464) > > at > > java.nio.channels.SocketChannel.open(SocketChannel.java:146) > > at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:78) > > ... 14 more > > [I/O reactor worker thread 5] ERROR ServerHandler - I/O Error : null > > java.nio.channels.ClosedChannelException > > at > > sun.nio.ch.SocketChannelImpl.ensureReadOpen(SocketChannelImpl.java:112) > > at > > sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:139) > > at org.apache.axis2.transport.nhttp.LoggingIOSession > > $LoggingByteChannel.read(LoggingIOSession.java:153) > > at > > org.apache.http.impl.nio.reactor.SessionInputBuffer.fill(SessionInputBuffer.java:75) > > at org.apache.http.impl.nio.codecs.ChunkDecoder.read > > (ChunkDecoder.java:157) > > at > > org.apache.axis2.transport.nhttp.ServerHandler.inputReady(ServerHandler.java:150) > > at > > org.apache.axis2.transport.nhttp.LoggingNHttpServiceHandler.inputReady(LoggingNHttpServiceHandler.java > > :103) > > at > > org.apache.http.impl.nio.DefaultNHttpServerConnection.consumeInput(DefaultNHttpServerConnection.java:108) > > at > > org.apache.axis2.transport.nhttp.PlainServerIOEventDispatch.inputReady(PlainServerIOEventDispatch.java > > :67) > > at > > org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:68) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:160) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents > > (AbstractIOReactor.java:145) > > at > > org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:127) > > at > > org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor > > $Worker.run(AbstractMultiworkerIOReactor.java :153) > > at java.lang.Thread.run(Thread.java:595) > > [I/O reactor worker thread 7] WARN ServerHandler - Connection Timeout > > for request to : /axis2/services/SimpleStockQuoteService > > [I/O reactor worker thread 5] WARN ServerHandler - Connection Timeout > > for request to : /axis2/services/SimpleStockQuoteService > > Terminate batch job (Y/N)? y > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
