On Mon, Mar 1, 2010 at 11:01 AM, Vidura Gamini Abhaya <[email protected]>wrote:
> Hi All, > > I'm using Axis2 and Synapse together for some experiments. Axis2 ver 1.4 is > used both as a client as well as to host a couple of web services. I use > Synapse (ver 1.2) as a load balancer/request redirector to 2 or more Axis2 > instances. Axis2 servers run with a modified axis2-kernel where I've done > changes to AxisEngine and some of the transports. Synapse also uses this > modified kernel. I generate requests which gets load balanced according to > some algorithms I implement in Synapse. I change the arrival rates of the > requests (at Synapse) for various experiments. > > I encounter some problems when there's a higher arrival of requests at > synapse (for lower arrival rates this seem to work fine). After some time > into the experiments, some of the requests time out on the client side > complaining there's no response from Synapse. When inspecting Synapse, I > could see that the following exceptions were thrown, > > "[HttpClientWorker-4] ERROR ServerHandler IO error submiting response : > Connection is closed > org.apache.http.ConnectionClosedException: Connection is closed > at > org.apache.http.impl.nio.NHttpConnectionBase.assertNotClosed(NHttpConnectionBase.java:238) > at > org.apache.http.impl.nio.DefaultNHttpServerConnection.submitResponse(DefaultNHttpServerConnection.java:200) > at > org.apache.synapse.transport.nhttp.ServerHandler.commitResponse(ServerHandler.java:213) > at > org.apache.synapse.transport.nhttp.HttpCoreNIOSender.sendAsyncResponse(HttpCoreNIOSender.java:416) > at > org.apache.synapse.transport.nhttp.HttpCoreNIOSender.invoke(HttpCoreNIOSender.java:271) > at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:567) > at > org.apache.synapse.core.axis2.Axis2Sender.sendBack(Axis2Sender.java:143) > at > org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send(Axis2SynapseEnvironment.java:219) > at > org.apache.synapse.mediators.builtin.SendMediator.mediate(SendMediator.java:154) > at > org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58) > at > org.apache.synapse.mediators.filters.OutMediator.mediate(OutMediator.java:60) > at > org.apache.synapse.mediators.AbstractListMediator.mediate(AbstractListMediator.java:58) > at > org.apache.synapse.mediators.base.SequenceMediator.mediate(SequenceMediator.java:125) > at > org.apache.synapse.core.axis2.Axis2SynapseEnvironment.injectMessage(Axis2SynapseEnvironment.java:176) > at > org.apache.synapse.core.axis2.SynapseCallbackReceiver.handleMessage(SynapseCallbackReceiver.java:344) > at > org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive(SynapseCallbackReceiver.java:139) > at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:274) > at > org.apache.synapse.transport.nhttp.ClientWorker.run(ClientWorker.java:200) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675) > at java.lang.Thread.run(Thread.java:713)" > > The number of requests synapse handles are logged and strangely that equals > to more than the number of requests sent to Synapse. Investigating further > on this I could see that Synapse had processed the same request more than > once and sometimes, the result hasn't been sent to the client. I'm guessing > that this may be due to retransmission of requests from the client due to > not having any response from synapse. > > The Axis2 client is used an asynchronous client and since the timing of the > requests is important, each request is created on a separate thread and use > a newly created serviceClient object inside each thread of execution. After > the non blocking call is done serviceClient.cleanup() and > serviceClient.cleanupTransport() is called. I'm wondering whether the > problems are connected with the version of the client I use (I've seen that > 1.4 has some problems) and/or the way I'm using the client. Here's a part of > the code how it's done, > For synapse issue try to increase the time out value in nhttp properties file. > > "ServiceClient serviceClient = new ServiceClient(); > Options opts = new Options(); > > opts.setTo(EPR); > opts.setAction(Action); > > opts.setTimeOutInMilliSeconds(5400000); > serviceClient.setOptions(opts); > > serviceClient.sendReceiveNonBlocking(this.createPayload(parameter), > callback); > ... > serviceClient.cleanup(); > serviceClient.cleanupTransport();" > you can not clean up the transports before getting the response. > > Migrating the Axis2 client libraries to 1.5.1 resulted in exceptions being > thrown and requests not reaching Synapse at all. I have a couple of > questions on the above, > With axis2 1.5.1 by default you have only 2 connections. Unless you increase this value it may gives exceptions for a high load scenarios. thanks, Amila. > > 1) Could this be due to bugs with the 1.4 version of Axis2? > 2) Are there any changes required/suggested in the way the client code > works? > 3) Could this be happening due to any problems with Synapse? (I posted this > to the Axis2 mailing list assuming that it might be to do with the client), > in which case please cross post this to the Synapse mailing list, thanks. > > Appreciate any help and suggestions you could give in solving this issue. > > Thanks, > > Vidura > > -- Amila Suriarachchi WSO2 Inc. blog: http://amilachinthaka.blogspot.com/
