On Wed, Oct 7, 2015 at 5:35 PM, Chanaka Fernando <[email protected]> wrote:
> Hi Nuwan, > > Here is the mail thread. > [Dev][ESB] EmptyStackException when resuming a paused message processor > > On Wed, Oct 7, 2015 at 5:22 PM, Nuwan Dias <[email protected]> wrote: > >> Hi Chanaka, >> >> Thanks for looking into this. Hope we can get it merged to the kernel >> soon :). >> >> Thanks, >> NuwanD. >> >> On Wed, Oct 7, 2015 at 5:19 PM, Chanaka Fernando <[email protected]> >> wrote: >> >>> Hi NuwanD, >>> >>> I have found the root cause for this issue. As mentioned in [1], this >>> error is coming from kernel level and that has been propagated to the >>> synapse level. Actual root cause is mentioned below. >>> >>> [2015-10-07 17:13:38,135] ERROR - ServerWorker Error processing GET >>> reguest for : /t/api.com/responseapi/weather?q=London. Error detail: >>> null. >>> java.util.EmptyStackException >>> at java.util.Stack.peek(Stack.java:102) >>> at java.util.Stack.pop(Stack.java:84) >>> at org.wso2.carbon.context.internal.CarbonContextDataHolder. >>> endTenantFlow(CarbonContextDataHolder.java:1291) >>> at org.wso2.carbon.context.PrivilegedCarbonContext.endTenantFlow( >>> PrivilegedCarbonContext.java:75) >>> at org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver. >>> processRequest(MultitenantMessageReceiver.java:229) >>> at org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver. >>> receive(MultitenantMessageReceiver.java:78) >>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>> at org.apache.synapse.transport.passthru.ServerWorker. >>> processNonEntityEnclosingRESTHandler(ServerWorker.java:317) >>> at org.apache.synapse.transport.passthru.ServerWorker.run( >>> ServerWorker.java:149) >>> at org.apache.axis2.transport.base.threads.NativeWorkerPool$ >>> 1.run(NativeWorkerPool.java:172) >>> at java.util.concurrent.ThreadPoolExecutor.runWorker( >>> ThreadPoolExecutor.java:1145) >>> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >>> ThreadPoolExecutor.java:615) >>> at java.lang.Thread.run(Thread.java:745) >>> >>> >>> There is a separate mail thread to discuss about this issue and >>> previously Jagath has given a fix but that was not merged to the kernel. I >>> will reply to that thread and continue the fix on that thread. >>> >>> [1] https://wso2.org/jira/browse/ESBJAVA-4142 >>> >>> Thanks, >>> Chanaka >>> >>> On Tue, Oct 6, 2015 at 4:49 PM, Chanaka Fernando <[email protected]> >>> wrote: >>> >>>> Hi NuwanD, >>>> >>>> I can reproduce the issue with the provided configuration. I have >>>> already created a fix for this issue and currently testing it. I'll update >>>> the thread once I finalized the fix and merge the PR. >>>> >>>> On Tue, Oct 6, 2015 at 2:59 PM, Nuwan Dias <[email protected]> wrote: >>>> >>>>> Attaching the two APIs I used for testing. To reproduce this issue, >>>>> create a tenant with the domain as 'wso2.com' and deploy these two >>>>> APIs within the tenant space and restart the server. >>>>> >>>>> Then access the API without the version using the following URL. >>>>> >>>>> http://localhost:8280/t/wso2.com/responseapi/weather?q=London >>>>> >>>>> Thanks, >>>>> NuwanD. >>>>> >>>>> On Tue, Oct 6, 2015 at 2:53 PM, Chanaka Fernando <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Nuwan, >>>>>> >>>>>> I had a look into the code which is causing this NPE. According to >>>>>> the way it is written, this NPE can be occurred. I have tried to >>>>>> replicate >>>>>> what NuwanD has mentioned but couldn't reproduce the same NPE. This can >>>>>> be >>>>>> occurred due to synapse configuration issue. >>>>>> >>>>>> @NuwanD: Can you share the configurations which you have used to >>>>>> reproduce this issue? >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Chanaka >>>>>> >>>>>> On Thu, Oct 1, 2015 at 10:41 AM, Chanaka Fernando <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> We are looking into this issue. >>>>>>> >>>>>>> On Thu, Oct 1, 2015 at 10:37 AM, Sandapa Handakumbura < >>>>>>> [email protected]> wrote: >>>>>>> >>>>>>>> Hi Nuwan, >>>>>>>> >>>>>>>> I faced a similar issue during ESB 4.9.0 release testing. First >>>>>>>> request for proxy services (after tenant unloading/ server restart) >>>>>>>> failed >>>>>>>> in tenant mode [1]. Unfortunately ESB team could not reproduce it and I >>>>>>>> could not identify the exact steps to reproduce it either. >>>>>>>> >>>>>>>> Hope the team will look in to this since you have identified steps >>>>>>>> to reproduce the issue. >>>>>>>> >>>>>>>> [1] https://wso2.org/jira/browse/ESBJAVA-4142 >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Sandapa >>>>>>>> >>>>>>>> >>>>>>>> On Thu, Oct 1, 2015 at 4:47 AM, Nuwan Dias <[email protected]> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I have a scenario on the ESB where I have created two APIs with >>>>>>>>> the same context in the tenant space. The first API doesn't have a >>>>>>>>> version >>>>>>>>> and the second API has a version. The first API receives the request >>>>>>>>> and >>>>>>>>> forwards to the second API. See illustration below. >>>>>>>>> >>>>>>>>> Client (/twitter) --> Twitter_API (/twitter/1.0.0) --> >>>>>>>>> Twitter_API_v1.0.0 >>>>>>>>> >>>>>>>>> This scenario however causes the following [1] NPE on the first >>>>>>>>> request that's sent to the server. The subsequent requests pass until >>>>>>>>> a >>>>>>>>> server restart happens. >>>>>>>>> >>>>>>>>> I tried logging into the management console of the ESB using >>>>>>>>> tenant credentials to check whether it was related to tenant loading. >>>>>>>>> The >>>>>>>>> first request still failed with the same error. Any idea why this is >>>>>>>>> happening? This issue first occurred to me in API Manager after >>>>>>>>> migrating >>>>>>>>> to the latest released version of Synapse. And is causing test >>>>>>>>> failures in >>>>>>>>> the product. It worked fine in the older versions of Synapse (2.1.2- >>>>>>>>> version range). >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> >>>>>>>>> [2015-10-01 04:41:39,519] ERROR - PassThroughHttpSender Failed to >>>>>>>>> submit the response >>>>>>>>> java.lang.NullPointerException >>>>>>>>> at org.apache.synapse.transport.passthru.util. >>>>>>>>> SourceResponseFactory.create(SourceResponseFactory.java:64) >>>>>>>>> at org.apache.synapse.transport.passthru.PassThroughHttpSender. >>>>>>>>> submitResponse(PassThroughHttpSender.java:462) >>>>>>>>> at org.apache.synapse.transport.passthru. >>>>>>>>> PassThroughHttpSender.invoke(PassThroughHttpSender.java:267) >>>>>>>>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442) >>>>>>>>> at org.wso2.carbon.core.multitenancy.transports. >>>>>>>>> TenantTransportSender.invoke(TenantTransportSender.java:231) >>>>>>>>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442) >>>>>>>>> at org.apache.synapse.core.axis2.Axis2Sender.sendBack( >>>>>>>>> Axis2Sender.java:212) >>>>>>>>> at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send( >>>>>>>>> Axis2SynapseEnvironment.java:444) >>>>>>>>> at org.apache.synapse.mediators.builtin.SendMediator.mediate( >>>>>>>>> SendMediator.java:102) >>>>>>>>> at org.apache.synapse.mediators.AbstractListMediator.mediate( >>>>>>>>> AbstractListMediator.java:81) >>>>>>>>> at org.apache.synapse.mediators.AbstractListMediator.mediate( >>>>>>>>> AbstractListMediator.java:48) >>>>>>>>> at org.apache.synapse.mediators.base.SequenceMediator.mediate( >>>>>>>>> SequenceMediator.java:149) >>>>>>>>> at org.apache.synapse.rest.Resource.process(Resource.java:297) >>>>>>>>> at org.apache.synapse.rest.API.process(API.java:335) >>>>>>>>> at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI( >>>>>>>>> RESTRequestHandler.java:97) >>>>>>>>> at org.apache.synapse.rest.RESTRequestHandler.process( >>>>>>>>> RESTRequestHandler.java:52) >>>>>>>>> at org.apache.synapse.core.axis2.Axis2SynapseEnvironment. >>>>>>>>> injectMessage(Axis2SynapseEnvironment.java:266) >>>>>>>>> at org.apache.synapse.core.axis2.SynapseCallbackReceiver. >>>>>>>>> handleMessage(SynapseCallbackReceiver.java:529) >>>>>>>>> at org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive( >>>>>>>>> SynapseCallbackReceiver.java:172) >>>>>>>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>>>>>>>> at org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver. >>>>>>>>> processResponse(MultitenantMessageReceiver.java:166) >>>>>>>>> at org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver. >>>>>>>>> receive(MultitenantMessageReceiver.java:82) >>>>>>>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>>>>>>>> at org.apache.synapse.transport.passthru.ClientWorker.run( >>>>>>>>> ClientWorker.java:247) >>>>>>>>> at org.apache.axis2.transport.base.threads.NativeWorkerPool$ >>>>>>>>> 1.run(NativeWorkerPool.java:172) >>>>>>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker( >>>>>>>>> ThreadPoolExecutor.java:1145) >>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >>>>>>>>> ThreadPoolExecutor.java:615) >>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>> [2015-10-01 04:41:39,521] ERROR - Axis2Sender Access-Control-Allow- >>>>>>>>> Credentials:true,Access-Control-Allow-Methods:GET, >>>>>>>>> POST,Access-Control-Allow-Origin:*,Content-Type:application/json; >>>>>>>>> charset=utf-8,X-Source:redis,<?xml version='1.0' >>>>>>>>> encoding='utf-8'?><soapenv:Envelope xmlns:soapenv="http://www.w3. >>>>>>>>> org/2003/05/soap-envelope"><soapenv:Body/></soapenv:Envelope> >>>>>>>>> Unexpected error sending message back >>>>>>>>> org.apache.axis2.AxisFault: Failed to submit the response >>>>>>>>> at org.apache.synapse.transport.passthru.PassThroughHttpSender. >>>>>>>>> handleException(PassThroughHttpSender.java:610) >>>>>>>>> at org.apache.synapse.transport.passthru. >>>>>>>>> PassThroughHttpSender.invoke(PassThroughHttpSender.java:269) >>>>>>>>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442) >>>>>>>>> at org.wso2.carbon.core.multitenancy.transports. >>>>>>>>> TenantTransportSender.invoke(TenantTransportSender.java:231) >>>>>>>>> at org.apache.axis2.engine.AxisEngine.send(AxisEngine.java:442) >>>>>>>>> at org.apache.synapse.core.axis2.Axis2Sender.sendBack( >>>>>>>>> Axis2Sender.java:212) >>>>>>>>> at org.apache.synapse.core.axis2.Axis2SynapseEnvironment.send( >>>>>>>>> Axis2SynapseEnvironment.java:444) >>>>>>>>> at org.apache.synapse.mediators.builtin.SendMediator.mediate( >>>>>>>>> SendMediator.java:102) >>>>>>>>> at org.apache.synapse.mediators.AbstractListMediator.mediate( >>>>>>>>> AbstractListMediator.java:81) >>>>>>>>> at org.apache.synapse.mediators.AbstractListMediator.mediate( >>>>>>>>> AbstractListMediator.java:48) >>>>>>>>> at org.apache.synapse.mediators.base.SequenceMediator.mediate( >>>>>>>>> SequenceMediator.java:149) >>>>>>>>> at org.apache.synapse.rest.Resource.process(Resource.java:297) >>>>>>>>> at org.apache.synapse.rest.API.process(API.java:335) >>>>>>>>> at org.apache.synapse.rest.RESTRequestHandler.dispatchToAPI( >>>>>>>>> RESTRequestHandler.java:97) >>>>>>>>> at org.apache.synapse.rest.RESTRequestHandler.process( >>>>>>>>> RESTRequestHandler.java:52) >>>>>>>>> at org.apache.synapse.core.axis2.Axis2SynapseEnvironment. >>>>>>>>> injectMessage(Axis2SynapseEnvironment.java:266) >>>>>>>>> at org.apache.synapse.core.axis2.SynapseCallbackReceiver. >>>>>>>>> handleMessage(SynapseCallbackReceiver.java:529) >>>>>>>>> at org.apache.synapse.core.axis2.SynapseCallbackReceiver.receive( >>>>>>>>> SynapseCallbackReceiver.java:172) >>>>>>>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>>>>>>>> at org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver. >>>>>>>>> processResponse(MultitenantMessageReceiver.java:166) >>>>>>>>> at org.wso2.carbon.core.multitenancy.MultitenantMessageReceiver. >>>>>>>>> receive(MultitenantMessageReceiver.java:82) >>>>>>>>> at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:180) >>>>>>>>> at org.apache.synapse.transport.passthru.ClientWorker.run( >>>>>>>>> ClientWorker.java:247) >>>>>>>>> at org.apache.axis2.transport.base.threads.NativeWorkerPool$ >>>>>>>>> 1.run(NativeWorkerPool.java:172) >>>>>>>>> at java.util.concurrent.ThreadPoolExecutor.runWorker( >>>>>>>>> ThreadPoolExecutor.java:1145) >>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run( >>>>>>>>> ThreadPoolExecutor.java:615) >>>>>>>>> at java.lang.Thread.run(Thread.java:745) >>>>>>>>> Caused by: java.lang.NullPointerException >>>>>>>>> at org.apache.synapse.transport.passthru.util. >>>>>>>>> SourceResponseFactory.create(SourceResponseFactory.java:64) >>>>>>>>> at org.apache.synapse.transport.passthru.PassThroughHttpSender. >>>>>>>>> submitResponse(PassThroughHttpSender.java:462) >>>>>>>>> at org.apache.synapse.transport.passthru. >>>>>>>>> PassThroughHttpSender.invoke(PassThroughHttpSender.java:267) >>>>>>>>> ... 25 more >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> NuwanD. >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Nuwan Dias >>>>>>>>> >>>>>>>>> Technical Lead - WSO2, Inc. http://wso2.com >>>>>>>>> email : [email protected] >>>>>>>>> Phone : +94 777 775 729 >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Sandapa Handakumbura >>>>>>>> Associate Technical Lead >>>>>>>> WSO2.Inc. ; http://wso2.com/ >>>>>>>> M : +94777115866 >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Dev mailing list >>>>>>>> [email protected] >>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> -- >>>>>>> Chanaka Fernando >>>>>>> Senior Technical Lead >>>>>>> WSO2, Inc.; http://wso2.com >>>>>>> lean.enterprise.middleware >>>>>>> >>>>>>> mobile: +94 773337238 >>>>>>> Blog : http://soatutorials.blogspot.com >>>>>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>>>>>> Twitter:https://twitter.com/chanakaudaya >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> -- >>>>>> Chanaka Fernando >>>>>> Senior Technical Lead >>>>>> WSO2, Inc.; http://wso2.com >>>>>> lean.enterprise.middleware >>>>>> >>>>>> mobile: +94 773337238 >>>>>> Blog : http://soatutorials.blogspot.com >>>>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>>>>> Twitter:https://twitter.com/chanakaudaya >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Nuwan Dias >>>>> >>>>> Technical Lead - WSO2, Inc. http://wso2.com >>>>> email : [email protected] >>>>> Phone : +94 777 775 729 >>>>> >>>> >>>> >>>> >>>> -- >>>> -- >>>> Chanaka Fernando >>>> Senior Technical Lead >>>> WSO2, Inc.; http://wso2.com >>>> lean.enterprise.middleware >>>> >>>> mobile: +94 773337238 >>>> Blog : http://soatutorials.blogspot.com >>>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>>> Twitter:https://twitter.com/chanakaudaya >>>> >>>> >>>> >>>> >>>> >>> >>> >>> -- >>> -- >>> Chanaka Fernando >>> Senior Technical Lead >>> WSO2, Inc.; http://wso2.com >>> lean.enterprise.middleware >>> >>> mobile: +94 773337238 >>> Blog : http://soatutorials.blogspot.com >>> LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 >>> Twitter:https://twitter.com/chanakaudaya >>> >>> >>> >>> >>> >> >> >> -- >> Nuwan Dias >> >> Technical Lead - WSO2, Inc. http://wso2.com >> email : [email protected] >> Phone : +94 777 775 729 >> > > > > -- > -- > Chanaka Fernando > Senior Technical Lead > WSO2, Inc.; http://wso2.com > lean.enterprise.middleware > > mobile: +94 773337238 > Blog : http://soatutorials.blogspot.com > LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0 > Twitter:https://twitter.com/chanakaudaya > > > > > -- Thank you and Best Regards, Chanaka Fernando Senior Technical Lead m: +94 773337238 https://wso2.com <https://wso2.com/signature>
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
