Hi Sameera/KasunG,
I have debugged the code to find the root cause for this empty stack
exception. This is causing several other issues at the ESB layer. What
actually happens is that inside the
MultitenantMessageReceiver.processRequest() method, we have the following
code segment.
try {
PrivilegedCarbonContext.startTenantFlow();
PrivilegedCarbonContext privilegedCarbonContext =
PrivilegedCarbonContext.getThreadLocalCarbonContext();
privilegedCarbonContext.setTenantDomain(tenantDomain, true);
// this is to prevent non-blocking transports from sending 202
mainInMsgContext.getOperationContext().setProperty(Constants.RESPONSE_WRITTEN,
"SKIP");
ConfigurationContext tenantConfigCtx =
TenantAxisUtils.getTenantConfigurationContext(tenantDomain,
mainConfigCtx);
if (tenantConfigCtx == null) {
// Throw AxisFault: Tenant does not exist
handleException(mainInMsgContext, new AxisFault("Tenant " +
tenantDomain +
" not found"));
return;
}
if (mainInMsgContext.isDoingREST()) { // Handle REST requests
doREST(mainInMsgContext, to, tenantDomain, tenantConfigCtx,
serviceAndOperation);
} else {
doSOAP(mainInMsgContext, tenantDomain, tenantConfigCtx,
serviceAndOperation);
}
} finally {
PrivilegedCarbonContext.endTenantFlow();
}
When we are calling the endTenantFlow() method, it will go inside the
following method within the CarbonContextDataHolder class.
/**
* This will end the tenant flow and restore the previous CarbonContext.
*/
public void endTenantFlow() {
Stack<CarbonContextDataHolder> carbonContextDataHolders =
parentContextHolderStack.get();
if (carbonContextDataHolders != null) {
currentContextHolder.set(carbonContextDataHolders.pop());
}
}
At the time of calling this method carbonContextDataHolders stack has
become empty and causing the EmptyStackException. When I debug the code, I
found that there is a scheduled task running in a different thread and
accessing the same method frequently. This is coming from
AbstractQuartzTaskManager class method.
public void triggerComplete(Trigger trigger, JobExecutionContext
jobExecutionContext,
Trigger.CompletedExecutionInstruction completedExecutionInstruction) {
PrivilegedCarbonContext.startTenantFlow();
PrivilegedCarbonContext.getThreadLocalCarbonContext().setTenantId(getTenantId(),
true);
if(trigger.getNextFireTime() == null) {
try {
TaskUtils.setTaskFinished(getTaskRepository(),
trigger.getJobKey().getName(), true);
} catch (TaskException e) {
log.error("Error in Finishing Task [" +
trigger.getJobKey().getName() +
"]: " + e.getMessage(), e);
}
}
PrivilegedCarbonContext.endTenantFlow();
}
What I could not figure out is the thread which is emptying the stack. Is
it possible that a different thread can access this
carbonContextDataHolders stack and popping out the element before the
PassThroughMessageProcessor thread access the same?
On Wed, Jul 8, 2015 at 11:12 AM, Jagath Sisirakumara Ariyarathne <
[email protected]> wrote:
> Hi Sameera,
>
> I will check it and update.
>
> Thanks.
>
> On Wed, Jul 8, 2015 at 11:10 AM, Sameera Jayasoma <[email protected]>
> wrote:
>
>> Hi Jagath,
>>
>> Can you debug and see whey the stack becomes empty? Thats a serious
>> problem. Stack should be become empty here.
>>
>> Checking whether the stack is empty will stop the error log, but it
>> doesn't fix the actual problem here.
>>
>> On Wed, Jul 8, 2015 at 10:50 AM, Sameera Jayasoma <[email protected]>
>> wrote:
>>
>>> I understand, but we need to understand why that stack becomes empty.
>>> AFAIK, if we follow the proper APIs, stack should not become empty
>>>
>>> On Tue, Jul 7, 2015 at 5:47 PM, Kasun Indrasiri <[email protected]> wrote:
>>>
>>>> Yeah, we should always check for an empty stack.
>>>>
>>>> On Tue, Jul 7, 2015 at 5:17 PM, Malaka Silva <[email protected]> wrote:
>>>>
>>>>> I think we need to check isEmpty as well.
>>>>>
>>>>> On Tue, Jul 7, 2015 at 3:41 PM, Jagath Sisirakumara Ariyarathne <
>>>>> [email protected]> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I am working on [1] and found that the cause of the exception
>>>>>> mentioned below is in the code segment in org
>>>>>> .wso2.carbon.context.internal.CarbonContextDataHolder in
>>>>>> carbon.utils.
>>>>>>
>>>>>> public void endTenantFlow() {
>>>>>>
>>>>>> Stack<CarbonContextDataHolder> carbonContextDataHolders =
>>>>>> parentContextHolderStack.get();
>>>>>> if (carbonContextDataHolders != null) {
>>>>>> currentContextHolder.set(carbonContextDataHolders.pop());
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> *Exception :*
>>>>>>
>>>>>> java.util.EmptyStackException
>>>>>> at java.util.Stack.peek(Stack.java:102)
>>>>>> at java.util.Stack.pop(Stack.java:84)
>>>>>> at
>>>>>> org.wso2.carbon.context.internal.CarbonContextDataHolder.endTenantFlow(CarbonContextDataHolder.java:1291)
>>>>>> at
>>>>>> org.wso2.carbon.context.PrivilegedCarbonContext.endTenantFlow(PrivilegedCarbonContext.java:75)
>>>>>> at
>>>>>> org.wso2.carbon.ntask.core.impl.TaskQuartzJobAdapter.execute(TaskQuartzJobAdapter.java:69)
>>>>>> at org.quartz.core.JobRunShell.run(JobRunShell.java:213)
>>>>>> at
>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>>>>>> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>>>>>> at
>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>>>>>> at java.lang.Thread.run(Thread.java:745)
>>>>>>
>>>>>> Issue occurs when it tries to pop elements from carbonContextDataHolders
>>>>>> stack when it is empty.
>>>>>>
>>>>>> Is it a possible scenario that this stack being empty and shouldn't it
>>>>>> be handled at CarbonContextDataHolder (check isEmpty in stack)?
>>>>>>
>>>>>> [1] - https://wso2.org/jira/browse/ESBJAVA-3832
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>> --
>>>>>> Jagath Ariyarathne
>>>>>> Technical Lead
>>>>>> WSO2 Inc. http://wso2.com/
>>>>>> Email: [email protected]
>>>>>> Mob : +94 77 386 7048
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Best Regards,
>>>>>
>>>>> Malaka Silva
>>>>> Senior Tech Lead
>>>>> M: +94 777 219 791
>>>>> Tel : 94 11 214 5345
>>>>> Fax :94 11 2145300
>>>>> Skype : malaka.sampath.silva
>>>>> LinkedIn : http://www.linkedin.com/pub/malaka-silva/6/33/77
>>>>> Blog : http://mrmalakasilva.blogspot.com/
>>>>>
>>>>> WSO2, Inc.
>>>>> lean . enterprise . middleware
>>>>> http://www.wso2.com/
>>>>> http://www.wso2.com/about/team/malaka-silva/
>>>>> <http://wso2.com/about/team/malaka-silva/>
>>>>>
>>>>> Save a tree -Conserve nature & Save the world for your future. Print
>>>>> this email only if it is absolutely necessary.
>>>>>
>>>>> _______________________________________________
>>>>> Dev mailing list
>>>>> [email protected]
>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Kasun Indrasiri
>>>> Software Architect
>>>> WSO2, Inc.; http://wso2.com
>>>> lean.enterprise.middleware
>>>>
>>>> cell: +94 77 556 5206
>>>> Blog : http://kasunpanorama.blogspot.com/
>>>>
>>>> _______________________________________________
>>>> Dev mailing list
>>>> [email protected]
>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>>>
>>>>
>>>
>>>
>>> --
>>> Sameera Jayasoma,
>>> Software Architect,
>>>
>>> WSO2, Inc. (http://wso2.com)
>>> email: [email protected]
>>> blog: http://blog.sameera.org
>>> twitter: https://twitter.com/sameerajayasoma
>>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
>>> Mobile: 0094776364456
>>>
>>> Lean . Enterprise . Middleware
>>>
>>>
>>
>>
>> --
>> Sameera Jayasoma,
>> Software Architect,
>>
>> WSO2, Inc. (http://wso2.com)
>> email: [email protected]
>> blog: http://blog.sameera.org
>> twitter: https://twitter.com/sameerajayasoma
>> flickr: http://www.flickr.com/photos/sameera-jayasoma/collections
>> Mobile: 0094776364456
>>
>> Lean . Enterprise . Middleware
>>
>>
>> _______________________________________________
>> Dev mailing list
>> [email protected]
>> http://wso2.org/cgi-bin/mailman/listinfo/dev
>>
>>
>
>
> --
> Jagath Ariyarathne
> Technical Lead
> WSO2 Inc. http://wso2.com/
> Email: [email protected]
> Mob : +94 77 386 7048
>
>
> _______________________________________________
> Dev mailing list
> [email protected]
> http://wso2.org/cgi-bin/mailman/listinfo/dev
>
>
--
--
Chanaka Fernando
Senior Technical Lead
WSO2, Inc.; http://wso2.com
lean.enterprise.middleware
mobile: +94 773337238
Blog : http://soatutorials.blogspot.com
LinkedIn:http://www.linkedin.com/pub/chanaka-fernando/19/a20/5b0
Twitter:https://twitter.com/chanakaudaya
_______________________________________________
Dev mailing list
[email protected]
http://wso2.org/cgi-bin/mailman/listinfo/dev
