We can still properly fix this by using the following way. The root cause was, the Map (which contains the deployers) was modified during scheduler task's run method execution (we can identify this with the error trace). It was modified due to adding CAppDeployer separately. If we control (synchronize) this addition based on axisConfig object, which is the same that the scheduler task is also synchronized on, then we can avoid this issue.
I have added the fix for this in r190400. On Thu, Sep 26, 2013 at 10:58 PM, Manoj Kumara <[email protected]> wrote: > Hi Shameera, > > This is an intermittent issue and during start-up only few times this > error is generated. Even is was really difficult to reproduced and no one > is really aware about re producing steps. I think that's why did't occurred > this during AS testing. > > I think what you mentioned by calling the deployer twice is explained in > the issue [1]. Is this what you refering ? > > [1] https://wso2.org/jira/browse/CARBON-14468 > > Thanks, > Manoj > > Best Regards.. > > > Manoj Kumara > Software Engineer > WSO2, Inc.; http://wso2.com > > Twitter: http://twitter.com/ManKuma > Mobile: +94713448188 > > > On Thu, Sep 26, 2013 at 9:39 PM, Shameera Rathnayaka <[email protected]>wrote: > >> Hi Manoj, >> >> On Thu, Sep 26, 2013 at 7:49 AM, Manoj Kumara <[email protected]> wrote: >> >>> Hi, >>> >>> With the help of Azeez and Sameera we figure out the root cause for the >>> problem. This was due to some previous fix by Adding the CApp deployer >>> separately in the code. >>> >> >> Great this is the real fix i think BTW wonder why we didn't get thhis >> issue with App Server?, root cause is not the one i explain, in a flow we >> are calling deployers twice If i am right. >> >> Thanks, >> Shameera. >> >> >> >>> Will commit the fix shortly. >>> >>> Thanks, >>> >>> Best Regards.. >>> >>> >>> Manoj Kumara >>> Software Engineer >>> WSO2, Inc.; http://wso2.com >>> >>> Twitter: http://twitter.com/ManKuma >>> Mobile: +94713448188 >>> >>> >>> On Thu, Sep 26, 2013 at 2:09 PM, Manoj Kumara <[email protected]> wrote: >>> >>>> Hi, >>>> >>>> The problem here is as Shameera mentioned on [1] the Map used inside >>>> the concurrentHashMap is not thread safe. >>>> >>>> >>>> >>>> >>>> private Map<String, Map<String, Deployer>> deployerMap = new >>>> ConcurrentHashMap<String, Map<String, Deployer>>(); >>>> >>>> I think we can relove this by making this Map thread safe as given on >>>> [2]. >>>> >>>> [1] https://wso2.org/jira/browse/CARBON-14275 >>>> [2] >>>> http://docs.oracle.com/javase/7/docs/api/java/util/Collections.html#synchronizedMap(java.util.Map) >>>> >>>> Mean time I will try to patch this and find this get solved. >>>> >>>> Thanks, >>>> Manoj >>>> >>>> Best Regards.. >>>> >>>> >>>> Manoj Kumara >>>> Software Engineer >>>> WSO2, Inc.; http://wso2.com >>>> >>>> Twitter: http://twitter.com/ManKuma >>>> Mobile: +94713448188 >>>> >>>> >>>> On Thu, Sep 26, 2013 at 10:36 AM, Manoj Kumara <[email protected]> wrote: >>>> >>>>> Hi Shameera, >>>>> >>>>> Thanks for the info. I'll check. >>>>> >>>>> Thanks, >>>>> Manoj >>>>> >>>>> Best Regards.. >>>>> >>>>> >>>>> Manoj Kumara >>>>> Software Engineer >>>>> WSO2, Inc.; http://wso2.com >>>>> >>>>> Twitter: http://twitter.com/ManKuma >>>>> Mobile: +94713448188 >>>>> >>>>> >>>>> On Wed, Sep 25, 2013 at 7:08 PM, Shameera Rathnayaka < >>>>> [email protected]> wrote: >>>>> >>>>>> Hi Ajith, >>>>>> >>>>>> Could you able to check whether this is a problem of HumenTask call >>>>>> twice in execution flow? >>>>>> >>>>>> Hi Manoj, >>>>>> >>>>>> I have created a JIRA for this issue[1] and added my finding there >>>>>> please have a look at that. As i explain in the JIRA we could solve it by >>>>>> making innere map to ConcurrentHashMap.This will produce another issue >>>>>> becuase we can't put null as a key or value to a Concurrent HashMap and >>>>>> we >>>>>> do this in above map in such a case where we don't have extension for a >>>>>> particular deployer (eg: jaggery deployer doesn't has any extension >>>>>> associate it hence it is registered with null as it extension). >>>>>> >>>>>> [1] https://wso2.org/jira/browse/CARBON-14275 >>>>>> >>>>>> >>>>>> Thanks, >>>>>> Shameera. >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Sep 25, 2013 at 8:37 AM, Manoj Kumara <[email protected]> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> Tried to reproduce with 21st Sept GReg pack but could not >>>>>>> reproduced. I will keep on looking to this. >>>>>>> >>>>>>> Thanks, >>>>>>> Manoj >>>>>>> >>>>>>> Best Regards.. >>>>>>> >>>>>>> >>>>>>> Manoj Kumara >>>>>>> Software Engineer >>>>>>> WSO2, Inc.; http://wso2.com >>>>>>> >>>>>>> Twitter: http://twitter.com/ManKuma >>>>>>> Mobile: +94713448188 >>>>>>> >>>>>>> >>>>>>> On Wed, Sep 25, 2013 at 10:49 AM, Manoj Kumara <[email protected]>wrote: >>>>>>> >>>>>>>> Hi Sumedha, >>>>>>>> >>>>>>>> I'll work on this. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Manoj >>>>>>>> >>>>>>>> Best Regards.. >>>>>>>> >>>>>>>> >>>>>>>> Manoj Kumara >>>>>>>> Software Engineer >>>>>>>> WSO2, Inc.; http://wso2.com >>>>>>>> >>>>>>>> Twitter: http://twitter.com/ManKuma >>>>>>>> Mobile: +94713448188 >>>>>>>> >>>>>>>> >>>>>>>> On Wed, Sep 25, 2013 at 10:32 AM, Sumedha Rubasinghe < >>>>>>>> [email protected]> wrote: >>>>>>>> >>>>>>>>> Sameera, >>>>>>>>> Can we have someone to attend to this pls? >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Sep 13, 2013 at 3:57 PM, Ajith Vitharana >>>>>>>>> <[email protected]>wrote: >>>>>>>>> >>>>>>>>>> Hi All, >>>>>>>>>> >>>>>>>>>> Observed the following error in the latest pack (G-Reg) - chunk02 >>>>>>>>>> >>>>>>>>>> [2013-09-13 15:52:39,357] ERROR >>>>>>>>>> {org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask} - >>>>>>>>>> Error >>>>>>>>>> while running deployment scheduler.. >>>>>>>>>> java.util.ConcurrentModificationException >>>>>>>>>> at java.util.HashMap$HashIterator.nextEntry(HashMap.java:793) >>>>>>>>>> at java.util.HashMap$EntryIterator.next(HashMap.java:834) >>>>>>>>>> at java.util.HashMap$EntryIterator.next(HashMap.java:832) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.deployment.RepositoryListener.loadOtherDirectories(RepositoryListener.java:270) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.deployment.RepositoryListener.checkServices(RepositoryListener.java:253) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.deployment.RepositoryListener.startListener(RepositoryListener.java:371) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.deployment.scheduler.SchedulerTask.checkRepository(SchedulerTask.java:59) >>>>>>>>>> at >>>>>>>>>> org.apache.axis2.deployment.scheduler.SchedulerTask.run(SchedulerTask.java:67) >>>>>>>>>> at >>>>>>>>>> org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.runAxisDeployment(CarbonDeploymentSchedulerTask.java:65) >>>>>>>>>> at >>>>>>>>>> org.wso2.carbon.core.deployment.CarbonDeploymentSchedulerTask.run(CarbonDeploymentSchedulerTask.java:110) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>>>>>>>>> at >>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>>>>>>>>> at java.lang.Thread.run(Thread.java:662) >>>>>>>>>> [2013-09-13 15:52:39,614] INFO >>>>>>>>>> {org.apache.tomcat.util.net.NioSelectorPool} - Using a shared >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Ajith Vitharana. >>>>>>>>>> WSO2 Inc. - http://wso2.org >>>>>>>>>> Email : [email protected] >>>>>>>>>> Mobile : +94772217350 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> _______________________________________________ >>>>>>>>>> Dev mailing list >>>>>>>>>> [email protected] >>>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> /sumedha >>>>>>>>> m: +94 773017743 >>>>>>>>> b : bit.ly/sumedha >>>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> Dev mailing list >>>>>>>>> [email protected] >>>>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Dev mailing list >>>>>>> [email protected] >>>>>>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> *Software Engineer - WSO2 Inc.* >>>>>> *email: shameera AT wso2.com <[email protected]> , shameera AT >>>>>> apache.org <[email protected]>* >>>>>> *phone: +9471 922 1454 <%2B9471%20922%201454>* >>>>>> >>>>>> *Linked in : * >>>>>> http://lk.linkedin.com/pub/shameera-rathnayaka/1a/661/561 >>>>>> *Twitter : *https://twitter.com/Shameera_R >>>>>> >>>>> >>>>> >>>> >>> >> >> >> -- >> *Software Engineer - WSO2 Inc.* >> *email: shameera AT wso2.com <[email protected]> , shameera AT apache.org >> <[email protected]>* >> *phone: +9471 922 1454 <%2B9471%20922%201454>* >> >> *Linked in : *http://lk.linkedin.com/pub/shameera-rathnayaka/1a/661/561 >> *Twitter : *https://twitter.com/Shameera_R >> > > > _______________________________________________ > Dev mailing list > [email protected] > http://wso2.org/cgi-bin/mailman/listinfo/dev > > -- *Kishanthan Thangarajah* Senior Software Engineer, Platform Technologies Team, WSO2, Inc. lean.enterprise.middleware Mobile - +94773426635 Blog - *http://kishanthan.wordpress.com <http://kishanthan.wordpress.com>* Twitter - *http://twitter.com/kishanthan <http://twitter.com/kishanthan>*
_______________________________________________ Dev mailing list [email protected] http://wso2.org/cgi-bin/mailman/listinfo/dev
