Thank you very much Mauricio. You've been of great help. Now is time for us to cook all this valuable information and go for a solution. xD
Alberto R. Galdo arga...@gmail.com On Tue, May 29, 2012 at 5:59 PM, Mauricio Salatino <sala...@gmail.com>wrote: > Comments inline, > > On Tue, May 29, 2012 at 12:24 PM, Alberto R. Galdo <arga...@gmail.com>wrote: > >> Sure!. I didn't mention that we're running in a OSGi environment and >> we're getting the session from JPAKnowledgeService like that: >> >> >> >> EntityManagerFactory emf = >> Persistence.createEntityManagerFactory("com.package"); >> >> Environment env = KnowledgeBaseFactory.newEnvironment(); >> >> env.set(EnvironmentName.ENTITY_MANAGER_FACTORY, emf); >> >> env.set(EnvironmentName.TRANSACTION_MANAGER,(TransactionManager) >> bundleContext.getService(this.tmServiceRef)); >> >> env.set(EnvironmentName.TRANSACTION, (UserTransaction) >> bundleContext.getService(this.userTransactionServiceRef)); >> >> env.set(EnvironmentName.GLOBALS, new MapGlobalResolver()); >> >> env.set(EnvironmentName.OBJECT_MARSHALLING_STRATEGIES, new >> ObjectMarshallingStrategy[] { >> MarshallerFactory.newSerializeMarshallingStrategy() }); >> >> final StatefulKnowledgeSession ksession = >> JPAKnowledgeService.newStatefulKnowledgeSession(kbase, sessionConfig, env); >> >> >> We are just following the manual ( and some acquired knowledge through >> the days ... ) .... I understand that inserting an event and starting a >> process at the same time would lead to the current situation, but Isn't >> this the reason why we are using drools with drools fusion + jbpm to build >> a CEP? >> > I'm not sure about your reasons :) > >> >> We chose Drools and a statefulsession in a fireUntilHalt loop just >> because we would be able to have our rules and processes reacting to events >> that enter the system at their own pace ... and found ourselves in this >> nightmare when trying to make the system fault-tolerant through >> persistence... :( >> >> FireUntilHalt and Persistence will cause "conflicts", because > fireUntilHalts by default creates a different thread == race conditions. > > >> Do you think we should change our fault-tolerance strategy by >> reconstructing the knowledge session, process and tasks state by our own >> means leaving Drools and JBPM JPA persistence appart? >> >> A common problem is to think that a session will do everything for you. > I'm not saying that you need to separate your processes from your rules, > I'm just saying that probably decoupling responsibilities will help you to > tackle down some of the problems that you are having. If you have in memory > processes and fusion entry points, you can keep those together and in a > separate persistence session handle all the human task interactions and > long running processes. If you are getting events in real time and you want > to process them, you cannot expect to persist all the session state and > have no performance impacts. > >> >> Alberto R. Galdo >> arga...@gmail.com >> >> >> Cheers > >> >> On Tue, May 29, 2012 at 4:43 PM, Mauricio Salatino <sala...@gmail.com>wrote: >> >>> Ok, so probably I'm not understanding your scenario, but are you using >>> the JPAKnowledgeService to create persistent sessions? >>> If you use that everything is aware of everything :) Drools and jBPM5 >>> will run using the same persistence resources. For each interaction: >>> startProcess, insert, update, fireAllRules a transaction will be created. >>> If you are starting a process and at the same time inserting a Fusion event >>> you will be generating two transaction that will fight for the resources >>> (the same resources because you have only one session). >>> Obviously if you take the persistence mechanisms outside of the >>> discussion everything will work perfectly fine, because you don't have any >>> transactional resource to fight for :) >>> >>> Hope it helps. >>> Cheers >>> >>> >>> On Tue, May 29, 2012 at 11:36 AM, Alberto R. Galdo <arga...@gmail.com>wrote: >>> >>>> Your answer is confusing me, I think I'm not getting it right. >>>> >>>> It seems that we you're saying that will have to modify JBPM or >>>> Drools behaviour to get our architecture done ... but I don't think we are >>>> doing nothing special than what Drools & JBPM are able to do in a >>>> non-persisting enviroment ( at least without JBPM persisted ). >>>> >>>> Maybe deeping a bit more the architecture will lead to a better >>>> understanding of the problem: >>>> >>>> We have a Drools StatefulKnowledgeSession wich is populated with >>>> packages of rules and different BPMN 2.0 process definitions. That session >>>> in Drools is persisted using JPA and our JTA provider. Our solution is a >>>> CEP that uses events that get into the session using Drools Fusion. There, >>>> several rules fire and then several consequences start processes wich >>>> contain both Tasks and HumanTasks. Those rule activations are already in a >>>> queue, the Agenda. Given that JBPM acts as the processmanager provider for >>>> Drools, our processes get started in JBPM, and it is persisted using it's >>>> own JPA manager. Just rules, that start processes in response to events in >>>> a fireUntilHalt loop in a persistent enviroment, both for Drools & JBPM. >>>> Then some processes interact with the knowledge session using BussinessTask >>>> nodes( which make other rules fire and maybe launch a different set of >>>> processes ) and other simply get to the Stop node. I see nothing wrong >>>> here, in fact, taking the persistence appart from the equation, everything >>>> runs as beautifuly as expected. We've tested that. >>>> >>>> I'm just trying to undersand the problem, I'm pretty 90% sure I'm >>>> wrong, but what I think is happening here is that Drools isn't aware that >>>> JBPM may be using the same resources and vice-versa, Isn't it? >>>> >>>> Shouldn't the solution be to make Drools & JBPM aware of each other >>>> in what is related to the syncronization of the transaction management to >>>> avoid race conditions? >>>> >>>> >>>> >>>> >>>> Alberto R. Galdo >>>> arga...@gmail.com >>>> >>>> >>>> On Tue, May 29, 2012 at 3:58 PM, Mauricio Salatino >>>> <sala...@gmail.com>wrote: >>>> >>>>> I'm in no way saying this: >>>>> " Let me see if I'm getting this right, What you mean is that there >>>>> is no way for the JPA persistence of Drools & JBPM to coexist in the same >>>>> knowledge session because of race conditions between them competing for >>>>> their shared resources? I'm afraid so.. :( >>>>> >>>>> They can coexist. As you can see they are working together. If you >>>>> take a look at the underlying layers you will see that each operation that >>>>> you run against the session is wrapped in a transaction. That leads to >>>>> your >>>>> architecture, If you have a single entry point for your session everything >>>>> will work correctly. You can achieve this by receiving all the operations >>>>> in a queue and then execute one after the other. >>>>> If you have multiple threads interacting with the session, you can >>>>> implement a retrying mechanism if the transaction gets rolled back and as >>>>> soon as the transaction win the right to commit it will work. It really >>>>> depends on the problem that you are trying to solve. >>>>> >>>>> Cheers >>>>> >>>>> On Tue, May 29, 2012 at 10:42 AM, Alberto R. Galdo >>>>> <arga...@gmail.com>wrote: >>>>> >>>>>> Thanks Mauricio for your kind response, and kick, as usual ;-) >>>>>> >>>>>> Sure, we see lots of rolled back transactions ... Everywhere! >>>>>> >>>>>> Let me see if I'm getting this right, What you mean is that there >>>>>> is no way for the JPA persistence of Drools & JBPM to coexist in the same >>>>>> knowledge session because of race conditions between them competing for >>>>>> their shared resources? I'm afraid so.. :( >>>>>> >>>>>> If it is so, Are there any plans for the integration of a shared >>>>>> JPA persistence management of both Drools & JBPM, wich may be the >>>>>> solution >>>>>> for this problem? ... Will we be better thinking to change our >>>>>> architecture >>>>>> ASAP? >>>>>> >>>>>> Greets, >>>>>> >>>>>> Alberto R. Galdo >>>>>> arga...@gmail.com >>>>>> >>>>>> >>>>>> >>>>>> On Tue, May 29, 2012 at 1:54 PM, Mauricio Salatino <sala...@gmail.com >>>>>> > wrote: >>>>>> >>>>>>> Ok, so what is happening is that two or more threads are fighting >>>>>>> for the resources, causing that two or more transactions wants to be >>>>>>> completed, but just one can win. All the other must be retried. Are you >>>>>>> seeing rolled back exceptions in your stack trace? >>>>>>> My question to you is if you really need to handle everything in >>>>>>> just one big persistent session. I've solved similar issues in the past >>>>>>> using more than one session, for example one session to run business >>>>>>> processes(persistent) and one or more for receiving and reacting to >>>>>>> events. >>>>>>> If you have an async way to communicate both sessions there should be no >>>>>>> problem. >>>>>>> Hope it helps! >>>>>>> Cheers >>>>>>> >>>>>>> On Tue, May 29, 2012 at 7:30 AM, Alberto R. Galdo <arga...@gmail.com >>>>>>> > wrote: >>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> We've been for long time now developing a complex event >>>>>>>> processing system that ( simplified version ahead !! ) involves a set >>>>>>>> of >>>>>>>> rules that in turn activates a set processes that fulfill human tasks >>>>>>>> and >>>>>>>> other kind of tasks.. This all is running in a StatefulKnowledgeSession >>>>>>>> with JPA persistence configured both for Drools and JBPM. We are >>>>>>>> running an >>>>>>>> stack of Drools, JBPM, Drools Integration, Drools fussion, etc.. >>>>>>>> >>>>>>>> We've been able to persist Drools sessions & JBPM processes in >>>>>>>> the same Persistence Context ( not without pain :( ) using different >>>>>>>> JTA >>>>>>>> implementations including ( but not limited to ) Bitronix & Atomikos. >>>>>>>> >>>>>>>> What we are observing here is what seems to be some kind of >>>>>>>> race condition between Drools and JBPM when running the knowledge >>>>>>>> session >>>>>>>> when JPA&JTA persistence is configured. Very often, as soon as after >>>>>>>> 2-3 >>>>>>>> processes get created as rule's consequences are fired in response of >>>>>>>> events inside the session we see how JBPM finds its instance nullified >>>>>>>> by >>>>>>>> Drools when it tries to end a process and persist it. >>>>>>>> >>>>>>>> We've been able to find where Drools decides to delete an >>>>>>>> instance of the process ... at a given time Drools executes >>>>>>>> JPAProcessInstanceManager.clearProcessInstances() [1] when it >>>>>>>> finalizes a >>>>>>>> SingleSessionCommand wich in turn calls disconnect() for *all* the >>>>>>>> local-stored processinstances ( wich gets populated with instances of >>>>>>>> processes every time a process is started in the knowledge session ): >>>>>>>> >>>>>>>> public void clearProcessInstances() { >>>>>>>> >>>>>>>> >>>>>>>> for (ProcessInstance processInstance: new >>>>>>>> ArrayList<ProcessInstance>(processInstances.values())) { >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> ((ProcessInstanceImpl) processInstance).disconnect(); >>>>>>>> >>>>>>>> } >>>>>>>> } >>>>>>>> >>>>>>>> >>>>>>>> So, Drools decides to disconnect all process instances in it's >>>>>>>> JPA context without taking in account the state the process is in, and >>>>>>>> when >>>>>>>> an processinstance that is not stopped gets removed then JBPM finds >>>>>>>> it's >>>>>>>> NullPointerException... >>>>>>>> >>>>>>>> We've modified the code to make Drools aware of the state of >>>>>>>> the process before wiping it from the context ( no problem here, >>>>>>>> there >>>>>>>> will be no leak as a running processinstance will be removed in future >>>>>>>> calls of "clearProcessInstances" given the process is closed ). But >>>>>>>> unfortunatelly this seems to resolve this problem, but lots of other >>>>>>>> problems ( wich seems also race conditions arise : for instance: >>>>>>>> Drools >>>>>>>> closes connections to the database and JBPM finds the connection >>>>>>>> closed, >>>>>>>> >>>>>>>> >>>>>>>> So, we are really worried about using Drools & JBPM in a >>>>>>>> persisted environment. Maybe our asumptions are wrong... Is it >>>>>>>> possible to >>>>>>>> have an scenario like ours given the current Drools & JBPM integration >>>>>>>> status for a persistent statefulKnowledge Session? Did anyone build a >>>>>>>> complex event processing system like ours in a unaltered persistence >>>>>>>> environment such as provided in Drools and JBPM by default? >>>>>>>> >>>>>>>> >>>>>>>> Greets, >>>>>>>> >>>>>>>> [1] >>>>>>>> https://github.com/droolsjbpm/jbpm/blob/master/jbpm-persistence-jpa/src/main/java/org/jbpm/persistence/processinstance/JPAProcessInstanceManager.java >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> Alberto R. Galdo >>>>>>>> arga...@gmail.co <arga...@gmail.com> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> rules-users mailing list >>>>>>>> rules-users@lists.jboss.org >>>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> - MyJourney @ http://salaboy.wordpress.com >>>>>>> - Co-Founder @ http://www.jugargentina.org >>>>>>> - Co-Founder @ http://www.jbug.com.ar >>>>>>> >>>>>>> - Salatino "Salaboy" Mauricio - >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> rules-users mailing list >>>>>>> rules-users@lists.jboss.org >>>>>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> rules-users mailing list >>>>>> rules-users@lists.jboss.org >>>>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> - MyJourney @ http://salaboy.wordpress.com >>>>> - Co-Founder @ http://www.jugargentina.org >>>>> - Co-Founder @ http://www.jbug.com.ar >>>>> >>>>> - Salatino "Salaboy" Mauricio - >>>>> >>>>> >>>>> _______________________________________________ >>>>> rules-users mailing list >>>>> rules-users@lists.jboss.org >>>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> rules-users mailing list >>>> rules-users@lists.jboss.org >>>> https://lists.jboss.org/mailman/listinfo/rules-users >>>> >>>> >>> >>> >>> -- >>> - MyJourney @ http://salaboy.wordpress.com >>> - Co-Founder @ http://www.jugargentina.org >>> - Co-Founder @ http://www.jbug.com.ar >>> >>> - Salatino "Salaboy" Mauricio - >>> >>> >>> _______________________________________________ >>> rules-users mailing list >>> rules-users@lists.jboss.org >>> https://lists.jboss.org/mailman/listinfo/rules-users >>> >>> >> >> _______________________________________________ >> rules-users mailing list >> rules-users@lists.jboss.org >> https://lists.jboss.org/mailman/listinfo/rules-users >> >> > > > -- > - MyJourney @ http://salaboy.wordpress.com > - Co-Founder @ http://www.jugargentina.org > - Co-Founder @ http://www.jbug.com.ar > > - Salatino "Salaboy" Mauricio - > > > _______________________________________________ > rules-users mailing list > rules-users@lists.jboss.org > https://lists.jboss.org/mailman/listinfo/rules-users > >
_______________________________________________ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users