Re: [rules-users] Drools Timers
Hi all, I've a big rule file that contains *many* rules like this: rule rule1 when $obj1: OBJ(a==false, b==C) $obj2: OBJ(b==B, a==false) then ... end Inspecting the RETE tree I've noticed that the first layer contains two alpha node, one for a==false and one for b==B, than the second layer contains the duplicate alpha node for a==false and one for b==C. This is somewhat (at least for me) counter intuitive, because I thought the algorithm would be a little bit smarter trying to order the clauses from the most restrictive to the least. I there a particular reason why this optimization it's not implemented? It's difficult or semantically incorrect? I've noticed this because compiling this file with many rules which have no particular order on clauses led to a tree with a extremely large first level. This obviously was detrimental but a simple ordering of the clauses fixed it. This became a problem only when we upgraded form 5.0.1 to 6.1.0-Final. Why I didn't noticed any performance problem with the previous version of Drools? Thanks in advance. Federico. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] StackOverflow when serializing KieBase
I'll try to investigating a bit (or at least I'll try to make something reproducible). Meanwhile, this is the stack trace. It seems to me that is not the case as https://issues.jboss.org/browse/JBRULES-2335. Any help is appreciated. Federico. On 07/31/2014 01:25 AM, Mark Proctor wrote: It should work, there should not be too much recursion. I’d suggest digging deeper to see if you can point us to the references that are suspect. Mark On 30 Jul 2014, at 17:28, Federico Bertola federico.bertol...@gmail.com wrote: Hi all, when I try to serialize a rather large KieBase I constantly get the aforementioned error. After a bit of tracking I see that the error seems to appear when the KnowledgeBase try to serialize the Rete object. I belive this is not an infinite loop because it seems that the graph is quite deep and the serializer recursively calls writeExternal in a (sort of) depth-first fashion. Increasing the stack size didn't really help, so I'm asking if there's a better way or if I should give up. Thanks :) Federico. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users java.lang.StackOverflowError at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at org.drools.core.common.BaseNode.writeExternal(BaseNode.java:78) at org.drools.core.reteoo.LeftTupleSource.writeExternal(LeftTupleSource.java:107) at org.drools.core.reteoo.LeftInputAdapterNode.writeExternal(LeftInputAdapterNode.java:143) at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1458) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at org.drools.core.reteoo.ObjectSinkNodeList.writeExternal(ObjectSinkNodeList.java:88) at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1458) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at org.drools.core.reteoo.CompositeObjectSinkAdapter.writeExternal(CompositeObjectSinkAdapter.java:88) at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1458) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at org.drools.core.reteoo.ObjectSource.writeExternal(ObjectSource.java:119) at org.drools.core.reteoo.AlphaNode.writeExternal(AlphaNode.java:106) at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1458) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at org.drools.core.reteoo.BetaNode.writeExternal(BetaNode.java:245) at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1458) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at org.drools.core.reteoo.SingleLeftTupleSinkAdapter.writeExternal(SingleLeftTupleSinkAdapter.java:192) at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1458) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at org.drools.core.reteoo.LeftTupleSource.writeExternal(LeftTupleSource.java:108) at org.drools.core.reteoo.BetaNode.writeExternal(BetaNode.java:254) at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1458) at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1429) at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1177) at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:347) at org.drools.core.reteoo.ObjectSinkNodeList.writeExternal(ObjectSinkNodeList.java:87) at java.io.ObjectOutputStream.writeExternalData(ObjectOutputStream.java:1458
[rules-users] StackOverflow when serializing KieBase
Hi all, when I try to serialize a rather large KieBase I constantly get the aforementioned error. After a bit of tracking I see that the error seems to appear when the KnowledgeBase try to serialize the Rete object. I belive this is not an infinite loop because it seems that the graph is quite deep and the serializer recursively calls writeExternal in a (sort of) depth-first fashion. Increasing the stack size didn't really help, so I'm asking if there's a better way or if I should give up. Thanks :) Federico. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] Serialization issue.
Dear all, Some time ago I've submitted a pull request (#353) fixing a situation where a rule with large text prevent the serialization of the knowledge package. This is now the only thing that prevent us form using the original distribution. If there's nothing wrong or risky in it, it is possible for you apply the fix so we can use the latest snapshot? Thank you very much for your patience :) Regards, Federico. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Is protobuf cache actually used?
Hi Edson, now I figured that out! Thanks for your patience :) Federico. On 07/08/2014 02:13 PM, Edson Tirelli wrote: It is a bit convoluted, indeed, but AFAIK, it is using the cache and working fine. The cache is populated here: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/main/java/org/drools/compiler/kie/builder/impl/AbstractKieModule.java#L301 It is set here: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/main/java/org/drools/compiler/kie/builder/impl/AbstractKieModule.java#L180 And we have some tests here: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/test/java/org/drools/compiler/integrationtests/KieCompilationCacheTest.java Edson On Mon, Jul 7, 2014 at 7:01 PM, Federico Bertola federico.bertol...@gmail.com wrote: Hi Edson, yes I saw that the JavaDialect class retrieve the compilation cache from the conf, but after a quick search it seems no one use the setCompilationCache method of KnowledgeBuilderConfigurationImpl, not even in the tests. So I was wondering what was the best way to use it... Maybe I get something wrong, in that case, I apologize :) Federico. On 07/07/2014 21:04, Edson Tirelli wrote: Federico, Not sure why you say it is not used? Drools tries to use any pre-compiled resource bytecode if it is available in the cache. If it is not available, it falls back to JIT compilation: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/main/java/org/drools/compiler/rule/builder/dialect/java/JavaDialect.java#L627 Edson On Sat, Jul 5, 2014 at 11:34 AM, Federico Bertola federico.bertol...@gmail.com wrote: Hi, from what I see, the protobuf cache that the KnowledgeBuilder produce and saves as *.cache, is not currently used anywhere, right? If so, what would it be required to reload the compilation data inside the KnowledgeBuilder? There's something I'm missing or there is some piece of code demonstrating this capability? Best regards, Federico. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Is protobuf cache actually used?
Hi Edson, yes I saw that the JavaDialect class retrieve the compilation cache from the conf, but after a quick search it seems no one use the setCompilationCache method of KnowledgeBuilderConfigurationImpl, not even in the tests. So I was wondering what was the best way to use it... Maybe I get something wrong, in that case, I apologize :) Federico. On 07/07/2014 21:04, Edson Tirelli wrote: Federico, Not sure why you say it is not used? Drools tries to use any pre-compiled resource bytecode if it is available in the cache. If it is not available, it falls back to JIT compilation: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/main/java/org/drools/compiler/rule/builder/dialect/java/JavaDialect.java#L627 Edson On Sat, Jul 5, 2014 at 11:34 AM, Federico Bertola federico.bertol...@gmail.com wrote: Hi, from what I see, the protobuf cache that the KnowledgeBuilder produce and saves as *.cache, is not currently used anywhere, right? If so, what would it be required to reload the compilation data inside the KnowledgeBuilder? There's something I'm missing or there is some piece of code demonstrating this capability? Best regards, Federico. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] Is protobuf cache actually used?
Hi, from what I see, the protobuf cache that the KnowledgeBuilder produce and saves as *.cache, is not currently used anywhere, right? If so, what would it be required to reload the compilation data inside the KnowledgeBuilder? There's something I'm missing or there is some piece of code demonstrating this capability? Best regards, Federico. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] Problems compiling huge rule base with kie-maven-plugin
Hi all, at work we have a huge rule base (2k rules with an average size of ~2Mib). I've found the building this RB with the latest kie-maven-plugin troublesome at best. I want to share my experience hoping that some Drools developer could help me. I'm using the latest Drools 6.x from the master branch. The first problem I encounterd was that the compilation process retain in memory all the compilation caches until the end. This would lead to a OOM very soon and we can't increase the heap size indefinitely. So the first thing I do was to extract the code from KieModuleMetaInfoBuilder and create a custom builder that would populate a guava cache for every KnowledgeBase compiled. The aforementioned cache would write the bytestreams when the objects get evicted. That worked well and leaded finally to a full compilation. The only problem was the long time one has to wait until completion. Some few rules take up to 30 min. to complete but the majority takes from 5 to 20 secs. So I extended the custom compiler to be multithreaded and finally the compilation time is acceptable again. I also extended the compiler to perform incremental compilations. Now my problem is that the generated kjar is rather huge and when I load it I get an OOM again. Peeking at the code again confirm (if I get this correctly) that all the caches gets loaded altogether, moving the problem to compile time to run time. Is it possible to force the KieContainer to load the caches lazily, based on calls to getKieBase? If not, would it be usefull? I understand that my case is rather on the edge but any help would be appreciated. Thank you very much :) Federico. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Problems compiling huge rule base with kie-maven-plugin
Hi Mario, thanks for the quick reply. Yes I could definitely open a jira and a pull request, just be patient though, as I've to isolate the code from our own. Reproducing the slowdown I experienced would be a little problematic because it depends, I think, entirely on the size (and shape) of our RB. Thanks again, Federico. On 06/18/2014 05:48 PM, Mario Fusco wrote: Hi Federico, so we have 2 problems here. For what regards the compilation one I'd be curious to see your solution. Could you please open a ticket on our jira and attach it there, or maybe send a pull request on github? As for the runtime, yes probably loading stuff lazier in the KieContainer could alleviate also that problem, but first I'd like to give a look at what you're doing to better understand your use case. Could you also attach a reproducer to that jira ticket? Thanks, Mario -- View this message in context: http://drools.46999.n3.nabble.com/Possible-problem-in-PathMemory-tp4030026p4030093.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] Possible bug with incremental compilation
Hi all,
executing the test in
drools/drools-compiler/src/test/java/org/drools/compiler/integrationtests/
all goes well except if i write a kmodule xml inside the KieFileSystem.
Is this supposed to happen? I cannot add or remove a KieBase during
incremental compilation, or I'm doing something dumb?
I'm on Ubuntu 12.04 64bit java 1.7.0_55
Heres a simple test case:
#
@Test
public void shouldRecompileWithNewBases() {
String drl1 = global java.util.List list\n + rule R1 when\n
+ $s : String() + then\n + list.add( \a\ + $s );
+ end\n;
String drl2 = global java.util.List list\n + rule R2 when\n
+ $s : String() + then\n + list.add( \b\ + $s );
+ end\n;
ReleaseId releaseId1 = KieServices.Factory.get().newReleaseId(
it.mine, test, 1.1);
ReleaseId releaseId2 = KieServices.Factory.get().newReleaseId(
it.mine, test, 1.2);
KieServices ks = KieServices.Factory.get();
KieModuleModel kproj = ks.newKieModuleModel();
KieBaseModel kieBaseModel1 = kproj.newKieBaseModel(KBase1)
.addPackage(it.mine.test);
kieBaseModel1.newKieSessionModel(KSession1).setDefault(true);
KieFileSystem kfs = ks.newKieFileSystem();
KieBuilder kieBuilder = ks.newKieBuilder(kfs);
kfs.generateAndWritePomXML(releaseId1);
kfs.write(src/main/resources/it/mine/test/rule.drl, drl1);
// FIXME: without this line all goes well.
kfs.writeKModuleXML(kproj.toXML());
kieBuilder.buildAll();
assertEquals(0, kieBuilder.getResults().getMessages().size());
KieModule kieModule = kieBuilder.getKieModule();
assertEquals(releaseId1, kieModule.getReleaseId());
KieContainer kc = ks.newKieContainer(releaseId1);
KieSession ksession = kc.newKieSession();
ListString list = new ArrayListString();
ksession.setGlobal(list, list);
ksession.insert(Foo);
ksession.fireAllRules();
list.clear();
kfs.generateAndWritePomXML(releaseId2);
kfs.write(src/main/resources/it/mine/test2/rule.drl, drl2);
IncrementalResults results = ((InternalKieBuilder) kieBuilder)
.incrementalBuild();
assertEquals(0, results.getAddedMessages().size());
kieModule = kieBuilder.getKieModule();
assertEquals(releaseId2, kieModule.getReleaseId());
Results updateResults = kc.updateToVersion(releaseId2);
assertEquals(0, updateResults.getMessages().size());
}
#
Heres the stack trace:
#
java.lang.NullPointerException
at
org.drools.compiler.kie.util.ChangeSetBuilder.build(ChangeSetBuilder.java:51)
at
org.drools.compiler.kie.builder.impl.KieContainerImpl.update(KieContainerImpl.java:126)
at
org.drools.compiler.kie.builder.impl.KieContainerImpl.updateToVersion(KieContainerImpl.java:105)
at
it.mine.IncrementalCompilationTest.shouldRecompileWithNewBases(IncrementalCompilationTest.java:165)
#
Thanks for the help!
Cheers,
Federico.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
