Re: [rules-users] java dialect and declared types
I've tried to reproduce this in a tester but all my unit tests show 6.0.0 marginally quicker. The rules and data are simplified for the test so I'll try something closer to what i am seeing on the full system. I've just done another comparison and the results are shown in the attached image. http://drools.46999.n3.nabble.com/file/n4027884/Screenshot_2.png There are a number of spikes in the 6.0.1 run that don't help, but it is clear that the 5.5.0 execution is better. I've run the tests several times and all show the same thing. 1st col is average time - 250 jobs to process 2nd col is total time for test to complete - 10 processes executing - 48 core machine 4gb per process - mvel dialect For 5.5.0 5854215 5707193 5629189 5512183 5607189 5395183 5369183 For 6.0.0 6229244 5828219 6076223 5767220 5735220 5821214 -- View this message in context: http://drools.46999.n3.nabble.com/java-dialect-and-declared-types-tp4027822p4027884.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] java dialect and declared types
Mark Proctor wrote Also there will be no performance games for rule bases that do not have joins Can you elaborate on what a join is? -- View this message in context: http://drools.46999.n3.nabble.com/java-dialect-and-declared-types-tp4027822p4027871.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] java dialect and declared types
If I declare a type in a drl and reference the attributes in a rule then all is fine for either dialect. But, if one of the attributes is a java class that has been compiled and inserted then under the java dialect only I get a compilation error: rule compilation error the field ... is not visible for example declare DroolsTransaction ORG : String PRODUCT : String raw : Transaction end rule create classes salience 100 when $t : Transaction() then insert(new DroolsTransaction((String)$t.fieldFor(ORG), (String)$t.fieldFor(PRODUCT), $t)); end I insert Transaction objects in the session and this works fine for the following rule if I have the dialect set to mvel. rule 1 when $t : DroolsTransaction(ORG == A , PRODUCT == 001, raw.complicated()) then do something end but when I switch dialects to java, the compilation complains that raw is not visible. I guess drools is dynamically creating the DroolsTransaction object with private fields... and I need public accessors - how to define them? -- View this message in context: http://drools.46999.n3.nabble.com/java-dialect-and-declared-types-tp4027822.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] java dialect and declared types
will do, although complicated() has parameters and so I think I'll only able to try getRaw.complicated() -- View this message in context: http://drools.46999.n3.nabble.com/java-dialect-and-declared-types-tp4027822p4027825.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] java dialect and declared types
I should qualify this with the way that we are using drools. In general perhaps 6 is quicker that 5.5. It will be extremely difficult to create a self contained test to demonstrate this as the tests are performed with tens of millions of transactions against ten of thousands of rules. I will however attempt to scale this down to something that can demonstrate these findings. This wont happen straight away so pin this thread and hopefully in a week or so I will be able to submit something. -- View this message in context: http://drools.46999.n3.nabble.com/java-dialect-and-declared-types-tp4027822p4027834.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] named consequences for an or...
laune wrote On 20/01/2014, pmander lt; paul.s.mander@ gt; wrote: I could do this: when $t : Transaction((org == us) || (expensive() == 111)) then insert(new Record($t, 1)); end Yes, that takes care of the unwanted duplication. If org is indeed equal to us is it guaranteed that expensive() doesn't get executed? You can test this quite easily: add a print to expensive(). -W I quick related question regarding performance. I now have rule 3 when $t : Transaction( ( ( fieldFor(SALES_ORG)==SO10167 fieldFor(CHANNEL)==A fieldFor(ONLINE_IND)==N colMatch(rulesMap,3,2000,baseData,PRODUCT,0) ) || ( fieldFor(CHANNEL)==A fieldFor(ONLINE_IND)==N fieldFor(ACCOUNT) in(A11329,A11330,A11331,A11332,A11333,A11334,A11335,A11336) colMatch(rulesMap,3,2000,baseData,PRODUCT,0) ) ) ) then do something end instead of rule 3 when $t : ( Transaction( fieldFor(CHANNEL)==A, fieldFor(ONLINE_IND)==N, fieldFor(SALES_ORG)==SO10167, colMatch(rulesMap,3,1370,baseData,PRODUCT,0)) or Transaction( fieldFor(CHANNEL)==A, fieldFor(ONLINE_IND)==N, fieldFor(ACCOUNT) in(A11329,A11330,A11331,A11332,A11333,A11334,A11335,A11336), colMatch(rulesMap,3,2000,baseData,PRODUCT,0) ) then do something end I would have thought that the first was slight better performance wise as the second produces twice as many matches which I need to filter out but putting 20,000 Transactions through this (2400 rules) takes 51 seconds for option 2 and 200 seconds for option 1. If I just have the first half of the rule: $t : ( Transaction( fieldFor(CHANNEL)==A, fieldFor(ONLINE_IND)==N, fieldFor(SALES_ORG)==SO10167, colMatch(rulesMap,3,1370,baseData,PRODUCT,0)) then do something end then the performance is 10 seconds. The extra time being taken up with all those other or mis-matches. -- View this message in context: http://drools.46999.n3.nabble.com/named-consequences-for-an-or-tp4027775p4027792.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] named consequences for an or...
I could do this: when $t : Transaction((org == us) || (expensive() == 111)) then insert(new Record($t, 1)); end If org is indeed equal to us is it guaranteed that expensive() doesn't get executed? -- View this message in context: http://drools.46999.n3.nabble.com/named-consequences-for-an-or-tp4027775p402.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] permgen leak
Hi, You mention that the brand new kbase evertime could be a problem and you may get this from re-deployments. In my situation I am not in an app server and so do not have a re-deployment issue but I am creating a new kbase every time I run through the system. The system dynamically creates the drl from database definitions and this is done for each run of the application. Is there an alternative for this? -- View this message in context: http://drools.46999.n3.nabble.com/permgen-leak-tp4027038p4027321.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] drools 6 equivalent of addKnowledgePackages
I head previously read that but didn't spot the bit that mentions creating from a drl defined as a String. I take it this can be done from KieFileSystem. String rules = ... KieFileSystem kfs = kieServices.newKieFileSystem(); kfs.write(kieServices.getResources().newReaderResource(new StringReader(rules))); This throws an exception complaining that the resource doesn't have a source or target path set. Is there a way to push a dynamically created drl into this without writing it to disk first? -- View this message in context: http://drools.46999.n3.nabble.com/drools-6-equivalent-of-addKnowledgePackages-tp4027044p4027059.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] permgen leak
permgen.zip http://drools.46999.n3.nabble.com/file/n4027065/permgen.zip Here's the test I was using. You can change the dialect, number of runs, batches, rules etc as attributes in the test class. -- View this message in context: http://drools.46999.n3.nabble.com/permgen-leak-tp4027038p4027065.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] permgen leak
Oh, and run the test with -Xmx2048m -XX:PermSize=512m -XX:MaxPermSize=512m or else it will fail straight away. Paul -- View this message in context: http://drools.46999.n3.nabble.com/permgen-leak-tp4027038p4027066.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] drools 6 equivalent of addKnowledgePackages
Just doing some investigation of the drools 6 release. We dynamically build our rules and use some thing like the following to bring them into a stateful session: KnowledgeBuilder builder = KnowledgeBuilderFactory.newKnowledgeBuilder(); builder.add(ResourceFactory.newReaderResource(new StringReader( compiledRules)), ResourceType.DRL); KnowledgeBuilderErrors errors = builder.getErrors(); kbase.addKnowledgePackages(builder.getKnowledgePackages()); This appears to be deprecated but there is no indication in the javadoc for what the alternative is. -- View this message in context: http://drools.46999.n3.nabble.com/drools-6-equivalent-of-addKnowledgePackages-tp4027044.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Drools Perm gen grows constantly
I appear to have resolved this issue by changing dialect to MVEL and setting drools.permgenThreshold=0 in META-INF/drools.rulebase.conf. I only stumbled upon this based on some hints in another post and when it didn't work I had to debug the point at which the classes got created and worked back that the permgen threshold was only used by mvel dialect rules. Appalling documentation from a performance and system configuration perspective. -- View this message in context: http://drools.46999.n3.nabble.com/Drools-Perm-gen-grows-constantly-tp4026673p4026806.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Drools Perm gen grows constantly
Thanks for clearing this up. The overhead for us is negligible and more important that the application is survivable. Using Java as the dialect would appear to not be a option for anyone then? We are using drools to process around 50Mil transactions against 15k rules. This is distributed over 32 nodes to get the end to end performance we need. Using java dialect on initial run with a mac perm size of 512Mb copes ok but on subsequent runs we run out of perm gen space and the processes take up tons of cpu and then run out of memory. For each run we have to recreate the knowledge base. -- View this message in context: http://drools.46999.n3.nabble.com/Drools-Perm-gen-grows-constantly-tp4026673p4026812.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Drools Perm gen grows constantly
I'll try and get a unit test together to show this. Have a deadline tomorrow though so will take a few days. -- View this message in context: http://drools.46999.n3.nabble.com/Drools-Perm-gen-grows-constantly-tp4026673p4026816.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] Drools Perm gen grows constantly
Fairly new to drools here but we are using it on a standalone application to do some batch processing. The application is spring based and single-threaded. I am using a single stateful session that I build around 15,000 rules dynamically on process startup. The application then receives a message to process a set of records through the rules and write out the results. To do this I insert the data as facts and run a query to get the results. I must then remove all the facts that I created and also all the object that where created by the rules so that the session can be used for the next batch of records. This works fine and meets our performance requirements. However, after a time the process fails and upon investigation this is due to the perm gen space filling up with classes. The heap itself is fine and steady by the number of classes in the JVM keeps growing. These classes are all DefaultConsequenceInvokerGenerated classes appear to be created when my new facts are inserted. This is running on a 1.7 oracle JVM and I've tried -XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled but nothing helps. So is this expected behaviour? Is there an issue with how I am using drools? Any help much appreciated. -- View this message in context: http://drools.46999.n3.nabble.com/Drools-Perm-gen-grows-constantly-tp4026673.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
