Re: [rules-users] Slow compilation (4h) for a single rule
Thanks for your explanation Edson, it makes sense now I think! Basically what you are saying is that there is a limit, beyond which is no longer feasible to use more ORs on a rule. Edson Tirelli-4 wrote You are trying to avoid the issue. As reported by others, your conditions should be inside the patterns, not in or'd evals in the end, and yes, it is possible to generate rules like that in an automated program. Sadly I don't see how, as some of my rules have arbitrary arithmetic involving more than one variable inside the Number(), for more context see the example I've put on http://drools.46999.n3.nabble.com/drools-arithmetics-without-eval-td3823232.html. For this case I think I will try to invert the logic and get rid of the ORs: not(A or B) = not(A) and not(B). Best regards, TL -- View this message in context: http://drools.46999.n3.nabble.com/Slow-compilation-4h-for-a-single-rule-tp4018855p4018894.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Slow compilation (4h) for a single rule
Thank you all for your feedback, I will try to implement at least the part of collapsing the multiple evals() into a single one. Edson Tirelli-4 wrote Also, I just read the example in your link where you have multiple accumulates in a single rule... that is *bad* The accumulate() over-usage is a known problem for me, but so far the best solution for my case was proposed by Vincent Legendre on this http://drools.46999.n3.nabble.com/drools-arithmetics-without-eval-tp3823232p3825280.html post . As a comic note, the previous rule with the 26 ORs, had 26 accumulates on it :) Best regards, TL -- View this message in context: http://drools.46999.n3.nabble.com/Slow-compilation-4h-for-a-single-rule-tp4018855p4018900.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] Slow compilation (4h) for a single rule
Hi, Recently I was trying to optimize the rule compilation time for my KB (~4K rules). I do a full KB compilation every night, as most of the rules are dynamic generated/converted and managed by the business operators. The compile time was not very good (around 3 hours on a 8 core Xeon CPU), but is was acceptable. On the other night I was alarmed that the compile time was doubled overnight without an obvious explanation or any relevant rule count increase. After some debuging I've traced the problem and was shocked to find that most of the rules KB (4K rules) compile under 5 minutes, except one. This single rule was taking all the 4h to compile! I'm using DROOLS 5.2 Final. The rule example below samples the problem (don't mind the nonsense Number() conditions): rule CONFIG_1-3UCO07 salience -90 when Number(rel_4226982244: intValue) Number(rel_7521194: intValue) Number(rel_787633980: intValue) Number(qty_1331544548: intValue) Number(rel_1425187049: intValue) Number(rel_1180441096: intValue) Number(rel_3132221704: intValue) Number(rel_1663554156: intValue) Number(rel_1940612775: intValue) Number(rel_1735126416: intValue) Number(rel_3962361266: intValue) Number(rel_882187: intValue) Number(rel_1169008280: intValue) Number(rel_3495503197: intValue) Number(rel_70290066: intValue) Number(rel_1333860961: intValue) Number(rel_2793542368: intValue) Number(rel_952404632: intValue) Number(rel_2712335119: intValue) Number(qty_4276673135: intValue) Number(qty_3950051097: intValue) Number(rel_3391032645: intValue) Number(rel_2738029181: intValue) Number(qty_4125201080: intValue) Number(rel_663254919: intValue) Number(rel_3059142355: intValue) not((eval(qty_1331544548 == 1) or eval(rel_1425187049 == 1) or eval(rel_1180441096 == 1) or eval(rel_787633980 = 1) or eval(rel_7521194 = 1) or eval(rel_4226982244 = 1) or eval(rel_2712335119 == 1) or eval(qty_3950051097 == 1) or eval(qty_4276673135 == 1) or eval(rel_1940612775 == 1) or eval(rel_3132221704 == 1) or eval(rel_882187 == 1) or eval(rel_1663554156 == 1) or eval(rel_1735126416 == 1) or eval(rel_3962361266 == 1) or eval(rel_1169008280 == 1) or eval(rel_2793542368 == 1) or eval(rel_3495503197 == 1) or eval(qty_4125201080 == 1) or eval(rel_70290066 == 1) or eval(rel_3391032645 == 1) or eval(rel_1333860961 == 1) or eval(rel_2738029181 == 1) or eval(rel_663254919 == 1) or eval(rel_952404632 == 1) or eval(rel_3059142355 == 1))) then vh.error(kcontext, Error ...); end I've noticed that the last not(...) is the problematic condition. Is there a limit using eval() and or-clauses, or a known problem? Regards, Tiago Lopes -- View this message in context: http://drools.46999.n3.nabble.com/Slow-compilation-4h-for-a-single-rule-tp4018855.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Slow compilation (4h) for a single rule
laune wrote Well, I do mind the nonsense Number(). There's no point in discussing this if you don't post an exact image of your rule. -W There is a point, because the rule I've posted compiles and causes the same problem, I've just removed redundant code to expose the problematic section. TL -- View this message in context: http://drools.46999.n3.nabble.com/Slow-compilation-4h-for-a-single-rule-tp4018855p4018857.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Slow compilation (4h) for a single rule
I understand that the rule looks pointless and under-optimized, but it's the only way I could represent this kind of rule in an automated way (I didn't write the rule by hand). Basically with these rules I try to pre-compute the amount of some products on the WM (using count() inside a accumulate function) resulting in a Number(), and then on the last condition I evaluate some arbitrary arithmetic operation using the variables inside the Number() object. For more details for examples see my previous post where I've tried to address this issue: http://drools.46999.n3.nabble.com/drools-arithmetics-without-eval-td3823232.html So, considerations apart of the way the rule is written, I still think there is a bug or something similar on the RETE compiler, that causes this rule to be processed in a very expensive way. Can someone confirm this, and try to sugest an alternative? Maybe It's something that is already fixed on newer versions as sugested by Esteban? Regards, Tiago Lopes -- View this message in context: http://drools.46999.n3.nabble.com/Slow-compilation-4h-for-a-single-rule-tp4018855p4018870.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] drools arithmetics without eval()
Hi, These rules are all auto-generated every day, and I don't control what is going into them, so this kind of optimizations won't work for me. The basic problem here, is that I have tons of rules that count facts (PortfolioProducts) and use the result as rule conditions. Some conditions could look like this: count(PortfolioProduct(A)) + count(PortfolioProduct(B)) = ( count(PortfolioProduct(C)) - count(PortfolioProduct(D)) ) - count(PortfolioProduct(E)) and so on... All rules look like the rule I've posted: I first calculate all relevant fact counts to be used by the rule (accumulates), and then perform the logic/arithmetic evaluation using evals(). Some rules end up having 15 accumulates and 2 evals as conditions... My question is if using accumulate and eval() is the only choice I have to write these kind of rules. Regards, Tiago Lopes -- View this message in context: http://drools.46999.n3.nabble.com/drools-arithmetics-without-eval-tp3823232p3824958.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] drools arithmetics without eval()
I have a question regarding eval() use. My rulebase is around ~3k rules, most of them are auto-generated by templates, and they end up looking like this: rule CONFIG_114 salience 0 when client: Client() contextProd: PortfolioProduct(prodAdded == true, productId == PROFESSIONAL) Number(qty_1: intValue) from accumulate(pp: PortfolioProduct(productId in (BOX_001), prodAdded == true), count(pp)) Number(qty_2: intValue) from accumulate(pp: PortfolioProduct(productId in (BOX_002), prodAdded == true), count(pp)) Number(qty_3: intValue) from accumulate(pp: PortfolioProduct(productId in (BOX_003), prodAdded == true), count(pp)) Number(qty_4: intValue) from accumulate(pp: PortfolioProduct(productId in (BOX_004), prodAdded == true), count(pp)) Number(qty_5: intValue) from accumulate(pp: PortfolioProduct(productId in (BOX_005), prodAdded == true), count(pp)) eval(qty_1 == 1) not(eval(qty_2 = (qty_3 + qty_4 + qty_5))) then balance(kcontext, contextProd, qty_2, =, (qty_3 + qty_4 + qty_5)); end These rules have all one or two evals in the end, comparing product quantities that are present in the Working Memory. My question is: Is there any better way to write this kind of rules without resorting to eval()? I'm currently getting hit (in performance terms) by over-using these, so I wonder if there is a better way to do this. Thanks! Tiago Lopes -- View this message in context: http://drools.46999.n3.nabble.com/drools-arithmetics-without-eval-tp3823232p3823232.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Drools Execution Server memory leak?
Hi, We are having the same problem with a similar setup (drools 5.2.0 Final) using stateless sessions and camel integration, using a KB with ~3000 rules. Session memory is never reclaimed, and after some hours we got an out of memory error. We still dont know the cause, but the profiler shows that these objects are increasing rapidly: org.drools.reteoo.BetaMemory 176.478 +176.163 (+55925 %) 6.893 kB org.drools.core.util.LeftTupleList 175.467 +175.156 (+56320 %) 6.854 kB org.drools.core.util.RightTupleList 172.806 +172.536 (+63902 %) 6.750 kB org.drools.rule.ContextEntry[ ] 164.784 +164.724 (+274540 %)2.659 kB org.drools.core.util.Entry[ ] 160.520 +160.449 (+225985 %)16.561 kB org.drools.core.util.ObjectHashMap 158.309 +158.253 (+282595 %) 6.183 kB org.drools.reteoo.AccumulateNode$AccumulateMemory154.505+154.503 (+7725150 %)4.828 kB org.mvel2.integration.impl.IndexVariableResolver 76.262+76.234 (+272264 %) 1.787 kB We used drools 5.1.1 before with no problems, so we think this leak is upgrade related. Regards, TL -- View this message in context: http://drools.46999.n3.nabble.com/rules-users-Drools-Execution-Server-memory-leak-tp3118864p3235954.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
