[rules-users] Package footprint
Hi all, I'm trying to reduce the memory footprint of packages in my application (for performance reasons they are dynamically loaded from disk), and I was wondering if some of you had any experience with the subject? So far the optimizations that I can come up with in 5.4.0.Final version are all about reducing the sizes of generated classes: - Package name in the drl file should be as small as possible, one letter is best (even better that no package at all) - Rule name should be as small as possible as it is used to generate the consequence class name - Write as less code as possible in the consequence - Use as few as possible eval statements - By using my own JavaDialectConfiguration and MVELDialectConfiguration when building the knowledge, I can remove the random number on class names that was added in the recent versions (don't understand why it was added in the first place? Does anyone know?) I see some leeway to shorten the names of the following generated classes but the amount of overriding to change the "default" consequence name and the suffixes is a bit too exotic for me: - .DefaultConsequenceInvoker - .DefaultConsequenceInvokerGenerated Any advice, other leads some of you might have? Thanks in advance -- View this message in context: http://drools.46999.n3.nabble.com/Package-footprint-tp4019721.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Guvnor Declarative Model
Hi guys, I just stumbled into the same issue in 5.4.0.Final, I think this is caused by https://issues.jboss.org/browse/JBRULES-2652 When declaring a type, a constructor with all fields as parameters is generated automatically, but this causes in our case to hit a barrier of the Java language because the generated constructor has more than 255 parameters Anyone knows if this generation can be skipped (annotation or something else), maybe a safeguard not to generate the constructor when the limit is reached would be nice too? -- View this message in context: http://drools.46999.n3.nabble.com/Guvnor-Declarative-Model-tp4019419p4019637.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Migrating from 4.0.7 to 5.4.0.Final
Hi again, I just tried the updated 5.4.x branch with 50% perm gen threshold and all my tests are now working as expected. As a side note, I've also ran a memory analysis of the tests and I can note that there are definitely less class loaders being created. I see that CompositeClassLoader still retain classes (hard to tell why though), but since I have the magical option to control consumption of PermGen, I consider that everything is okay for me Again many thanks for your help, hoping to see 5.4.1 out soon :-) -- View this message in context: http://drools.46999.n3.nabble.com/Migrating-from-4-0-7-to-5-4-0-Final-tp4018215p4018543.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Migrating from 4.0.7 to 5.4.0.Final
Mario Fusco wrote > > Yes, of course it will be possible, even if my suggestion is to avoid this > in production otherwise you will have ALL the constraints running in > interpreted mode. Actually you could already achieve this if you can > rebuild drools from the sources (you already did it, so I assume it won't > be a problem). If you want to try open the class MvelConstraint and modify > the constant JIT_THRESOLD. That constant defines after how many times a > constraint can be evaluated in interpreted mode before to get JITted, so > if you set it to a very high value (e.g. Integer.MAX_VALUE) you should be > able to completely avoid any JITting. Let me know if this works. > > I hope this helps, > Mario > I just tested it with JIT_THRESOLD=Integer.MAX_VALUE and it effectively does wonders for both perm gen consumption (as classes no longer held by the executor thread) and - believe it or not - even performances in some cases :-) It would be great to have this configurable (either this or your other option which will limit the amount of memory used by Jitting) -- View this message in context: http://drools.46999.n3.nabble.com/Migrating-from-4-0-7-to-5-4-0-Final-tp4018215p4018481.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Migrating from 4.0.7 to 5.4.0.Final
Hi Mario, Just tested it and unfortunately found the same problems, looking at your commit in (https://issues.jboss.org/browse/JBRULES-3567) there seem to be no real code changes in class MvelConstraint, the major leak being the ExecutorService holding on to the threads and thus all the generated classes. Have I missed something? Maybe I didn't check out the correct branch (https://github.com/droolsjbpm/drools/tree/5.4.x) ? Thanks for any input -- View this message in context: http://drools.46999.n3.nabble.com/Migrating-from-4-0-7-to-5-4-0-Final-tp4018215p4018463.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Migrating from 4.0.7 to 5.4.0.Final
Mario Fusco wrote > > I just fixed the leaks you reported on the master repository and > backported the fix to the 5.4.x branch. I also closed the JIRA ticket I > opened a few hours ago, but it is still possible to comment on it or even > reopen it if you will find further issues. I don't know if you have the > possibility to build Drools from the sources but if so it could be great > if you could give a try to this fix. > > Nevertheless it is still a fact that Drools 5.4 requires more PermGen > space, for the reasons that both Mark and myself explained in our former > emails. > > I hope this helps, > Mario > Thanks, that was quick! Yes I am aware if the PermGen consumption, but I'm hoping to tone it down by looking at the rules themselves (in 4.0.7 I had managed to reduce it by simply shortening the rule names, let's see) I'll try to access the drools sources and let you know if your patch fixes the problem, again thanks that was real quick -- View this message in context: http://drools.46999.n3.nabble.com/Migrating-from-4-0-7-to-5-4-0-Final-tp4018215p4018454.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Migrating from 4.0.7 to 5.4.0.Final
laune wrote
>
> It seems that the problems reported by the OP are mostly (if not all)
> related to building KnowledgePackages from DRL files. Drools has been
> through several 5.x versions with (among others) many extensions to
> the DRL language, so some increase in resource consumption is to be
> expected.
>
> Most production scenarios, however, use the recommended practice of
> loading compiled KnowledgePackages that were serialized in a separate
> runs.
>
> Does the reported increase in execution times ("2-3 times slower")
> relate to rule execution only, without compiling? If I understand the
> OPs statement correctly, this increase was observed after the patch
> reducing PermGen space.
>
> -W
>
Hi there,
Actually the tests are the same on both versions, and you are right they
include the compilation of the rule set. In production however, we use the
technique you mentioned.
However, difference in compilation times between both versions is acceptable
for us (about 30-40% slower in big sets), if I drill down a bit on the
execution part however I see that it can get to 6 times slower at times.
I've done comparative analysis with YourKit lately on both versions to get a
grasp of where the time was spent. I can see the JIT part with the
ClassGenerator come into place, by that's not what baffles me: I see a lot
more DefaultConsequenceInvoker.evaluate() calls on the stack in 5.4.0 than
4.0.7 and that's where most of the time is spent
Does that make any sense? Are consequence evaluated even if rules are not
fired ? Or is the way the rule is written that causes this (no-loop
statements, ...) ?
Any way I can deactivate the JIT behavior to have a more solid ground for
comparison?
More questions than answers I'm afraid, hoping to get to the bottom of this
:-)
--
View this message in context:
http://drools.46999.n3.nabble.com/Migrating-from-4-0-7-to-5-4-0-Final-tp4018215p4018453.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users
[rules-users] Migrating from 4.0.7 to 5.4.0.Final
Hi all, I'm currently in the process of migrating an application using drools 4.0.7 to 5.4.0.Final and I've stumbled into a few issues with the new version that I find disturbing: - Our unit tests failed because the JVM ran out of perm gen space, which looks odd as in 4.0.7 we've had quite a margin to start with. - Doing several Yourkit analysis, I finally find classloading leaks: * *org.drools.rule.builder.dialect.asm.ClassGenerator.EMPTY_METHOD_BODY* is a static instance which will hold to the last instance of InternalClassLoader created * *org.drools.rule.constraint.MvelConstraint* uses an ExecutorHolder that spawns daemon threads and keeps them in a pool. Problem is these threads create class loaders, and can only be garbage collected when the thread dies, meaning never in my case... - I've quick fixed both issues, the second one by simply disabling the thread pool. - Once this is done, the perm gen behaves correctly, but the execution times are 2-3 times slower on very large set of rules (~1000). Looking at Yourkit analysis again, I see that the number of classes generated and the perm gen consumption is about 3 times higher in 5.4.0.Final. I honestly don't know how to address this. I've tried to find some other posts concerning these issues, to no avail. I don't see any work around this, short of code changes, and for reducing the number of generated classes I simply have no idea. Please correct me if I am wrong, any help would be greatly appreciated -- View this message in context: http://drools.46999.n3.nabble.com/Migrating-from-4-0-7-to-5-4-0-Final-tp4018215.html Sent from the Drools: User forum mailing list archive at Nabble.com. ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
