Hi Folks, If you've been on this list for a while, you'll probably recall various discussions of the Manners benchmark. Once a user posted a table of various rule engine performance results on this benchmark. Jess was more or less average in that table.
One of the measurements, marked "JRULES-OPT", was strikingly faster than all the others, and I questioned that result. Apparently this result comes from the JRules people's own "tweaked" version of the benchmark code. Someone who understands the tweak has shared the ideas with me, and it turns out that the optimization works as well, or better, for Jess. On my machine, Jess does this "optimized" Manners benchmark at N=128 in 2-3 seconds, as compared to 38 seconds for the classic version; Jess can actually run the optimized program with N=256 (a previously unheard-of problem size!) in 22 seconds. The optimization consists of adding a single "not" pattern to a single rule. The outputs are identical, and in fact, if you use "watch rules" to watch everything that happens, you can see that the exact same sequence of rules fires, activated by the same facts, in the same order. So how does the optimization work? Well, what Manners really tests is worst-case agenda handling. One of the rules (find_seating) is activated and then immediately deactivated a large number of times every "cycle" (there are N cycles, one for each guest.) The number of activations actually grows on each cycle; by 50 cycles or so it's nearly 2000 activations being created and immediately destroyed, and it grows to many tens of thousands per cycle by N=128. The optimization adds a pattern to find_seating so that many of these activations are eliminated. Thus, the very thing that the problem is supposed to test is being skipped! This is therefore not an "optimization" at all -- it changes the problem into a different, far easier problem. This vindicates my original assertion that the JRULES-0PT measurement shouldn't count. It also should serve as a lesson to anybody who hasn't already figured this out: don't trust benchmarks! --------------------------------------------------------- Ernest Friedman-Hill Distributed Systems Research Phone: (925) 294-2154 Sandia National Labs FAX: (925) 294-2234 PO Box 969, MS 9012 [EMAIL PROTECTED] Livermore, CA 94550 http://herzberg.ca.sandia.gov -------------------------------------------------------------------- To unsubscribe, send the words 'unsubscribe jess-users [EMAIL PROTECTED]' in the BODY of a message to [EMAIL PROTECTED], NOT to the list (use your own address!) List problems? Notify [EMAIL PROTECTED] --------------------------------------------------------------------
