If I understand the rules correctly. the example is using a fact to define intervals. Doing it that way incurs a huge cost, since the engine has to retract the fact at the end of the time window.
It would be more efficient to write a time function(s) and use them in the LHS. That avoids hammering the engine, but doesn't completely solve the issue. Rule_ce2, it retracts the stockEvent. Rule_ce1 doesn't retract the stockEvent, so over time the engine is going to run out of memory. Taking a step back, the bigger issue is temporal logic as it relates to complex rulesets. In the example, there's only 2 rules that perform a simple pattern match. One can't assume it's safe to retract the stockEvent fact from the engine if some other rules use it. without looking at the bigger picture and having a clear understanding of temporal logic, any attempt to use a rule engine for event processing is going to filled with potholes. I would suggest taking time to study temporal logic and pattern matching theory more thoroughly first. peter On Wed, Sep 30, 2009 at 3:22 PM, PF <[email protected]> wrote: > We are trying to apply Jess for complex event processing, but we have some > problems with its speed. I want to check with this mailing list our sample > program before we abandon the approach altogether. We also wonder if there > is any way to speed it up. In our experiments, we detected complex events in > streams of stock tickers or transactions of the form > stock(Company,Price,Volume) (see below the rules in Jess while the events > are represented as the facts pumped into the system with time stamps). > > I can provide all of our code for anyone that needs it (beside what I > already pasted below). > We get some very bad times and we also run of of memory for large datasets > (10k stock ticks), while we can go up to millions of ticks and ~1% of the > execution time with CEP systems (Drools Fusion, Esper, Etalis, etc.). I know > that Jess might not be fit for this task, but we just wanted to try it since > we saw some works applying it (or other production rule systems) for complex > event processing. > Please tell me if you see or know any optimizations. > > Thank you, regards, > Paul Fodor > > test.clp: > ; Complex events: > ; stock("GOOG",Pr1,_)*stock("GOOG",Pr2,_)* (Pr1*0.8 > Pr2)) -> ce1. > ; stock("MSFT",Pr1,_)*stock("MSFT",Pr2,_)* (Pr1*0.8 > Pr2)) -> ce2. > ; ce1 \/ ce2 -> ce3. > ; ce1 /\ ce2 -> ce4 {[T1,T2],T2-T1 <20}. > ; ce1 # ce2 -> ce5. > ; ce1*not(ce2)*ce1 -> ce6. > > (deftemplate stockEvent "define stock event template" (slot symbol) (slot > price) (slot time1) (slot time2)) > (deftemplate ce1 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate ce2 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate ce3 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate tmp1ce3 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate tmp2ce3 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate ce4 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate tmp1ce4 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate tmp2ce4 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate ce5 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate tmp1ce5 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate tmp2ce5 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate ce6 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate tmp1ce6 "Define complex event temp" (slot time1) (slot time2)) > (deftemplate tmp2ce6 "Define complex event temp" (slot time1) (slot time2)) > > (defrule rule_ce1 "Complex Event 1" > ?stockEvent1 <- (stockEvent {symbol == GOOG}(price ?p1)) > ?stockEvent2 <- (stockEvent {symbol == GOOG && time1 > stockEvent1.time2 } > (price ?p2&:(< ?p1 (* ?p2 1.2)))) > => (assert (ce1 (time1 ?stockEvent1.time1) (time2 ?stockEvent2.time2))) ) > > (defrule rule_ce2 "Complex Event 2" > ?stockEvent1 <- (stockEvent {symbol == MSFT } (price ?p1)) > ?stockEvent2 <- (stockEvent {symbol == MSFT && time1 > stockEvent1.time2 } > (price ?p2&:(< ?p1 (* ?p2 1.2)))) > => (retract ?stockEvent1 ?stockEvent2) > (assert (ce2 (time1 ?stockEvent1.time1) (time2 ?stockEvent2.time2))) ) > > (defrule rule_ce1_multiply "Multiply Complex Event 1" > ?temp <- (ce1 (time1 ?t1) (time2 ?t2)) > => > (assert (tmp1ce3 (time1 ?t1) (time2 ?t2))) > (assert (tmp1ce4 (time1 ?t1) (time2 ?t2))) > (assert (tmp1ce5 (time1 ?t1) (time2 ?t2))) > (assert (tmp1ce6 (time1 ?t1) (time2 ?t2))) > (retract ?temp) ) > > (defrule rule_ce2_multiply "Multiply Complex Event 2" > ?temp <- (ce2 (time1 ?t1) (time2 ?t2)) > => > (assert (tmp2ce3 (time1 ?t1) (time2 ?t2))) > (assert (tmp2ce4 (time1 ?t1) (time2 ?t2))) > (assert (tmp2ce5 (time1 ?t1) (time2 ?t2))) > (assert (tmp2ce6 (time1 ?t1) (time2 ?t2))) > (retract ?temp) ) > > (defrule rule_ce3 "Complex Event ce3" > ?temp <- (tmp1ce3) > => > (assert (ce3 (time1 ?temp.time1) (time2 ?temp.time2))) > (retract ?temp) ) > > (defrule rule_ce3 "Complex Event ce3" > ?temp <- (tmp2ce3) > => > (assert (ce3 (time1 ?temp.time1) (time2 ?temp.time2))) > (retract ?temp) ) > > (defrule rule_ce41 "Complex Event 4" > ?temp1 <- (tmp1ce4 (time2 ?ce1t2)) > ?temp2 <- (tmp2ce4 {time1 > temp1.time2} (time2 ?ce2t2&:(> 20 (- ?ce2t2 > ?ce1t2)))) > => > (assert (ce4 (time1 ?temp1.time1) (time2 ?temp2.time2))) > (retract ?temp1 ?temp2) ) > > (defrule rule_ce42 "Complex Event 4" > ?temp1 <- (tmp2ce4 (time2 ?ce1t2)) > ?temp2 <- (tmp1ce4 {time1 > temp1.time2} (time2 ?ce2t2&:(> 20 (- ?ce2t2 > ?ce1t2)))) > => > (assert (ce4 (time1 ?temp1.time1) (time2 ?temp2.time2))) > (retract ?temp1 ?temp2) ) > > (defrule rule_ce51 "Complex Event 5 rule1" > ?temp1 <- (tmp1ce5) > ?temp2 <- (tmp2ce5 {time1 < temp1.time2 && time1 > temp1.time1 && time2 > > temp1.time2}) > => > (assert (ce5 (time1 ?temp1.time1) (time2 ?temp2.time2))) > (retract ?temp1 ?temp2) ) > > (defrule rule_ce52 "Complex Event 5 rule2" > ?temp1 <- (tmp2ce5) > ?temp2 <- (tmp1ce5 {time1 < temp1.time2 && time1 > temp1.time1 && time2 > > temp1.time2}) > => > (assert (ce5 (time1 ?temp1.time1) (time2 ?temp2.time2))) > (retract ?temp1 ?temp2) ) > > (defrule ce6 "Complex Event 6" > ?temp1 <- (tmp1ce6 (time1 ?ce1t1) (time2 ?ce1t2) ) > ?temp2 <- (tmp1ce6 {time1 > temp1.time1 && time2 > temp1.time2}(time1 > ?ce2t1) (time2 ?ce2t2)) > (not (tmp2ce6 (time1 ?t1&:(> ?t1 ?ce1t2)) (time2 ?t2&:(< ?t2 ?ce2t2)) )) > => > (assert (ce6 (time1 ?ce1t1) (time2 ?ce2t2))) > (retract ?temp1 ?temp2 ) ) > > (defquery query-ce6 (ce6 (time1 ?time1) (time2 ?time2))) > > (open "result.txt" output a) > (printout output crlf) > (reset) > (load-facts "tmp_data.clp") > (bind ?tmx (call java.lang.management.ManagementFactory getThreadMXBean)) > (deffunction cputime () (return (* (?tmx getCurrentThreadCpuTime) 1E-9))) > (bind ?starttime_wall (time)) > (bind ?starttime_cpu (cputime)) > (run) > (bind ?query_result (run-query* query-ce6)) > ;(bind ?count 0) > ;(while (?query_result next) > ; (++ ?count) > ;) > ;(printout output "solutions: " ?count crlf) > (bind ?endtime_cpu (cputime)) > (bind ?endtime_wall (time)) > (bind ?walltime (- ?endtime_wall ?starttime_wall)) > (bind ?cputime (- ?endtime_cpu ?starttime_cpu)) > (printout output "computing cputime: " ?cputime crlf) > (printout output "computing walltime: " ?walltime crlf) > (close output) > > tmp_data.clp: > (stockEvent (symbol GOOG) (price 580.0) (time1 1) (time2 1)) > (stockEvent (symbol MSFT) (price 247.1) (time1 2) (time2 2)) > (stockEvent (symbol MSFT) (price 439.9) (time1 3) (time2 3)) > (stockEvent (symbol GOOG) (price 149.0) (time1 4) (time2 4)) > (stockEvent (symbol GOOG) (price 313.1) (time1 5) (time2 5)) > ... > > cp data/benchFacts_001k.clp tmp_data.clp > java -Xms1000m -Xmx1000m -cp ";./;.\;.;c:\Program > Files\Java\jdk1.6.0_12\lib\tools.jar;C:\Program > Files\Java\jre6\lib\ext\QTJava.zip;C:\Program > Files\pl\bin;lib\antlr-runtime-3.1.1.jar;lib\jess.jar;lib\jsr94.jar" > jess.Main test.clp > > > > > > -------------------------------------------------------------------- To unsubscribe, send the words 'unsubscribe jess-users [email protected]' in the BODY of a message to [email protected], NOT to the list (use your own address!) List problems? Notify [email protected]. --------------------------------------------------------------------
