We are trying to apply Jess for complex event processing, but we have some
problems with its speed. I want to check with this mailing list our sample
program before we abandon the approach altogether. We also wonder if there
is any way to speed it up. In our experiments, we detected complex events in
streams of stock tickers or transactions of the form
stock(Company,Price,Volume) (see below the rules in Jess while the events
are represented as the facts pumped into the system with time stamps).
I can provide all of our code for anyone that needs it (beside what I
already pasted below).
We get some very bad times and we also run of of memory for large datasets
(10k stock ticks), while we can go up to millions of ticks and ~1% of the
execution time with CEP systems (Drools Fusion, Esper, Etalis, etc.). I know
that Jess might not be fit for this task, but we just wanted to try it since
we saw some works applying it (or other production rule systems) for complex
event processing.
Please tell me if you see or know any optimizations.
Thank you, regards,
Paul Fodor
test.clp:
; Complex events:
; stock("GOOG",Pr1,_)*stock("GOOG",Pr2,_)* (Pr1*0.8 > Pr2)) -> ce1.
; stock("MSFT",Pr1,_)*stock("MSFT",Pr2,_)* (Pr1*0.8 > Pr2)) -> ce2.
; ce1 \/ ce2 -> ce3.
; ce1 /\ ce2 -> ce4 {[T1,T2],T2-T1 <20}.
; ce1 # ce2 -> ce5.
; ce1*not(ce2)*ce1 -> ce6.
(deftemplate stockEvent "define stock event template" (slot symbol) (slot
price) (slot time1) (slot time2))
(deftemplate ce1 "Define complex event temp" (slot time1) (slot time2))
(deftemplate ce2 "Define complex event temp" (slot time1) (slot time2))
(deftemplate ce3 "Define complex event temp" (slot time1) (slot time2))
(deftemplate tmp1ce3 "Define complex event temp" (slot time1) (slot time2))
(deftemplate tmp2ce3 "Define complex event temp" (slot time1) (slot time2))
(deftemplate ce4 "Define complex event temp" (slot time1) (slot time2))
(deftemplate tmp1ce4 "Define complex event temp" (slot time1) (slot time2))
(deftemplate tmp2ce4 "Define complex event temp" (slot time1) (slot time2))
(deftemplate ce5 "Define complex event temp" (slot time1) (slot time2))
(deftemplate tmp1ce5 "Define complex event temp" (slot time1) (slot time2))
(deftemplate tmp2ce5 "Define complex event temp" (slot time1) (slot time2))
(deftemplate ce6 "Define complex event temp" (slot time1) (slot time2))
(deftemplate tmp1ce6 "Define complex event temp" (slot time1) (slot time2))
(deftemplate tmp2ce6 "Define complex event temp" (slot time1) (slot time2))
(defrule rule_ce1 "Complex Event 1"
?stockEvent1 <- (stockEvent {symbol == GOOG}(price ?p1))
?stockEvent2 <- (stockEvent {symbol == GOOG && time1 > stockEvent1.time2 }
(price ?p2&:(< ?p1 (* ?p2 1.2))))
=> (assert (ce1 (time1 ?stockEvent1.time1) (time2 ?stockEvent2.time2))) )
(defrule rule_ce2 "Complex Event 2"
?stockEvent1 <- (stockEvent {symbol == MSFT } (price ?p1))
?stockEvent2 <- (stockEvent {symbol == MSFT && time1 > stockEvent1.time2 }
(price ?p2&:(< ?p1 (* ?p2 1.2))))
=> (retract ?stockEvent1 ?stockEvent2)
(assert (ce2 (time1 ?stockEvent1.time1) (time2 ?stockEvent2.time2))) )
(defrule rule_ce1_multiply "Multiply Complex Event 1"
?temp <- (ce1 (time1 ?t1) (time2 ?t2))
=>
(assert (tmp1ce3 (time1 ?t1) (time2 ?t2)))
(assert (tmp1ce4 (time1 ?t1) (time2 ?t2)))
(assert (tmp1ce5 (time1 ?t1) (time2 ?t2)))
(assert (tmp1ce6 (time1 ?t1) (time2 ?t2)))
(retract ?temp) )
(defrule rule_ce2_multiply "Multiply Complex Event 2"
?temp <- (ce2 (time1 ?t1) (time2 ?t2))
=>
(assert (tmp2ce3 (time1 ?t1) (time2 ?t2)))
(assert (tmp2ce4 (time1 ?t1) (time2 ?t2)))
(assert (tmp2ce5 (time1 ?t1) (time2 ?t2)))
(assert (tmp2ce6 (time1 ?t1) (time2 ?t2)))
(retract ?temp) )
(defrule rule_ce3 "Complex Event ce3"
?temp <- (tmp1ce3)
=>
(assert (ce3 (time1 ?temp.time1) (time2 ?temp.time2)))
(retract ?temp) )
(defrule rule_ce3 "Complex Event ce3"
?temp <- (tmp2ce3)
=>
(assert (ce3 (time1 ?temp.time1) (time2 ?temp.time2)))
(retract ?temp) )
(defrule rule_ce41 "Complex Event 4"
?temp1 <- (tmp1ce4 (time2 ?ce1t2))
?temp2 <- (tmp2ce4 {time1 > temp1.time2} (time2 ?ce2t2&:(> 20 (- ?ce2t2
?ce1t2))))
=>
(assert (ce4 (time1 ?temp1.time1) (time2 ?temp2.time2)))
(retract ?temp1 ?temp2) )
(defrule rule_ce42 "Complex Event 4"
?temp1 <- (tmp2ce4 (time2 ?ce1t2))
?temp2 <- (tmp1ce4 {time1 > temp1.time2} (time2 ?ce2t2&:(> 20 (- ?ce2t2
?ce1t2))))
=>
(assert (ce4 (time1 ?temp1.time1) (time2 ?temp2.time2)))
(retract ?temp1 ?temp2) )
(defrule rule_ce51 "Complex Event 5 rule1"
?temp1 <- (tmp1ce5)
?temp2 <- (tmp2ce5 {time1 < temp1.time2 && time1 > temp1.time1 && time2 >
temp1.time2})
=>
(assert (ce5 (time1 ?temp1.time1) (time2 ?temp2.time2)))
(retract ?temp1 ?temp2) )
(defrule rule_ce52 "Complex Event 5 rule2"
?temp1 <- (tmp2ce5)
?temp2 <- (tmp1ce5 {time1 < temp1.time2 && time1 > temp1.time1 && time2 >
temp1.time2})
=>
(assert (ce5 (time1 ?temp1.time1) (time2 ?temp2.time2)))
(retract ?temp1 ?temp2) )
(defrule ce6 "Complex Event 6"
?temp1 <- (tmp1ce6 (time1 ?ce1t1) (time2 ?ce1t2) )
?temp2 <- (tmp1ce6 {time1 > temp1.time1 && time2 > temp1.time2}(time1
?ce2t1) (time2 ?ce2t2))
(not (tmp2ce6 (time1 ?t1&:(> ?t1 ?ce1t2)) (time2 ?t2&:(< ?t2 ?ce2t2)) ))
=>
(assert (ce6 (time1 ?ce1t1) (time2 ?ce2t2)))
(retract ?temp1 ?temp2 ) )
(defquery query-ce6 (ce6 (time1 ?time1) (time2 ?time2)))
(open "result.txt" output a)
(printout output crlf)
(reset)
(load-facts "tmp_data.clp")
(bind ?tmx (call java.lang.management.ManagementFactory getThreadMXBean))
(deffunction cputime () (return (* (?tmx getCurrentThreadCpuTime) 1E-9)))
(bind ?starttime_wall (time))
(bind ?starttime_cpu (cputime))
(run)
(bind ?query_result (run-query* query-ce6))
;(bind ?count 0)
;(while (?query_result next)
; (++ ?count)
;)
;(printout output "solutions: " ?count crlf)
(bind ?endtime_cpu (cputime))
(bind ?endtime_wall (time))
(bind ?walltime (- ?endtime_wall ?starttime_wall))
(bind ?cputime (- ?endtime_cpu ?starttime_cpu))
(printout output "computing cputime: " ?cputime crlf)
(printout output "computing walltime: " ?walltime crlf)
(close output)
tmp_data.clp:
(stockEvent (symbol GOOG) (price 580.0) (time1 1) (time2 1))
(stockEvent (symbol MSFT) (price 247.1) (time1 2) (time2 2))
(stockEvent (symbol MSFT) (price 439.9) (time1 3) (time2 3))
(stockEvent (symbol GOOG) (price 149.0) (time1 4) (time2 4))
(stockEvent (symbol GOOG) (price 313.1) (time1 5) (time2 5))
...
cp data/benchFacts_001k.clp tmp_data.clp
java -Xms1000m -Xmx1000m -cp ";./;.\;.;c:\Program
Files\Java\jdk1.6.0_12\lib\tools.jar;C:\Program
Files\Java\jre6\lib\ext\QTJava.zip;C:\Program
Files\pl\bin;lib\antlr-runtime-3.1.1.jar;lib\jess.jar;lib\jsr94.jar"
jess.Main test.clp
