WG: JESS: Poor performance by loading large data

Negoita, Cristian Wed, 11 May 2005 08:40:06 -0700

Title: AW: JESS: Poor performance by loading large data

I realized that the format of email is gone, so I attach the rules in DOS format. For someone still willing to help, is easier to read.

Best regards,

Cristian

-----Urspr�ngliche Nachricht-----
Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Im Auftrag von Negoita, Cristian
Gesendet: Mittwoch, 11. Mai 2005 14:35
An: [email protected]
Betreff: AW: JESS: Poor performance by loading large data

Hi all,

First, thanks for the answers. As you well assumed, yes, I have rules for facts A and B, and some more objects.

So, I see two improvements points:

1) Load the facts BEFORE the rules - this way the partial matches times will be removed (loading without rules is indeed very fast)

2) Improve the rules itself, so when it will be loaded on already existing facts, will take as less as possible.

Below are the rules, with explanations on amount of data it should process; I believe rule2 is the most consuming one and is not a trivial one...

So,

- Request fact is Fact A from previous mail (about 50.000)

- CustomerHit is Fact B (500.000, 10 for each Request)

- CompanyLock only 500 facts

- Lock, the biggest one, has 2.500.000 facts, 5 Locks per each CustomerHit)

- All above are defined in Java objects

- "isStillValid" is a user function written in java

(defrule rule2

(declare (salience 10))

(Request

(requestID ?req_requestid)

)

(CustomerHit

(requestID ?hit_requestid&:(= ?req_requestid ?hit_requestid))

(hitID ?hit_hitid)

(companyID ?hit_companyid)

)

(CompanyLock

(companyID ?companylock_companyid&:(= ?companylock_companyid ?hit_companyid))

(type ?companylock_type)

(validityDays ?companylock_validitydays)

)

(Lock

(type ?lock_type&:(= ?companylock_type ?lock_type))

(hitID ?lock_hitid&:(= ?lock_hitid ?hit_hitid))

(lockID ?lock_lockid)

(creationDate ?lock_creationDate)

)

(test (isStillValid ?lock_creationDate ?companylock_validitydays))

=> (assert (BadHit (requestid ?hit_requestid)(companyid ?hit_companyid)))

)

(defquery countNoOfBadLocksPerRequest

(declare (variables ?requestid))

(BadHit (requestid ?requestid) (companyid ?X))

)

(defrule rule3

(declare (salience 5))

(Request

(requestID ?req_requestid)

)

(BadHit)

(test (>= (count-query-results countNoOfBadLocksPerRequest ?req_requestid) 2))

=> (printout t "Bad RequestID found = " ?req_requestid crlf)

)

Thanks for you support,

Cristian

> -----Urspr�ngliche Nachricht-----

> Von: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] Im

> Auftrag von [EMAIL PROTECTED]

> Gesendet: Mittwoch, 11. Mai 2005 13:43

> An: [email protected]

> Betreff: Re: JESS: Poor performance by loading large data

> I think Negoita, Cristian wrote:

> > Hi all,

> >

> > I am using Rete engine to load up large amount of facts in the working

> > area. (let's say 50.000 facts of type A and 500.000 facts of type B)

> >

> > The problem is that: if I load FIRST facts A, these are loaded fast, but

> > AFTER that, facts B are loaded very slowly.

> Remember that pattern-matching -- i.e., rule evaluation -- happens

> while facts are being asserted, modified, or retracted. The time it

> takes to assert a fact depends entirely on how much work Jess has to

> do to evaluate the rules that potentially match that fact.

> Based on what you've said, you've got one or more rules that match

> both an A and a B fact, with little restrictions, such that you're

> getting 50,000 times 500,000 or 25000000000 partial matches, which

> will consume plenty of both space and time. Some simple changes to the

> way in which you express your rules could reduce this by many orders

> of magnitude, drastically speeding things up.

> So we'll have to see your rules. Show us the ones that match both an A

> and a B fact.

> ---------------------------------------------------------

> Ernest Friedman-Hill

> Advanced Software Research Phone: (925) 294-2154

> Sandia National Labs FAX: (925) 294-2234

> PO Box 969, MS 9012 [EMAIL PROTECTED]

> Livermore, CA 94550 http://herzberg.ca.sandia.gov

> --------------------------------------------------------------------

> To unsubscribe, send the words 'unsubscribe jess-users [EMAIL PROTECTED]'

> in the BODY of a message to [EMAIL PROTECTED], NOT to the list

> (use your own address!) List problems? Notify [EMAIL PROTECTED]

> --------------------------------------------------------------------

    (defrule rule2
            (declare (salience 10))
            (Request
                (requestID ?req_requestid)
            )
            (CustomerHit
                (requestID ?hit_requestid&:(= ?req_requestid ?hit_requestid))
                (hitID ?hit_hitid)
                (companyID ?hit_companyid)
            )
            (CompanyLock
                (companyID ?companylock_companyid&:(= ?companylock_companyid 
?hit_companyid))
                (type ?companylock_type)
                (validityDays ?companylock_validitydays)
            )
            (Lock
                (type ?lock_type&:(= ?companylock_type ?lock_type))
                (hitID ?lock_hitid&:(= ?lock_hitid ?hit_hitid))
                (lockID ?lock_lockid)
                (creationDate ?lock_creationDate)
            )
            (test (isStillValid ?lock_creationDate ?companylock_validitydays))
        => 
           (assert (BadHit (requestid ?hit_requestid)(companyid 
?hit_companyid)))
        )


        (defquery countNoOfBadLocksPerRequest
                (declare (variables ?requestid))
                (BadHit (requestid ?requestid) (companyid ?X))
        )

        (defrule rule3
                (declare (salience 5))
                (Request
                  (requestID ?req_requestid)
                )
                (BadHit)
                (test (>= (count-query-results countNoOfBadLocksPerRequest 
?req_requestid) 2))
                => 
                (printout t "Bad RequestID found = " ?req_requestid crlf)
        )

WG: JESS: Poor performance by loading large data

Reply via email to