Re: JESS: On the Performance of Logical Retractions
Thank you Ernest. I am experimenting with the Lehigh university benchmark, where i transfer OWL TBox into their equivalent rules in Jess, with the logical construct. Specifically, I am using the dataset and transformations, as used in the OpenRuleBench http://rulebench.projects.semwebcentral.org/. As for the runtimes, I missed a point about the retractions. The fact is, even if the session does not contain any rules (no defrules, just assertions), loading the same set of retractions takes a considerable time. This indicates that the high runtime is mostly incurred by jess internal operations. but still, when the number of changes grows high (say more than 10%) the runtime is not acceptable, and rerunning with the retracted kb would be faster. I have another question as well: what type of truth maintenance method is implemented in jess? Do you solely rely on the Rete memory nodes and tokens for this purpose? --Oli. On Mon, Jun 6, 2011 at 7:37 PM, Ernest Friedman-Hill ejfr...@sandia.govwrote: I don't think there's a particular reason in general. Retracting a fact takes only a little longer than asserting one, on average. But if we assume liberal use of logical, retracting a single fact could result in a sort of cascade effect whereby retracting a single fact would result in many other facts, and many activations, being removed also due to dependencies. All of that would take time. Still, your case seems extreme. Maybe there's something pathological about this particular case. On Jun 5, 2011, at 3:18 PM, Md Oliya wrote: Hi, I am doing some experiments with a set of rules which contain the logical CE. I intend to see the performance of Jess on a set of assertions as well as retractions. After some experiments, I found that the runtime for assertions is much less than that of retractions. In fact, the performance on retractions is so bad that I would rather re (run) jess on a retracted kb. A sample test case: The KB size, number of assertions, number of retractions, and number of rules are 100K, 50K, 1k, and 100, respectively. runtimes are initial run: 860ms, assertions:320ms -- retractions: 4s. Would you please give some hints on the reason? Thanks in advance. --Oli. - Ernest Friedman-Hill Informatics Decision Sciences, Sandia National Laboratories PO Box 969, MS 9012, Livermore, CA 94550 http://www.jessrules.com To unsubscribe, send the words 'unsubscribe jess-users y...@address.com' in the BODY of a message to majord...@sandia.gov, NOT to the list (use your own address!) List problems? Notify owner-jess-us...@sandia.gov.
Re: JESS: On the Performance of Logical Retractions
Although it may be obvious to some people, I thought I'd mention this well known lesson. Do not load huge knowledge base into memory. This lesson is well documented in existing literature on knowledge base systems. it's also been discussed on JESS mailing list numerous times over the years, so I would suggest searching JESS mailing list to learn from other people's experience. It's better to intelligently load knowledge base into memory as needed, rather than blindly load everything. Even in the case where someone has 256Gb of memory, one should ask why load all that into memory up front. If the test is using RDF triples, it's well known that RDF triples produces excessive partial matches and often results in OutOfMemoryException. The real issue isn't JESS, it's how one tries to solve a problem. I would recommend reading Gary Riley's book on expert systems to avoid repeating a lot of mistakes that others have already documented. On Thu, Jun 9, 2011 at 11:41 AM, Md Oliya md.ol...@gmail.com wrote: Thank you Ernest. I am experimenting with the Lehigh university benchmark, where i transfer OWL TBox into their equivalent rules in Jess, with the logical construct. Specifically, I am using the dataset and transformations, as used in the OpenRuleBench. As for the runtimes, I missed a point about the retractions. The fact is, even if the session does not contain any rules (no defrules, just assertions), loading the same set of retractions takes a considerable time. This indicates that the high runtime is mostly incurred by jess internal operations. but still, when the number of changes grows high (say more than 10%) the runtime is not acceptable, and rerunning with the retracted kb would be faster. I have another question as well: what type of truth maintenance method is implemented in jess? Do you solely rely on the Rete memory nodes and tokens for this purpose? --Oli. On Mon, Jun 6, 2011 at 7:37 PM, Ernest Friedman-Hill ejfr...@sandia.gov wrote: I don't think there's a particular reason in general. Retracting a fact takes only a little longer than asserting one, on average. But if we assume liberal use of logical, retracting a single fact could result in a sort of cascade effect whereby retracting a single fact would result in many other facts, and many activations, being removed also due to dependencies. Â All of that would take time. Â Still, your case seems extreme. Maybe there's something pathological about this particular case. On Jun 5, 2011, at 3:18 PM, Md Oliya wrote: Hi, I am doing some experiments with a set of rules which contain the logical CE. I intend to see the performance of Jess on a set of assertions as well as retractions. After some experiments, I found that the runtime for assertions is much less than that of retractions. In fact, the performance on retractions is so bad that I would rather re (run) jess on a retracted kb. A sample test case: The KB size, Â number of assertions, number of retractions, and number of rules are 100K, 50K, 1k, and 100, respectively. runtimes are initial run: 860ms, Â assertions:320ms -- Â retractions: 4s. Would you please give some hints on the reason? Thanks in advance. --Oli. - Ernest Friedman-Hill Informatics Decision Sciences, Sandia National Laboratories PO Box 969, MS 9012, Livermore, CA 94550 http://www.jessrules.com To unsubscribe, send the words 'unsubscribe jess-users y...@address.com' in the BODY of a message to majord...@sandia.gov, NOT to the list (use your own address!) List problems? Notify owner-jess-us...@sandia.gov. To unsubscribe, send the words 'unsubscribe jess-users y...@address.com' in the BODY of a message to majord...@sandia.gov, NOT to the list (use your own address!) List problems? Notify owner-jess-us...@sandia.gov.
Re: JESS: On the Performance of Logical Retractions
Thank you very much Peter for the useful information. I will definitely look into that. but in the context of this message, i am not loading a huge (subjective interpretation?) knowledge base. It's 100k assertions, with the operations taking around 400 MB. Secondly, in my experiments, I subtracted the loading time of the assertions/retractions in jess, as I'm focusing on the performance of the Rete. Lastly, I am not doing an RDF based mapping; rather, I follow the method of Description Logic Programs for translating each Class/Property of OWL into its corresponding template. --Oli. On Fri, Jun 10, 2011 at 12:03 AM, Peter Lin wool...@gmail.com wrote: Although it may be obvious to some people, I thought I'd mention this well known lesson. Do not load huge knowledge base into memory. This lesson is well documented in existing literature on knowledge base systems. it's also been discussed on JESS mailing list numerous times over the years, so I would suggest searching JESS mailing list to learn from other people's experience. It's better to intelligently load knowledge base into memory as needed, rather than blindly load everything. Even in the case where someone has 256Gb of memory, one should ask why load all that into memory up front. If the test is using RDF triples, it's well known that RDF triples produces excessive partial matches and often results in OutOfMemoryException. The real issue isn't JESS, it's how one tries to solve a problem. I would recommend reading Gary Riley's book on expert systems to avoid repeating a lot of mistakes that others have already documented. On Thu, Jun 9, 2011 at 11:41 AM, Md Oliya md.ol...@gmail.com wrote: Thank you Ernest. I am experimenting with the Lehigh university benchmark, where i transfer OWL TBox into their equivalent rules in Jess, with the logical construct. Specifically, I am using the dataset and transformations, as used in the OpenRuleBench. As for the runtimes, I missed a point about the retractions. The fact is, even if the session does not contain any rules (no defrules, just assertions), loading the same set of retractions takes a considerable time. This indicates that the high runtime is mostly incurred by jess internal operations. but still, when the number of changes grows high (say more than 10%) the runtime is not acceptable, and rerunning with the retracted kb would be faster. I have another question as well: what type of truth maintenance method is implemented in jess? Do you solely rely on the Rete memory nodes and tokens for this purpose? --Oli. On Mon, Jun 6, 2011 at 7:37 PM, Ernest Friedman-Hill ejfr...@sandia.gov wrote: I don't think there's a particular reason in general. Retracting a fact takes only a little longer than asserting one, on average. But if we assume liberal use of logical, retracting a single fact could result in a sort of cascade effect whereby retracting a single fact would result in many other facts, and many activations, being removed also due to dependencies. All of that would take time. Still, your case seems extreme. Maybe there's something pathological about this particular case. On Jun 5, 2011, at 3:18 PM, Md Oliya wrote: Hi, I am doing some experiments with a set of rules which contain the logical CE. I intend to see the performance of Jess on a set of assertions as well as retractions. After some experiments, I found that the runtime for assertions is much less than that of retractions. In fact, the performance on retractions is so bad that I would rather re (run) jess on a retracted kb. A sample test case: The KB size, number of assertions, number of retractions, and number of rules are 100K, 50K, 1k, and 100, respectively. runtimes are initial run: 860ms, assertions:320ms -- retractions: 4s. Would you please give some hints on the reason? Thanks in advance. --Oli. - Ernest Friedman-Hill Informatics Decision Sciences, Sandia National Laboratories PO Box 969, MS 9012, Livermore, CA 94550 http://www.jessrules.com To unsubscribe, send the words 'unsubscribe jess-users y...@address.com' in the BODY of a message to majord...@sandia.gov, NOT to the list (use your own address!) List problems? Notify owner-jess-us...@sandia.gov. To unsubscribe, send the words 'unsubscribe jess-users y...@address.com' in the BODY of a message to majord...@sandia.gov, NOT to the list (use your own address!) List problems? Notify owner-jess-us...@sandia.gov.
Re: JESS: On the Performance of Logical Retractions
I think I need to see the actual test program, or otherwise we need to get on the same page somehow. As a counter example, here's a little program with no rules that asserts about 10,000 facts one at a time and then retracts them. It takes 1.9 seconds (including JVM startup) on my Macbook. If I comment out the retract part, it takes 1.6 seconds. These would be faster if the facts weren't being parsed out of strings this way, twice, but regardless of that, this doesn't bear out the idea that retractions are pathologically slow. (foreach ?a (create$ a b c d e f g h i j k l m nn o p q r s t u v w x y z) (foreach ?b (create$ a b c d e f g h i j k l m n o p q r s t u v w x y z) (foreach ?c (create$ a b c d e f g h i j k l m n o p q r s t u v w x y z) (bind ?x (str-cat ?a ?b ?c)) (assert-string (str-cat ( ?x )) (foreach ?a (create$ a b c d e f g h i j k l m nn o p q r s t u v w x y z) (foreach ?b (create$ a b c d e f g h i j k l m n o p q r s t u v w x y z) (foreach ?c (create$ a b c d e f g h i j k l m n o p q r s t u v w x y z) (bind ?x (str-cat ?a ?b ?c)) (retract-string (str-cat ( ?x )) On Jun 9, 2011, at 11:41 AM, Md Oliya wrote: Thank you Ernest. I am experimenting with the Lehigh university benchmark, where i transfer OWL TBox into their equivalent rules in Jess, with the logical construct. Specifically, I am using the dataset and transformations, as used in the OpenRuleBench. As for the runtimes, I missed a point about the retractions. The fact is, even if the session does not contain any rules (no defrules, just assertions), loading the same set of retractions takes a considerable time. This indicates that the high runtime is mostly incurred by jess internal operations. but still, when the number of changes grows high (say more than 10%) the runtime is not acceptable, and rerunning with the retracted kb would be faster. I have another question as well: what type of truth maintenance method is implemented in jess? Do you solely rely on the Rete memory nodes and tokens for this purpose? --Oli. On Mon, Jun 6, 2011 at 7:37 PM, Ernest Friedman-Hill ejfr...@sandia.gov wrote: I don't think there's a particular reason in general. Retracting a fact takes only a little longer than asserting one, on average. But if we assume liberal use of logical, retracting a single fact could result in a sort of cascade effect whereby retracting a single fact would result in many other facts, and many activations, being removed also due to dependencies. All of that would take time. Still, your case seems extreme. Maybe there's something pathological about this particular case. On Jun 5, 2011, at 3:18 PM, Md Oliya wrote: Hi, I am doing some experiments with a set of rules which contain the logical CE. I intend to see the performance of Jess on a set of assertions as well as retractions. After some experiments, I found that the runtime for assertions is much less than that of retractions. In fact, the performance on retractions is so bad that I would rather re (run) jess on a retracted kb. A sample test case: The KB size, number of assertions, number of retractions, and number of rules are 100K, 50K, 1k, and 100, respectively. runtimes are initial run: 860ms, assertions:320ms -- retractions: 4s. Would you please give some hints on the reason? Thanks in advance. --Oli. - Ernest Friedman-Hill Informatics Decision Sciences, Sandia National Laboratories PO Box 969, MS 9012, Livermore, CA 94550 http://www.jessrules.com To unsubscribe, send the words 'unsubscribe jess-users y...@address.com' in the BODY of a message to majord...@sandia.gov, NOT to the list (use your own address!) List problems? Notify owner-jess-us...@sandia.gov . - Ernest Friedman-Hill Informatics Decision Sciences, Sandia National Laboratories PO Box 969, MS 9012, Livermore, CA 94550 http://www.jessrules.com To unsubscribe, send the words 'unsubscribe jess-users y...@address.com' in the BODY of a message to majord...@sandia.gov, NOT to the list (use your own address!) List problems? Notify owner-jess-us...@sandia.gov.
Re: JESS: On the Performance of Logical Retractions
By performance of RETE what are you referring to? There are many aspects of RETE, which one must study carefully. It's good that you're translating RDF to OWL, but the larger question is why use OWL/RDF in the first place? Unless the knowledge easily fits into axioms like sky is blue or typical RDF examples, there's no benefit to storing or using RDF. My own bias perspective on RDF/OWL. The real question isn't should I use RETE or how does RETE perform. The real question is how do I solve the problem efficiently? I've built compliance engines for trading systems using JESS. I can say from first hand experience, it's how you use the engine that has the biggest factor. I've done things like load 500K records to check compliance across a portfolio set with minimal latency for nightly batch processes. the key though is taking time to study existing literature and understanding things before jumping to a solution. providing concrete examples of what your doing will likely get better advice than making general statements. On Thu, Jun 9, 2011 at 12:17 PM, Md Oliya md.ol...@gmail.com wrote: Thank you very much Peter for the useful information. I will definitely look into that. but in the context of this message, i am not loading a huge (subjective interpretation?) knowledge base. It's 100k assertions, with the operations taking around 400 MB. Secondly, in my experiments, I subtracted the loading time of the assertions/retractions in jess, as I'm focusing on the performance of the Rete. Lastly, I am not doing an RDF based mapping; rather, I follow the method of Description Logic Programs for translating each Class/Property of OWL into its corresponding template. --Oli. On Fri, Jun 10, 2011 at 12:03 AM, Peter Lin wool...@gmail.com wrote: Although it may be obvious to some people, I thought I'd mention this well known lesson. Do not load huge knowledge base into memory. This lesson is well documented in existing literature on knowledge base systems. it's also been discussed on JESS mailing list numerous times over the years, so I would suggest searching JESS mailing list to learn from other people's experience. It's better to intelligently load knowledge base into memory as needed, rather than blindly load everything. Even in the case where someone has 256Gb of memory, one should ask why load all that into memory up front. If the test is using RDF triples, it's well known that RDF triples produces excessive partial matches and often results in OutOfMemoryException. The real issue isn't JESS, it's how one tries to solve a problem. I would recommend reading Gary Riley's book on expert systems to avoid repeating a lot of mistakes that others have already documented. On Thu, Jun 9, 2011 at 11:41 AM, Md Oliya md.ol...@gmail.com wrote: Thank you Ernest. I am experimenting with the Lehigh university benchmark, where i transfer OWL TBox into their equivalent rules in Jess, with the logical construct. Specifically, I am using the dataset and transformations, as used in the OpenRuleBench. As for the runtimes, I missed a point about the retractions. The fact is, even if the session does not contain any rules (no defrules, just assertions), loading the same set of retractions takes a considerable time. This indicates that the high runtime is mostly incurred by jess internal operations. but still, when the number of changes grows high (say more than 10%) the runtime is not acceptable, and rerunning with the retracted kb would be faster. I have another question as well: what type of truth maintenance method is implemented in jess? Do you solely rely on the Rete memory nodes and tokens for this purpose? --Oli. On Mon, Jun 6, 2011 at 7:37 PM, Ernest Friedman-Hill ejfr...@sandia.gov wrote: I don't think there's a particular reason in general. Retracting a fact takes only a little longer than asserting one, on average. But if we assume liberal use of logical, retracting a single fact could result in a sort of cascade effect whereby retracting a single fact would result in many other facts, and many activations, being removed also due to dependencies. Â All of that would take time. Â Still, your case seems extreme. Maybe there's something pathological about this particular case. On Jun 5, 2011, at 3:18 PM, Md Oliya wrote: Hi, I am doing some experiments with a set of rules which contain the logical CE. I intend to see the performance of Jess on a set of assertions as well as retractions. After some experiments, I found that the runtime for assertions is much less than that of retractions. In fact, the performance on retractions is so bad that I would rather re (run) jess on a retracted kb. A sample test case: The KB size, Â number of assertions, number of retractions, and number of rules are 100K, 50K, 1k, and 100, respectively. runtimes are initial run:
