On 07/03/14 10:36, Christophe FAGOT wrote:
Hi there,
I’m actually using Jena in forward rete configuration, with Jena 2.11.1. If I
use some very simple rules like the following, the memory used by my JVM
becomes huge with few statements (around 100Mo for only 3500 statements). I ran
a simple test in which instances and data property values were always
different, in order to avoid combinatorial combinaisons.
The sample rule :
[rule: (?instance rdf:type test:Instance),
(?object rdf:type test:Object),
(?instance test:instanceAge ?age),
(?object test:objectAge ?age),
->
(?instance test:isLinked "true"^^xsd:boolean)]
Does anyone see a reason for such huge memory space required by the RETE engine
? Any workaround to use ?
It's due to the crudeness of the Jena RETE implementation.
It is in effect doing (pseudo-notation):
(?instance rdf:type test:Instance)
(?object rdf:type test:Object)
(?instance test:instanceAge ?age1)
(?object test:objectAge ?age2)
(= ?age1 ?age2)
So that it builds a queue of all possible instance/object pairs before
the final join on age.
Which means that queue ends up as N(objects) * N(instances) long. So
quadratic rather than combinatorial but that's enough for the blow up.
Can't think of a work around.
If you can use backward rules instead of RETE that should help.
The RETE engine could certainly be improved (patches welcome). Though
the looooong term best option might be to consider phasing out both the
existing rule engines in favour of something simpler and more scalable.
Not something I could personally contribute to right now.
Dave
