[
https://issues.apache.org/jira/browse/JENA-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stian Soiland-Reyes updated JENA-904:
-------------------------------------
Description:
I found that activeInterpreters in LPBRuleEngine leaks also if you don't
iterate through to the end - calling .close() on the iterator is not enough.
Clean-up seems to only happen when it.hasNext() is called and it returns false
- so this could happen also in cases like where you return after getting the
first hit.
{code}
@Test
public void testNotLeakingActiveInterpreters() throws Exception {
Graph data = Factory.createGraphMem();
data.add(new Triple(a, ty, C1));
data.add(new Triple(b, ty, C1));
List<Rule> rules = Rule
.parseRules("[r1: (?x p ?t) <- (?x rdf:type
C1), makeInstance(?x, p, C2, ?t)]"
+ "[r2: (?t rdf:type C2) <-
(?x rdf:type C1), makeInstance(?x, p, C2, ?t)]");
FBRuleInfGraph infgraph = (FBRuleInfGraph)
createReasoner(rules).bind(
data);
LPBRuleEngine engine = getEngineForGraph(infgraph);
assertEquals(0, engine.activeInterpreters.size());
assertEquals(0, engine.tabledGoals.size());
// we ask for a non-hit -- it works, but only because we call
it.hasNext()
ExtendedIterator<Triple> it = infgraph.find(nohit, ty, C1);
assertFalse(it.hasNext());
it.close();
assertEquals(0, engine.activeInterpreters.size());
// and again.
// Ensure this is not cached by asking for a different triple
pattern
ExtendedIterator<Triple> it2 = infgraph.find(nohit, ty, C2);
// uuups, forgot to call it.hasNext(). But .close() should tidy
it2.close();
assertEquals(0, engine.activeInterpreters.size());
// OK, let's ask for something that is in the graph
ExtendedIterator<Triple> it3 = infgraph.find(a, ty, C1);
assertTrue(it3.hasNext());
assertEquals(a, it3.next().getMatchSubject());
// .. and what if we forget to call next() to consume b?
// (e.g. return from a method with the first hit)
// this should be enough
it3.close();
// without leaks of activeInterpreters
assertEquals(0, engine.activeInterpreters.size());
}
{code}
When investigating this, I got as far as seeing that the activeInterpreters is
normally closed from hasNext() when it reaches the end here:
https://github.com/apache/jena/blob/master/jena-core/src/main/java/com/hp/hpl/jena/reasoner/rulesys/impl/Generator.java#L303
which is a zombie check.. but this doesn't work when hasNext() hasn't reach the
end, even though it is also called from close() here:
https://github.com/apache/jena/blob/master/jena-core/src/main/java/com/hp/hpl/jena/reasoner/rulesys/impl/LPTopGoalIterator.java#L199
Moving the checkForCompletions further down and calling .close() on any
nextToRun or lookAhead was not sufficient - there is always a secondary
Generator in the list which is not removed - only the top-level one is removed
normally.
I was unable to investigate any further as I could not understand the classes
that were creating the two Generators.
The inner LPInterpreter 147 is first created from generatorFor() call from
ConsumerChoicePointFrame constructor within LPInterpreter.setupTabledCall which
is coming from the ruleEngine.find.
This outer LPInterpreter is then added as part of the find().
In "normal operation" the inner one is removed through the zombie clearing,
while the outer one is removed through .close()
If you don't iterate through to the end, the inner one is not removed.
> LPBRuleEngine leaks activeInterpreters
> --------------------------------------
>
> Key: JENA-904
> URL: https://issues.apache.org/jira/browse/JENA-904
> Project: Apache Jena
> Issue Type: Bug
> Components: Ontology API
> Reporter: Stian Soiland-Reyes
>
> I found that activeInterpreters in LPBRuleEngine leaks also if you don't
> iterate through to the end - calling .close() on the iterator is not enough.
> Clean-up seems to only happen when it.hasNext() is called and it returns
> false - so this could happen also in cases like where you return after
> getting the first hit.
> {code}
> @Test
> public void testNotLeakingActiveInterpreters() throws Exception {
> Graph data = Factory.createGraphMem();
> data.add(new Triple(a, ty, C1));
> data.add(new Triple(b, ty, C1));
> List<Rule> rules = Rule
> .parseRules("[r1: (?x p ?t) <- (?x rdf:type
> C1), makeInstance(?x, p, C2, ?t)]"
> + "[r2: (?t rdf:type C2) <-
> (?x rdf:type C1), makeInstance(?x, p, C2, ?t)]");
> FBRuleInfGraph infgraph = (FBRuleInfGraph)
> createReasoner(rules).bind(
> data);
> LPBRuleEngine engine = getEngineForGraph(infgraph);
> assertEquals(0, engine.activeInterpreters.size());
> assertEquals(0, engine.tabledGoals.size());
> // we ask for a non-hit -- it works, but only because we call
> it.hasNext()
> ExtendedIterator<Triple> it = infgraph.find(nohit, ty, C1);
> assertFalse(it.hasNext());
> it.close();
> assertEquals(0, engine.activeInterpreters.size());
> // and again.
> // Ensure this is not cached by asking for a different triple
> pattern
> ExtendedIterator<Triple> it2 = infgraph.find(nohit, ty, C2);
> // uuups, forgot to call it.hasNext(). But .close() should tidy
> it2.close();
> assertEquals(0, engine.activeInterpreters.size());
>
> // OK, let's ask for something that is in the graph
>
> ExtendedIterator<Triple> it3 = infgraph.find(a, ty, C1);
> assertTrue(it3.hasNext());
> assertEquals(a, it3.next().getMatchSubject());
>
> // .. and what if we forget to call next() to consume b?
> // (e.g. return from a method with the first hit)
>
> // this should be enough
> it3.close();
> // without leaks of activeInterpreters
> assertEquals(0, engine.activeInterpreters.size());
> }
> {code}
> When investigating this, I got as far as seeing that the activeInterpreters
> is normally closed from hasNext() when it reaches the end here:
> https://github.com/apache/jena/blob/master/jena-core/src/main/java/com/hp/hpl/jena/reasoner/rulesys/impl/Generator.java#L303
> which is a zombie check.. but this doesn't work when hasNext() hasn't reach
> the end, even though it is also called from close() here:
> https://github.com/apache/jena/blob/master/jena-core/src/main/java/com/hp/hpl/jena/reasoner/rulesys/impl/LPTopGoalIterator.java#L199
> Moving the checkForCompletions further down and calling .close() on any
> nextToRun or lookAhead was not sufficient - there is always a secondary
> Generator in the list which is not removed - only the top-level one is
> removed normally.
> I was unable to investigate any further as I could not understand the classes
> that were creating the two Generators.
> The inner LPInterpreter 147 is first created from generatorFor() call from
> ConsumerChoicePointFrame constructor within LPInterpreter.setupTabledCall
> which is coming from the ruleEngine.find.
> This outer LPInterpreter is then added as part of the find().
> In "normal operation" the inner one is removed through the zombie clearing,
> while the outer one is removed through .close()
> If you don't iterate through to the end, the inner one is not removed.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
