Have you tried TDB2 and TIM yet?
On 30/12/17 00:09, Claude Warren wrote:
When I tried to add a transaction on dest there was an error with a
transaction already being active.
This is what I mean by details - you are now saying that graphs are from
the same dataset. That's an additional feature imposed by this use of
the test. I'm sure if source and est were the same graph, nasty things
will happen. (Elsewhere, GraphUtil works carefully for the cases when
the graphs are the same).
As a general Java comment on iterators over the same data structure:
Don't modify the datastructure while iterating.
You'll get either an exception (best case) or inconsistent results and
possibly differently inconsistent from run to run.
Does the concurrent modification error indicate that the model was somehow
changed outside of an iterator? That is where I am used to seeing it.
Graphs from dataset are stateless views of datasets. See the code in
GraphView.
(IIRC -- they are in TDB2, and I've been slowly working on improving the
situation for TDB1 without making a breaking change. TIM - I think it's OK.)
The iterator call is in a recursive call to extractInfo().
Both graphs are in the same dataset.
all txnX() methods are of the form:
public static void txnBegin(Graph g) {
if (g.getTransactionHandler().transactionsSupported()) {
g.getTransactionHandler().begin();
}
}
----
Unrelated to the iterator: The test code could be shorted if:
public static void txn(Graph g, Runnable action) {
if ( g.getTransactionHandler().transactionsSupported() )
g.getTransactionHandler().execute(action);
else
action.run();
}
then
txn(source, ()->
GraphExtract e = ....
e.extractInto(dest, node("a"), source)
);
The try-catch stuff is handled by TransactionHandler.execute.
BTW You are lucky - you are passing graphs across transactions which was
a no-no in TDB1 until quite recently.
Graph transactions and dataset transactions used to be unconnected.
Graph transactions were written for the pre-dataset world.
Andy
Stepping through the extract code there is this method:
public void extractInto( Node root )
{
active.add( root );
Iterator<Triple> it = extractFrom.find( root, Node.ANY,
Node.ANY );
while (it.hasNext())
{
Triple t = it.next();
Node subRoot = t.getObject();
toUpdate.add( t );
if (! (active.contains( subRoot ) || b.stopAt( t )))
extractInto( subRoot );
}
}
}
When the toUpdate.add(t) is called the
DatasetControlMRSW$IteratorCheckNotConcurrent.eCount
is incremented.
I see in the code comments for IteratorCheckNotConcurrent that it expects
writer to be on a different thread than reader. Is there a way around that
restriction? I ask because all other graphs that I have tested work fine
when reading and writing on the same thread.
Thoughts?
Claude
On Fri, Dec 29, 2017 at 10:28 PM, Andy Seaborne <[email protected]> wrote:
The exception says there is incorrect MRSW use of iterators in the test.
Iterator checking is very old code and I believe it works reliably - it is
not guaranteed to always catch errors though. That was never the claim.
Too many details missing to say much else but I don't see any txn on dest.
Andy
On 29/12/17 13:10, Claude Warren wrote:
fair enough.
I created a ContractTest for GraphTxnTDB and discovered a lot of issues
with transactions in the contract tests. I cleaned most of them up but am
left with the following:
/**
* This test exposed that the update-existing-graph functionality was
broken
* if the target graph already contained any statements with a
subject S
* appearing as subject in the source graph - no further Spo
statements
were
* added.
*/
@ContractTest
public void testPartialUpdate() {
Graph source = graphWith(producer.newInstance(), "a R b; b S
e");
Graph dest = graphWith(producer.newInstance(), "b R d");
txnBegin(source);
try {
GraphExtract e = new GraphExtract(TripleBoundary.st
opNowhere);
e.extractInto(dest, node("a"), source);
txnCommit(source);
}
catch (RuntimeException e)
{
txnRollback(source);
fail( e.getMessage() );
}
txnBegin(source);
assertIsomorphic(
graphWith( "a R b; b S e; b R d"),
dest);
txnRollback(source);
}
Notes:
graphWith populates the graph with the triples defined in the string
using a transaction.
txnRollback performs an abort if transactions are supported.
The problem is that an exception is thrown in the extractInfo() call.
java.util.ConcurrentModificationException: Iterator: started at 6, now 7
at
org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(Datas
etControlMRSW.java:147)
at
org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(Datase
tControlMRSW.java:32)
at
org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
oncurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103)
at
org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC
oncurrent.hasNext(DatasetControlMRSW.java:110)
at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265)
at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886)
at
org.apache.jena.util.iterator.WrappedIterator.hasNext(Wrappe
dIterator.java:90)
at
org.apache.jena.graph.GraphExtract$Extraction.extractInto(
GraphExtract.java:80)
at
org.apache.jena.graph.GraphExtract$Extraction.extractInto(
GraphExtract.java:85)
at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract.
java:52)
at
org.apache.jena.graph.GraphContractTest.testPartialUpdate(Gr
aphContractTest.java:1564)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce
ssorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe
thodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(
FrameworkMethod.java:45)
at
org.junit.internal.runners.model.ReflectiveCallable.run(Refl
ectiveCallable.java:15)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr
ameworkMethod.java:42)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(
InvokeMethod.java:20)
at
org.junit.internal.runners.statements.RunAfters.evaluate(Run
Afters.java:30)
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
at
org.xenei.junit.contract.ContractTestRunner.runChild(Contrac
tTestRunner.java:138)
at
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit
4ClassRunner.java:47)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369)
at org.xenei.junit.contract.ContractSuite.runChild(ContractSuit
e.java:1)
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
at
org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.
run(JUnit4TestReference.java:86)
at
org.eclipse.jdt.internal.junit.runner.TestExecution.run(
TestExecution.java:38)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
sts(RemoteTestRunner.java:459)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe
sts(RemoteTestRunner.java:678)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(
RemoteTestRunner.java:382)
at
org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(
RemoteTestRunner.java:192)
Is this a bug, or a known limitation of GraphTxnTDB?
Claude
On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <[email protected]> wrote:
Try it :-)
Then try running it twice in a JVM.
And then try TDB2 vs TDB1 vs TIM.
Andy
On 28/12/17 23:49, Claude Warren wrote:
Would the following code snippit work
Graph g = // a TDB graph instance
if (g.getTransactionHandler().transactionsSupported()) {
Graph initial = graphWith( GraphFactory.createGraphMem(),
"initial hasValue 42; also hasURI hello");
Graph extra = graphWith(GraphFactory.createGraphMem(),
"extra hasValue 17; also hasURI world");
GraphUtil.addInto(g, initial);
g.getTransactionHandler().begin();
GraphUtil.addInto(g, extra);
g.getTransactionHandler().abort();
assertIsomorphic(initial, g);
Notes:
graphWith populates the graph with the triples defined in the
string.
GraphUtil.addInto() performs no transaction handling and copies
the
contents of the second param into the first param.
I suspect that this will/should fail when the begin() is called as the
transaction as we are switching from not using transactions to using
them.
Claude
