TIM uses a subclass of GraphView called GraphInMemory which is indeed also stateless.
ajs6f > On Dec 30, 2017, at 5:54 AM, Andy Seaborne <[email protected]> wrote: > > Have you tried TDB2 and TIM yet? > > On 30/12/17 00:09, Claude Warren wrote: >> When I tried to add a transaction on dest there was an error with a >> transaction already being active. > > This is what I mean by details - you are now saying that graphs are from the > same dataset. That's an additional feature imposed by this use of the test. > I'm sure if source and est were the same graph, nasty things will happen. > (Elsewhere, GraphUtil works carefully for the cases when the graphs are the > same). > > As a general Java comment on iterators over the same data structure: > > Don't modify the datastructure while iterating. > > You'll get either an exception (best case) or inconsistent results and > possibly differently inconsistent from run to run. > >> Does the concurrent modification error indicate that the model was somehow >> changed outside of an iterator? That is where I am used to seeing it. > > Graphs from dataset are stateless views of datasets. See the code in > GraphView. > > (IIRC -- they are in TDB2, and I've been slowly working on improving the > situation for TDB1 without making a breaking change. TIM - I think it's OK.) > >> The iterator call is in a recursive call to extractInfo(). >> Both graphs are in the same dataset. >> all txnX() methods are of the form: >> public static void txnBegin(Graph g) { >> if (g.getTransactionHandler().transactionsSupported()) { >> g.getTransactionHandler().begin(); >> } >> } > > ---- > Unrelated to the iterator: The test code could be shorted if: > > public static void txn(Graph g, Runnable action) { > if ( g.getTransactionHandler().transactionsSupported() ) > g.getTransactionHandler().execute(action); > else > action.run(); > } > > then > txn(source, ()-> > GraphExtract e = .... > e.extractInto(dest, node("a"), source) > ); > > The try-catch stuff is handled by TransactionHandler.execute. > > BTW You are lucky - you are passing graphs across transactions which was a > no-no in TDB1 until quite recently. > > Graph transactions and dataset transactions used to be unconnected. Graph > transactions were written for the pre-dataset world. > > Andy > >> Stepping through the extract code there is this method: >> public void extractInto( Node root ) >> { >> active.add( root ); >> Iterator<Triple> it = extractFrom.find( root, Node.ANY, >> Node.ANY ); >> while (it.hasNext()) >> { >> Triple t = it.next(); >> Node subRoot = t.getObject(); >> toUpdate.add( t ); >> if (! (active.contains( subRoot ) || b.stopAt( t ))) >> extractInto( subRoot ); >> } >> } >> } >> When the toUpdate.add(t) is called the >> DatasetControlMRSW$IteratorCheckNotConcurrent.eCount >> is incremented. >> I see in the code comments for IteratorCheckNotConcurrent that it expects >> writer to be on a different thread than reader. Is there a way around that >> restriction? I ask because all other graphs that I have tested work fine >> when reading and writing on the same thread. >> Thoughts? >> Claude >> On Fri, Dec 29, 2017 at 10:28 PM, Andy Seaborne <[email protected]> wrote: >>> The exception says there is incorrect MRSW use of iterators in the test. >>> >>> Iterator checking is very old code and I believe it works reliably - it is >>> not guaranteed to always catch errors though. That was never the claim. >>> >>> Too many details missing to say much else but I don't see any txn on dest. >>> >>> Andy >>> >>> >>> On 29/12/17 13:10, Claude Warren wrote: >>> >>>> fair enough. >>>> >>>> I created a ContractTest for GraphTxnTDB and discovered a lot of issues >>>> with transactions in the contract tests. I cleaned most of them up but am >>>> left with the following: >>>> >>>> /** >>>> * This test exposed that the update-existing-graph functionality was >>>> broken >>>> * if the target graph already contained any statements with a >>>> subject S >>>> * appearing as subject in the source graph - no further Spo >>>> statements >>>> were >>>> * added. >>>> */ >>>> @ContractTest >>>> public void testPartialUpdate() { >>>> Graph source = graphWith(producer.newInstance(), "a R b; b S >>>> e"); >>>> Graph dest = graphWith(producer.newInstance(), "b R d"); >>>> txnBegin(source); >>>> try { >>>> GraphExtract e = new GraphExtract(TripleBoundary.st >>>> opNowhere); >>>> e.extractInto(dest, node("a"), source); >>>> txnCommit(source); >>>> } >>>> catch (RuntimeException e) >>>> { >>>> txnRollback(source); >>>> fail( e.getMessage() ); >>>> >>>> } >>>> txnBegin(source); >>>> assertIsomorphic( >>>> graphWith( "a R b; b S e; b R d"), >>>> dest); >>>> txnRollback(source); >>>> >>>> } >>>> >>>> >>>> Notes: >>>> graphWith populates the graph with the triples defined in the string >>>> using a transaction. >>>> txnRollback performs an abort if transactions are supported. >>>> >>>> The problem is that an exception is thrown in the extractInfo() call. >>>> >>>> java.util.ConcurrentModificationException: Iterator: started at 6, now 7 >>>> at >>>> org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(Datas >>>> etControlMRSW.java:147) >>>> at >>>> org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(Datase >>>> tControlMRSW.java:32) >>>> at >>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC >>>> oncurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103) >>>> at >>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC >>>> oncurrent.hasNext(DatasetControlMRSW.java:110) >>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265) >>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265) >>>> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886) >>>> at >>>> org.apache.jena.util.iterator.WrappedIterator.hasNext(Wrappe >>>> dIterator.java:90) >>>> at >>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto( >>>> GraphExtract.java:80) >>>> at >>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto( >>>> GraphExtract.java:85) >>>> at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract. >>>> java:52) >>>> at >>>> org.apache.jena.graph.GraphContractTest.testPartialUpdate(Gr >>>> aphContractTest.java:1564) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >>>> ssorImpl.java:62) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >>>> thodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> at >>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( >>>> FrameworkMethod.java:45) >>>> at >>>> org.junit.internal.runners.model.ReflectiveCallable.run(Refl >>>> ectiveCallable.java:15) >>>> at >>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr >>>> ameworkMethod.java:42) >>>> at >>>> org.junit.internal.runners.statements.InvokeMethod.evaluate( >>>> InvokeMethod.java:20) >>>> at >>>> org.junit.internal.runners.statements.RunAfters.evaluate(Run >>>> Afters.java:30) >>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) >>>> at >>>> org.xenei.junit.contract.ContractTestRunner.runChild(Contrac >>>> tTestRunner.java:138) >>>> at >>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit >>>> 4ClassRunner.java:47) >>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) >>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) >>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) >>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) >>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) >>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300) >>>> at >>>> org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369) >>>> at org.xenei.junit.contract.ContractSuite.runChild(ContractSuit >>>> e.java:1) >>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) >>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60) >>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229) >>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50) >>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222) >>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300) >>>> at >>>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. >>>> run(JUnit4TestReference.java:86) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.TestExecution.run( >>>> TestExecution.java:38) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >>>> sts(RemoteTestRunner.java:459) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >>>> sts(RemoteTestRunner.java:678) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( >>>> RemoteTestRunner.java:382) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( >>>> RemoteTestRunner.java:192) >>>> >>>> Is this a bug, or a known limitation of GraphTxnTDB? >>>> >>>> Claude >>>> >>>> On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <[email protected]> wrote: >>>> >>>> Try it :-) >>>>> >>>>> Then try running it twice in a JVM. >>>>> >>>>> And then try TDB2 vs TDB1 vs TIM. >>>>> >>>>> Andy >>>>> >>>>> >>>>> On 28/12/17 23:49, Claude Warren wrote: >>>>> >>>>> Would the following code snippit work >>>>>> >>>>>> Graph g = // a TDB graph instance >>>>>> if (g.getTransactionHandler().transactionsSupported()) { >>>>>> Graph initial = graphWith( GraphFactory.createGraphMem(), >>>>>> "initial hasValue 42; also hasURI hello"); >>>>>> Graph extra = graphWith(GraphFactory.createGraphMem(), >>>>>> "extra hasValue 17; also hasURI world"); >>>>>> GraphUtil.addInto(g, initial); >>>>>> g.getTransactionHandler().begin(); >>>>>> GraphUtil.addInto(g, extra); >>>>>> g.getTransactionHandler().abort(); >>>>>> assertIsomorphic(initial, g); >>>>>> >>>>>> >>>>>> Notes: >>>>>> graphWith populates the graph with the triples defined in the >>>>>> string. >>>>>> GraphUtil.addInto() performs no transaction handling and copies >>>>>> the >>>>>> contents of the second param into the first param. >>>>>> >>>>>> I suspect that this will/should fail when the begin() is called as the >>>>>> transaction as we are switching from not using transactions to using >>>>>> them. >>>>>> >>>>>> Claude >>>>>> >>>>>> >>>>>> >>>> >>>>
