I am just looking at the contract test code that has been around for awhile. Old test I suppose, or difference between TDB and other graph implementations. Hence my argument for contract tests, it helps identify where the implementations differ and allows us to document exactly what the expectations are.
In this case, based on my understanding of the above conversation, if I make one of the graphs a memory graph the test would pass. Thx for the assistance, Claude On Sat, Dec 30, 2017 at 10:54 AM, Andy Seaborne <[email protected]> wrote: > Have you tried TDB2 and TIM yet? > > On 30/12/17 00:09, Claude Warren wrote: > >> When I tried to add a transaction on dest there was an error with a >> transaction already being active. >> > > This is what I mean by details - you are now saying that graphs are from > the same dataset. That's an additional feature imposed by this use of the > test. I'm sure if source and est were the same graph, nasty things will > happen. (Elsewhere, GraphUtil works carefully for the cases when the graphs > are the same). > > As a general Java comment on iterators over the same data structure: > > Don't modify the datastructure while iterating. > > You'll get either an exception (best case) or inconsistent results and > possibly differently inconsistent from run to run. > > Does the concurrent modification error indicate that the model was somehow >> changed outside of an iterator? That is where I am used to seeing it. >> > > Graphs from dataset are stateless views of datasets. See the code in > GraphView. > > (IIRC -- they are in TDB2, and I've been slowly working on improving the > situation for TDB1 without making a breaking change. TIM - I think it's OK.) > > The iterator call is in a recursive call to extractInfo(). >> >> Both graphs are in the same dataset. >> >> all txnX() methods are of the form: >> >> public static void txnBegin(Graph g) { >> if (g.getTransactionHandler().transactionsSupported()) { >> g.getTransactionHandler().begin(); >> } >> } >> > > ---- > Unrelated to the iterator: The test code could be shorted if: > > public static void txn(Graph g, Runnable action) { > if ( g.getTransactionHandler().transactionsSupported() ) > g.getTransactionHandler().execute(action); > else > action.run(); > } > > then > txn(source, ()-> > GraphExtract e = .... > e.extractInto(dest, node("a"), source) > ); > > The try-catch stuff is handled by TransactionHandler.execute. > > BTW You are lucky - you are passing graphs across transactions which was a > no-no in TDB1 until quite recently. > > Graph transactions and dataset transactions used to be unconnected. Graph > transactions were written for the pre-dataset world. > > Andy > > > Stepping through the extract code there is this method: >> >> public void extractInto( Node root ) >> { >> active.add( root ); >> Iterator<Triple> it = extractFrom.find( root, Node.ANY, >> Node.ANY ); >> while (it.hasNext()) >> { >> Triple t = it.next(); >> Node subRoot = t.getObject(); >> toUpdate.add( t ); >> if (! (active.contains( subRoot ) || b.stopAt( t ))) >> extractInto( subRoot ); >> } >> } >> } >> >> When the toUpdate.add(t) is called the >> DatasetControlMRSW$IteratorCheckNotConcurrent.eCount >> is incremented. >> >> I see in the code comments for IteratorCheckNotConcurrent that it expects >> writer to be on a different thread than reader. Is there a way around >> that >> restriction? I ask because all other graphs that I have tested work fine >> when reading and writing on the same thread. >> >> Thoughts? >> >> Claude >> >> >> >> >> On Fri, Dec 29, 2017 at 10:28 PM, Andy Seaborne <[email protected]> wrote: >> >> The exception says there is incorrect MRSW use of iterators in the test. >>> >>> Iterator checking is very old code and I believe it works reliably - it >>> is >>> not guaranteed to always catch errors though. That was never the claim. >>> >>> Too many details missing to say much else but I don't see any txn on >>> dest. >>> >>> Andy >>> >>> >>> On 29/12/17 13:10, Claude Warren wrote: >>> >>> fair enough. >>>> >>>> I created a ContractTest for GraphTxnTDB and discovered a lot of issues >>>> with transactions in the contract tests. I cleaned most of them up but >>>> am >>>> left with the following: >>>> >>>> /** >>>> * This test exposed that the update-existing-graph functionality >>>> was >>>> broken >>>> * if the target graph already contained any statements with a >>>> subject S >>>> * appearing as subject in the source graph - no further Spo >>>> statements >>>> were >>>> * added. >>>> */ >>>> @ContractTest >>>> public void testPartialUpdate() { >>>> Graph source = graphWith(producer.newInstance(), "a R b; b S >>>> e"); >>>> Graph dest = graphWith(producer.newInstance(), "b R d"); >>>> txnBegin(source); >>>> try { >>>> GraphExtract e = new GraphExtract(TripleBoundary.st >>>> opNowhere); >>>> e.extractInto(dest, node("a"), source); >>>> txnCommit(source); >>>> } >>>> catch (RuntimeException e) >>>> { >>>> txnRollback(source); >>>> fail( e.getMessage() ); >>>> >>>> } >>>> txnBegin(source); >>>> assertIsomorphic( >>>> graphWith( "a R b; b S e; b R d"), >>>> dest); >>>> txnRollback(source); >>>> >>>> } >>>> >>>> >>>> Notes: >>>> graphWith populates the graph with the triples defined in the >>>> string >>>> using a transaction. >>>> txnRollback performs an abort if transactions are supported. >>>> >>>> The problem is that an exception is thrown in the extractInfo() call. >>>> >>>> java.util.ConcurrentModificationException: Iterator: started at 6, now >>>> 7 >>>> at >>>> org.apache.jena.tdb.sys.DatasetControlMRSW.policyError(Datas >>>> etControlMRSW.java:147) >>>> at >>>> org.apache.jena.tdb.sys.DatasetControlMRSW.access$000(Datase >>>> tControlMRSW.java:32) >>>> at >>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC >>>> oncurrent.checkIterConcurrentModification(DatasetControlMRSW.java:103) >>>> at >>>> org.apache.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotC >>>> oncurrent.hasNext(DatasetControlMRSW.java:110) >>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265) >>>> at org.apache.jena.atlas.iterator.Iter$2.hasNext(Iter.java:265) >>>> at org.apache.jena.atlas.iterator.Iter.hasNext(Iter.java:886) >>>> at >>>> org.apache.jena.util.iterator.WrappedIterator.hasNext(Wrappe >>>> dIterator.java:90) >>>> at >>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto( >>>> GraphExtract.java:80) >>>> at >>>> org.apache.jena.graph.GraphExtract$Extraction.extractInto( >>>> GraphExtract.java:85) >>>> at org.apache.jena.graph.GraphExtract.extractInto(GraphExtract. >>>> java:52) >>>> at >>>> org.apache.jena.graph.GraphContractTest.testPartialUpdate(Gr >>>> aphContractTest.java:1564) >>>> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) >>>> at >>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAcce >>>> ssorImpl.java:62) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMe >>>> thodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> at >>>> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( >>>> FrameworkMethod.java:45) >>>> at >>>> org.junit.internal.runners.model.ReflectiveCallable.run(Refl >>>> ectiveCallable.java:15) >>>> at >>>> org.junit.runners.model.FrameworkMethod.invokeExplosively(Fr >>>> ameworkMethod.java:42) >>>> at >>>> org.junit.internal.runners.statements.InvokeMethod.evaluate( >>>> InvokeMethod.java:20) >>>> at >>>> org.junit.internal.runners.statements.RunAfters.evaluate(Run >>>> Afters.java:30) >>>> at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263) >>>> at >>>> org.xenei.junit.contract.ContractTestRunner.runChild(Contrac >>>> tTestRunner.java:138) >>>> at >>>> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit >>>> 4ClassRunner.java:47) >>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) >>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java: >>>> 60) >>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner. >>>> java:229) >>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java: >>>> 50) >>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java: >>>> 222) >>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300) >>>> at >>>> org.xenei.junit.contract.ContractSuite.runChild(ContractSuite.java:369) >>>> at org.xenei.junit.contract.ContractSuite.runChild(ContractSuit >>>> e.java:1) >>>> at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231) >>>> at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java: >>>> 60) >>>> at org.junit.runners.ParentRunner.runChildren(ParentRunner. >>>> java:229) >>>> at org.junit.runners.ParentRunner.access$000(ParentRunner.java: >>>> 50) >>>> at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java: >>>> 222) >>>> at org.junit.runners.ParentRunner.run(ParentRunner.java:300) >>>> at >>>> org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference. >>>> run(JUnit4TestReference.java:86) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.TestExecution.run( >>>> TestExecution.java:38) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >>>> sts(RemoteTestRunner.java:459) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTe >>>> sts(RemoteTestRunner.java:678) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( >>>> RemoteTestRunner.java:382) >>>> at >>>> org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( >>>> RemoteTestRunner.java:192) >>>> >>>> Is this a bug, or a known limitation of GraphTxnTDB? >>>> >>>> Claude >>>> >>>> On Fri, Dec 29, 2017 at 11:14 AM, Andy Seaborne <[email protected]> >>>> wrote: >>>> >>>> Try it :-) >>>> >>>>> >>>>> Then try running it twice in a JVM. >>>>> >>>>> And then try TDB2 vs TDB1 vs TIM. >>>>> >>>>> Andy >>>>> >>>>> >>>>> On 28/12/17 23:49, Claude Warren wrote: >>>>> >>>>> Would the following code snippit work >>>>> >>>>>> >>>>>> Graph g = // a TDB graph instance >>>>>> if (g.getTransactionHandler().transactionsSupported()) { >>>>>> Graph initial = graphWith( >>>>>> GraphFactory.createGraphMem(), >>>>>> "initial hasValue 42; also hasURI hello"); >>>>>> Graph extra = graphWith(GraphFactory.createGraphMem(), >>>>>> "extra hasValue 17; also hasURI world"); >>>>>> GraphUtil.addInto(g, initial); >>>>>> g.getTransactionHandler().begin(); >>>>>> GraphUtil.addInto(g, extra); >>>>>> g.getTransactionHandler().abort(); >>>>>> assertIsomorphic(initial, g); >>>>>> >>>>>> >>>>>> Notes: >>>>>> graphWith populates the graph with the triples defined in the >>>>>> string. >>>>>> GraphUtil.addInto() performs no transaction handling and copies >>>>>> the >>>>>> contents of the second param into the first param. >>>>>> >>>>>> I suspect that this will/should fail when the begin() is called as the >>>>>> transaction as we are switching from not using transactions to using >>>>>> them. >>>>>> >>>>>> Claude >>>>>> >>>>>> >>>>>> >>>>>> >>>> >>>> >> >> -- I like: Like Like - The likeliest place on the web <http://like-like.xenei.com> LinkedIn: http://www.linkedin.com/in/claudewarren
