Rupert, Thanks for the additional explanation.
Regards, Minto Op 14-3-2013 10:31, Rupert Westenthaler schreef: > Hi Minto > > I am traveling this week and do not have time to work on this until > the weekend but I will have a look into this. > > Let me try to explain my concern again and make it more clear: > > The Jena TDB named graphs are hold in a single quad store table (SPOC > - Subject Predicate Object Context). On the Clerezza side you have a > TripleCollections (SPO) with a name (C). What that means is that all > Clerezza TripleCollections provided by the same > SingleTdbDatasetTcProvider do share the same SPOC table. meaning that > a change of any of those TripleCollections will cause a modification > in the Jena TDB Backend. This means that Iterators of all > TripleCollections need to make a ReadLock on the SPOC table (and not > only on the SPO section represented by the TripleCollection). > > While Clerezza allows to build a LockableMGraphWrapper over an MGrpah > this is not sufficient for the SingleTdbDatasetTcProvider as this will > only protect the SPO section and not the SPOC table used by the > backend. So changes in other graphs - or the creation of a new graph - > are still possible and will cause ConcurrentModificationExceptions as > reported. > > To solve this issue one needs to ensure that a single ReadWrite lock > is used for all TripleCollections provided by the > SingleTdbDatasetTcProvider as this will allow users to lock the whole > SPOC table of the backend when they perform operations on the Clerezza > TripleCollections. > > best > Rupert > > > On Thu, Mar 14, 2013 at 9:50 AM, Minto van der Sluis <[email protected]> wrote: >> Hi, >> >> Half of what the 2 of you write is not very clear to me. Probably due to >> being a novice when it comes to Clerezza internals. >> >> Maybe I will start with giving CLEREZZA-726 another try and then check >> if I still get these exceptions. >> >> Regard, >> >> Minto >> >> Op 13-3-2013 18:35, Reto Bachmann-Gmür schreef: >>> On Wed, Mar 13, 2013 at 6:04 PM, Rupert Westenthaler < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> I think that this is cased by the fact that if you create a >>>> LockableMGraph over MGraphs provided by the SingleTdbDatasetTcProvider >>>> you end up in a situation where you have multiple ReadWrite Locks on >>>> the same quad store (the Jena TDB dataset). This means that acquiring >>>> a write lock on one MGraph will not prohibit changes in other graphs - >>>> or the creation of new graphs. Because of that you will end up with >>>> ConcurrentModificationException when using iterators over triples >>>> (such as going over SPARQL results). >>>> >>> True. But where is the graph locked in the first place? It should aquire a >>> lock before iterating though the graph, does this happen? >>> >>> cheers, >>> reto >>> >>>> The solution would be to >>>> >>>> * create a single ReadWirte lock for the SingleTdbDatasetTcProvider >>>> * replace all synchronized(dataset){..} block with read/wirte locks >>>> * all methods returning MGraphs need to return LockableMGraph >>>> instances that do use the ReadWrite lock used by the >>>> SingleTdbDatasetTcProvider >>>> * users would than need to use the LockableMGraph instance provided by >>>> the provider and NOT wrap those with an other LockableMGraph instance >>>> (e.g. the LockableMGraphWrapper). >>>> >>>> best >>>> Rupert >>>> >>>> >>>> On Wed, Mar 13, 2013 at 5:31 PM, Minto van der Sluis <[email protected]> wrote: >>>>> Hi Folks, >>>>> >>>>> I ran into an issue is both the existing SingleTdbDatasetTcProvider and >>>>> my customized version (see CLEREZZA-736). >>>>> >>>>> How to reproduce: >>>>> 1) Have some process constantly inject new named graphs (I had a process >>>>> injecting 1000 named graphs) >>>>> 2) perform a query while 1 is still running. I used the following query: >>>>> >>>>> SELECT ?graphName WHERE { GRAPH ?graphName {} } LIMIT 10 OFFSET 0 >>>>> >>>>> 3) repeat step 2 a number of times (since the error does not always >>>> occur) >>>>> This results in a ConcurrentModificationException (see stacktrace >>>>> below). I am not sure whether this is a Clerezza or Jena issue. >>>>> >>>>> Anyone an idea what is causing this? Or more importantly how to fix it? >>>>> >>>>> Should I create a Jira issue for this? >>>>> >>>>> Regards, >>>>> >>>>> -- >>>>> ir. ing. Minto van der Sluis >>>>> Software innovator / renovator >>>>> Xup BV >>>>> >>>>> >>>>> Stacktrace: >>>>> java.util.ConcurrentModificationException: Iterator: started at 7103, >>>> now 7105 >>>>> at >>>> com.hp.hpl.jena.tdb.sys.DatasetControlMRSW.policyError(DatasetControlMRSW.java:157) >>>>> at >>>> com.hp.hpl.jena.tdb.sys.DatasetControlMRSW.access$000(DatasetControlMRSW.java:32) >>>>> at >>>> com.hp.hpl.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.checkCourrentModification(DatasetControlMRSW.java:110) >>>>> at >>>> com.hp.hpl.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.hasNext(DatasetControlMRSW.java:118) >>>>> at org.openjena.atlas.iterator.Iter$4.hasNext(Iter.java:295) >>>>> at >>>> com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.hasNext(GraphTDBBase.java:173) >>>>> at >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:76) >>>>> at >>>> org.apache.clerezza.rdf.jena.storage.JenaGraphAdaptor$1.hasNext(JenaGraphAdaptor.java:106) >>>>> at >>>> org.apache.clerezza.rdf.core.impl.AbstractTripleCollection$1.hasNext(AbstractTripleCollection.java:78) >>>>> at >>>> org.apache.clerezza.rdf.core.access.LockingIterator.hasNext(LockingIterator.java:47) >>>>> at >>>> org.apache.clerezza.rdf.jena.facade.JenaGraph$1.hasNext(JenaGraph.java:95) >>>>> at >>>> com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:76) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMapper.hasNextBinding(QueryIterTriplePattern.java:151) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:79) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBinding(QueryIterBlockTriples.java:64) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterGraph$QueryIterGraphInner.hasNextBinding(QueryIterGraph.java:123) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:79) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:59) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding(QueryIterSlice.java:76) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112) >>>>> at >>>> com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72) >>>>> at >>>> org.apache.clerezza.rdf.jena.sparql.ResultSetWrapper.<init>(ResultSetWrapper.java:39) >>>>> at >>>> org.apache.clerezza.rdf.jena.sparql.JenaSparqlEngine.execute(JenaSparqlEngine.java:68) >>>>> at >>>> org.apache.clerezza.rdf.core.access.TcManager.executeSparqlQuery(TcManager.java:272) >>>>> ... >>>>> >>>> >>>> -- >>>> | Rupert Westenthaler [email protected] >>>> | Bodenlehenstraße 11 ++43-699-11108907 >>>> | A-5500 Bischofshofen >>>> >> >> -- >> ir. ing. Minto van der Sluis >> Software innovator / renovator >> Xup BV >> >> Mobiel: +31 (0) 626 014541 >> > > > -- > | Rupert Westenthaler [email protected] > | Bodenlehenstraße 11 ++43-699-11108907 > | A-5500 Bischofshofen > > -- ir. ing. Minto van der Sluis Software innovator / renovator Xup BV Mobiel: +31 (0) 626 014541
