Re: ConcurrentModicationException on TDB storage provider (SingleDataset)

Andy Seaborne Thu, 14 Mar 2013 06:44:04 -0700

On 14/03/13 09:39, Minto van der Sluis wrote:

Rupert,


Thanks for the additional explanation.

Regards,

Minto

Op 14-3-2013 10:31, Rupert Westenthaler schreef:

Hi Minto

I am traveling this week and do not have time to work on this until
the weekend but I will have a look into this.

Let me try to explain my concern again and make it more clear:

The Jena TDB named graphs are hold in a single quad store table (SPOC
- Subject Predicate Object Context). On the Clerezza side you have a
TripleCollections (SPO) with a name (C). What that means is that all
Clerezza TripleCollections provided by the same
SingleTdbDatasetTcProvider do share the same SPOC table. meaning that
a change of any of those TripleCollections will cause a modification
in the Jena TDB Backend. This means that Iterators of all
TripleCollections need to make a ReadLock on the SPOC table (and not
only on the SPO section represented by the TripleCollection).

While Clerezza allows to build a LockableMGraphWrapper over an MGrpah
this is not sufficient for the SingleTdbDatasetTcProvider as this will
only protect the SPO section and not the SPOC table used by the
backend. So changes in other graphs - or the creation of a new graph -
are still possible and will cause ConcurrentModificationExceptions as
reported.

To solve this issue one needs to ensure that a single ReadWrite lock
is used for all TripleCollections provided by the
SingleTdbDatasetTcProvider as this will allow users to lock the whole
SPOC table of the backend when they perform operations on the Clerezza
TripleCollections.

A TDB dataset provides a single Lock you can reuse/wrap so all the graphlocks are related when needed. The GraphTDB.getLock() is the dataset lock.

Transactions would be better. Better concurrency (concurrent writer andmultiple readers).


        Andy


best
Rupert


On Thu, Mar 14, 2013 at 9:50 AM, Minto van der Sluis <[email protected]> wrote:

Hi,

Half of what the 2 of you write is not very clear to me. Probably due to
being a novice when it comes to Clerezza internals.

Maybe I will start with giving CLEREZZA-726 another try and then check
if I still get these exceptions.

Regard,

Minto

Op 13-3-2013 18:35, Reto Bachmann-Gmür schreef:

On Wed, Mar 13, 2013 at 6:04 PM, Rupert Westenthaler <
[email protected]> wrote:

Hi,

I think that this is cased by the fact that if you create a
LockableMGraph over MGraphs provided by the SingleTdbDatasetTcProvider
you end up in a situation where you have multiple ReadWrite Locks on
the same quad store (the Jena TDB dataset). This means that acquiring
a write lock on one MGraph will not prohibit changes in other graphs -
or the creation of new graphs. Because of that you will end up with
ConcurrentModificationException when using iterators over triples
(such as going over SPARQL results).

True. But where is the graph locked in the first place? It should aquire a
lock  before iterating though the graph, does this happen?

cheers,
reto

The solution would be to

* create a single ReadWirte lock for the SingleTdbDatasetTcProvider
* replace all synchronized(dataset){..} block with read/wirte locks
* all methods returning MGraphs need to return LockableMGraph
instances that do use the ReadWrite lock used by the
SingleTdbDatasetTcProvider
* users would than need to use the LockableMGraph instance provided by
the provider and NOT wrap those with an other LockableMGraph instance
(e.g. the LockableMGraphWrapper).

best
Rupert


On Wed, Mar 13, 2013 at 5:31 PM, Minto van der Sluis <[email protected]> wrote:

Hi Folks,

I ran into an issue is both the existing SingleTdbDatasetTcProvider and
my customized version (see CLEREZZA-736).

How to reproduce:
1) Have some process constantly inject new named graphs (I had a process
injecting 1000 named graphs)
2) perform a query while 1 is still running. I used the following query:

     SELECT ?graphName WHERE {   GRAPH ?graphName {} } LIMIT 10 OFFSET 0

3) repeat step 2 a number of times (since the error does not always

occur)

This results in a ConcurrentModificationException (see stacktrace
below). I am not sure whether this is a Clerezza or Jena issue.

Anyone an idea what is causing this? Or more importantly how to fix it?

Should I create a Jira issue for this?

Regards,

--
ir. ing. Minto van der Sluis
Software innovator / renovator
Xup BV


Stacktrace:
java.util.ConcurrentModificationException: Iterator: started at 7103,

now 7105

at

com.hp.hpl.jena.tdb.sys.DatasetControlMRSW.policyError(DatasetControlMRSW.java:157)

at

com.hp.hpl.jena.tdb.sys.DatasetControlMRSW.access$000(DatasetControlMRSW.java:32)

at

com.hp.hpl.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.checkCourrentModification(DatasetControlMRSW.java:110)

at

com.hp.hpl.jena.tdb.sys.DatasetControlMRSW$IteratorCheckNotConcurrent.hasNext(DatasetControlMRSW.java:118)

         at org.openjena.atlas.iterator.Iter$4.hasNext(Iter.java:295)
         at

com.hp.hpl.jena.tdb.store.GraphTDBBase$ProjectQuadsToTriples.hasNext(GraphTDBBase.java:173)

at

com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:76)

at

org.apache.clerezza.rdf.jena.storage.JenaGraphAdaptor$1.hasNext(JenaGraphAdaptor.java:106)

at

org.apache.clerezza.rdf.core.impl.AbstractTripleCollection$1.hasNext(AbstractTripleCollection.java:78)

at

org.apache.clerezza.rdf.core.access.LockingIterator.hasNext(LockingIterator.java:47)

at

org.apache.clerezza.rdf.jena.facade.JenaGraph$1.hasNext(JenaGraph.java:95)

at

com.hp.hpl.jena.util.iterator.WrappedIterator.hasNext(WrappedIterator.java:76)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIterTriplePattern$TripleMapper.hasNextBinding(QueryIterTriplePattern.java:151)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:79)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIterBlockTriples.hasNextBinding(QueryIterBlockTriples.java:64)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.main.iterator.QueryIterGraph$QueryIterGraphInner.hasNextBinding(QueryIterGraph.java:123)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIterRepeatApply.hasNextBinding(QueryIterRepeatApply.java:79)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIterConvert.hasNextBinding(QueryIterConvert.java:59)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIterSlice.hasNextBinding(QueryIterSlice.java:76)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorWrapper.hasNextBinding(QueryIteratorWrapper.java:40)

at

com.hp.hpl.jena.sparql.engine.iterator.QueryIteratorBase.hasNext(QueryIteratorBase.java:112)

at

com.hp.hpl.jena.sparql.engine.ResultSetStream.hasNext(ResultSetStream.java:72)

at

org.apache.clerezza.rdf.jena.sparql.ResultSetWrapper.<init>(ResultSetWrapper.java:39)

at

org.apache.clerezza.rdf.jena.sparql.JenaSparqlEngine.execute(JenaSparqlEngine.java:68)

at

org.apache.clerezza.rdf.core.access.TcManager.executeSparqlQuery(TcManager.java:272)

...


--
| Rupert Westenthaler             [email protected]
| Bodenlehenstraße 11                             ++43-699-11108907
| A-5500 Bischofshofen


--
ir. ing. Minto van der Sluis
Software innovator / renovator
Xup BV

Mobiel: +31 (0) 626 014541



--
| Rupert Westenthaler             [email protected]
| Bodenlehenstraße 11                             ++43-699-11108907
| A-5500 Bischofshofen

Re: ConcurrentModicationException on TDB storage provider (SingleDataset)

Reply via email to