On 08/06/12 23:05, Simon Helsen wrote:
that seems to work. So the lock attribute is useless now. I am not
entirely sure why this fixes things since I cannot see anyone else
synchronizing on sConn, but I assume that happens somewhere outside.
Each StoreConnection has a TransactionManager, and
TransactionManager.begin$ is synchronized so implicitly sConn is locked.
I added a comment to https://issues.apache.org/jira/browse/JENA-252
More details there.
(if JIRA is running; there is a maintenance notice for major reorg of
the hardware/VMs today)
Andy
From: Simon Helsen/Toronto/IBM@IBMCA
To: [email protected]
Cc: Andy Seaborne <[email protected]>, [email protected]
Date: 06/08/2012 05:01 PM
Subject: Re: Release 2.7.1
------------------------------------------------------------------------
I'll try the proposed fix.
If you can tell me what you're looking for, I can debug this further. What
exactly do you mean by "It's hard to tell without how the
DatasetGraphTransaction's get made"?
As for the style, I followed the documentation on multiple threads, so I
create a dataset for each transaction and close it. A transaction is
unique per thread, i.e. not shared as recommended. You saw the model
already in the test I provide for 250, so we have something like this
private <E> E storeOperation(String query, QueryActivity<E>
queryActivity) {
Dataset dataset = TDBFactory.createDataset(this.location);
QueryExecution qe = null;
try {
dataset.begin(ReadWrite.READ) ;
qe = QueryExecutionFactory.create(query, dataset);
E e = queryActivity.run(qe);
dataset.commit();
return e;
} catch (Throwable e) {
dataset.abort();
throw new RuntimeException(e);
} finally {
try {
if (qe != null) {
qe.close();
}
} finally {
dataset.end();
}
}
}
for a query - and FYI, we do not allow update queries. There are some
other variations of this for type of reads, but in my test case it looks
like this is the one being used by the read transactions. Then we use this
one for writes:
private <E> E storeOperation(String graphName, ModelWriteActivity<E>
modelWriteActivity) {
Dataset dataset = TDBFactory.createDataset(this.location);
try {
dataset.begin(ReadWrite.WRITE);
Model m = dataset.getNamedModel(graphName);
E e = modelWriteActivity.run(m);
dataset.commit();
return e;
} catch (Throwable e) {
dataset.abort();
throw new RuntimeException(e);
} finally {
dataset.end();
}
}
and we have one for multiple models per transaction. Each of these are
executed within their own thread. The dataset is (obviously) never shared
Simon
From:
Andy Seaborne <[email protected]>
To:
[email protected]
Date:
06/08/2012 04:07 PM
Subject:
Re: Release 2.7.1
On 08/06/12 19:55, Simon Helsen wrote:
> hi again,
>
> k, so, it is tricky to distill a test case out of our framework because
> this one is based on multiple threads, but I am debugging it myself
> instead.
>
> So, I put the breakpoint in DatasetControlMRSW.policyError where the cme
> is thrown. When I do that and rerun the test suite, I get into a
scenario
> where 4 threads hang on the breakpoint. 3 of these are 'read' threads
and
> 1 is a 'write' thread. Now, the writeCounter says it has 4 (and the
> readCounter is 0), which is why the cme is thrown. So, the question is
why
> the writeCounter says 4.
>
> In order to find that out, I set a different breakpoint at startUpdate
in
> DatasetControlMRSW. I notice that it came from a query, i.e. a READ
> transaction. The violating code is this:
>
>
> @Override
> protected void _begin(ReadWrite readWrite)
> {
> synchronized(lock)
> {
> if ( ! haveUsedInTransaction )
> getBaseDatasetGraph().sync() ;
> haveUsedInTransaction = true ;
> DatasetGraphTxn dsgTxn = sConn.begin(readWrite) ;
> txn.set(dsgTxn) ;
> inTransaction.set(true) ;
> }
> }
>
>
> The bit
>
> if ( ! haveUsedInTransaction )
> getBaseDatasetGraph().sync() ;
>
> is the problem. It calls prefixes.sync() which calls
nodeTupleTable.sync()
> which calls startWrite() ;
>
> any ideas how to fix this?
>
> Simon
sync's are writes.
It's making sure that any non-transactional changes are put to disk
before the whole thing swicths into transactional mode. Not a usage to
be encouraged but the code needs to be safe.
It's hard to tell without how the DatasetGraphTransaction's get made but
try changing
synchronized(lock)
to
synchronized(sConn)
(the test suite passes - not surprising as it is a more conservative
choice than the built code.)
It's late here - but that might be the cause. Otherwise, how are the
dataset being made? There seem to be several about - that should be OK,
as should multiple threads per DatasetGraphTransaction (transactions are
per thread). Which style are you using?
Andy
