Joshua,
I implemented a simple connection pool. Each client that wishes to write to the
SDB database asks the pool for a connection. If one is available it is used,
otherwise a new connection is made. When the writer is done with the connection
it is returned to the pool.
This has definitely improved the behavior. However, I find that I am still
getting SQLExceptions on some write operations. In all cases these exceptions
occur in a thread I don't recognize: "Thread-18" or "Thread-19", running the
"tuple loader."
An example:
------
WARN [Thread-18] (SDBConnection.java:326) - execUpdate: SQLException
ERROR: duplicate key value violates unique constraint "nodes_pkey"
Detail: Key (hash)=(3239162530368006953) already exists.
INSERT INTO Nodes (hash, lex, lang, datatype, type)
SELECT NNodeQuads1899993940.n0 , NNodeQuads1899993940.n1 ,
NNodeQuads1899993940.n2 , NNodeQuads1899993940.n3 , NNodeQuads1899993940.n4
FROM NNodeQuads1899993940 LEFT JOIN Nodes ON
(NNodeQuads1899993940.n0=Nodes.hash)
WHERE Nodes.hash IS NULL
ERROR [Thread-18] (LoaderTuplesNodes.java:345) - Error in thread: Exception
flushing
com.hp.hpl.jena.sdb.SDBException: Exception flushing
at
com.hp.hpl.jena.sdb.layout2.TupleLoaderBase.flush(TupleLoaderBase.java:220)
at
com.hp.hpl.jena.sdb.layout2.TupleLoaderBase.finish(TupleLoaderBase.java:155)
at
com.hp.hpl.jena.sdb.layout2.LoaderTuplesNodes.commitTuples(LoaderTuplesNodes.java:283)
at
com.hp.hpl.jena.sdb.layout2.LoaderTuplesNodes.access$100(LoaderTuplesNodes.java:31)
at
com.hp.hpl.jena.sdb.layout2.LoaderTuplesNodes$Commiter.run(LoaderTuplesNodes.java:318)
at java.lang.Thread.run(Thread.java:662)
Caused by: org.postgresql.util.PSQLException: ERROR: duplicate key value
violates unique constraint "nodes_pkey"
Detail: Key (hash)=(3239162530368006953) already exists.
at
org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2102)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1835)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:500)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:374)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeUpdate(AbstractJdbc2Statement.java:302)
at
com.hp.hpl.jena.sdb.sql.SDBConnection.execUpdate(SDBConnection.java:151)
at
com.hp.hpl.jena.sdb.layout2.TupleLoaderBase.flush(TupleLoaderBase.java:202)
... 5 more
------
Any ideas as to cause and cure?
Dave
-----Original Message-----
From: Joshua TAYLOR [mailto:[email protected]]
Sent: Friday, September 07, 2012 3:17 PM
To: [email protected]
Subject: Re: Model critical sections
On Fri, Sep 7, 2012 at 2:39 PM, Lebling, David (US SSA)
<[email protected]> wrote:
> There is a single service (and JVM) that writes the models (at least in this
> case), and many that read them. There is a single Store (and hence
> SDBConnection) for the service.
I'm sorry, I'm not sure I follow. Let me mirror this back, and please correct
me if I'm wrong. There is a single JVM running a bunch of threads. One of
these threads writes, and the rest only read. Since it's all within the same
JVM (and there's only one store), there's only one SDBConnection.
> I originally wrote the code so that it used the
> model.enterCriticalSection/model.leaveCriticalSection methods. This did not
> seem to prevent other threads from writing the same model inside the critical
> section.
>
> My belief is that model.begin/model.commit aren't permitted to nest, so it's
> key to avoid one thread trying to do a transaction when another already is.
>
> So, I tried doing both: model.enterCriticalSection + model.begin, finishing
> with model.commit + model.leaveCriticalSection. That doesn't seem to do the
> trick either. I'm still getting "already in transaction," meaning other
> threads are trying to write the same model and not being blocked by the
> critical section.
>
> So, is the ModelLock not a thread-safe lock?
I'm not sure what the purpose of the lock would be if it weren't thread safe;
it's there to permit concurrent access to a model. The model generation code
Model model = SDBFactory.connectNamedModel(store, name);
can produce different models on different invocation, and hence the Locks would
apply to different models, but the transaction handler can be the same for
them. After a coworker ran a quick test, it looks like the following can
happen.
Model m1 = SDBFactory.connectNamedModel(store, name); Model m2 =
SDBFactory.connectNamedModel(store, name);
m1 and m2 are distinct models, so their enter/leaveCriticalSections aren't
going to interfere; you'll be able to enter both critical sections at once.
However, they're built from the same underlying Store, and
m1.getTransactionHandler() == m2.getTransactionHandler()
so when you begin() with m1, and subsequently begin() with m2, you're getting
the nesting that you're seeing. So, you could either:
1) be sure to use the *same* model everywhere so that the
enter/leaveCriticalSections do what you want; or
2) give each writing thread its own Store and connection, and don't use
begin/commit in the reading threads. You shouldn't need to use
enter/leaveCriticalSection at all in this case. (The writers will be covered by
begin/commit, and the readers don't have to worry about read locks since
nothing will ever be writing to those models (the only model getting written to
is the one that the writer has access
to).)
The first is easier if you'll only be using one JVM, but the second should
scale to multiple JVMs.
//JT
--
Joshua Taylor, http://www.cs.rpi.edu/~tayloj/
