On 16/11/17 17:17, Jean-Marc Vanel wrote:
Sorry that was not clear , rewriting.
Good - because was wondering who wrote what :-)
2017-11-16 17:36 GMT+01:00 Jean-Marc Vanel <[email protected]>:
2017-11-16 16:11 GMT+01:00 Andy Seaborne <[email protected]>:
On 16/11/17 09:12, Jean-Marc Vanel wrote:
So I tested on the sandbox site,
and it got quickly frozen :( .
... <http://semantic-forms.cc:9111/>
In shell server side, I did a kill -3 to get the threads dump,
and there are 8 occurences of the same pattern in bold:
"application-akka.actor.default-dispatcher-13" #38 prio=5 os_prio=0
tid=0x00007fdb74007000 nid=0x5f5f waiting on condition
[0x00007fdb881b8000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x00000000f3f42378> (a
java.util.concurrent.Semaphore$FairSync)
Some other thread is holding the permit that control the number of
writers.
The WriterLock is a Semaphore with one permit.
The question becomes who has the permit and are they making progress
Note - the lock is not re-entrant. begin-begin on the same thread will
deadlock (in case Akka is being clever about actors to threads).
I assume from this that the error lies in my code, running fine with TDB1,
but TDB2 is mode picky .
I'm puzzled as to what to do.
Which bad practices to look for in the code ?
Instrumenting the code for run-time messages ?
My understading is that Play! framework activates one thread from a pool
to process the HTTP request, so here everything is sequential.
I also start 2 or 3 futures, so each in another thread.
So concurrent lock acquire can occur between threads, but I understood that
it's allowed, and even more efficient in TDB2 because several writes can
occur concurrently.
TDB2 is like TDB1 single-active-writer.
Embedded commits an single thread were always forbidden,
and I don't think that there are some, but this is possible.
When this happened in TDB1 , a TransactionException was thrown saying
"allready in transaction".
TDB2 does not throw in such case ?
I just checked it does.
So who is holding the permit and why aren't they giving it up?
Have they become blocked on something else ... which a waiting thread
has? (deadlock by starvation).
It's not clear what in TDB 1 doc is still good , and what not :
https://jena.apache.org/documentation/tdb/tdb_transactions.html
The limitations don't apply. The rest looks right.
Needs cleaning up to remove historical TDB1 stuff but the txn code is
much the same.
Could you send me the complete thread dump as an attachment please?
(Not knowing app the code it may be too hard to decipher and I don't
know when I will get to look at it if it isn't quickly providing a clue)
Andy
With TDB2, is it bad to share a Dataset across threads and transactions ?
What are the don't 's regarding TDB2 API use ?
Andy
* at
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAn
dCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcqu
ireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquir
eSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.Semaphore.acquire(Semaphore.java:312) at
org.apache.jena.dboe.transaction.txn.TransactionCoordinator.
acquireWriterLock(TransactionCoordinator.java:305)
at
org.apache.jena.dboe.transaction.txn.TransactionCoordinator.
begin(TransactionCoordinator.java:483)
at
org.apache.jena.dboe.transaction.txn.TransactionCoordinator.
begin(TransactionCoordinator.java:450)
at
org.apache.jena.dboe.transaction.txn.TransactionalBase.begin
(TransactionalBase.java:101)
at
org.apache.jena.tdb2.store.DatasetGraphTDB.begin(DatasetGrap
hTDB.java:418)
at
org.apache.jena.sparql.core.DatasetGraphWrapper.begin(Datase
tGraphWrapper.java:187)
at
org.apache.jena.sparql.core.DatasetGraphWrapper.begin(Datase
tGraphWrapper.java:187)
at
org.apache.jena.query.text.DatasetGraphText.begin(DatasetGra
phText.java:115)
at
org.apache.jena.sparql.core.DatasetImpl.begin(DatasetImpl.java:111)
at
org.w3.banana.jena.JenaDatasetStore$$anonfun$rw$1.apply(Jena
DatasetStore.scala:23)
at scala.util.Try$.apply(Try.scala:192) at
org.w3.banana.jena.JenaDatasetStore.rw
<http://org.w3.banana.jena.JenaDatasetStore.rw>(JenaDatasetS
tore.scala:22)
at org.w3.banana.jena.JenaDatasetStore.rw
<http://org.w3.banana.jena.JenaDatasetStore.rw>(JenaDatasetS
tore.scala:10)*
at
deductions.runtime.services.ApplicationFacadeImpl$class.labe
lForURITransaction(ApplicationFacadeImpl.scala:107)
at
controllers.Application$.labelForURITransaction(Application.scala:8)
at
controllers.WebPages$MainPagePrecomputePlay.<init>(WebPages.scala:46)
at
controllers.WebPages$MainPagePrecomputePlay$.apply(WebPages.scala:39)
at
controllers.WebPages$$anonfun$outputMainPageWithContent$1.ap
ply(WebPages.scala:64)
at
controllers.WebPages$$anonfun$outputMainPageWithContent$1.ap
ply(WebPages.scala:63)
at
play.api.mvc.ActionBuilder$$anonfun$apply$13.apply(Action.scala:371)
It looks like a dead lock.
The main database, concerned by these stacks, is not big: 181,799 quads.
uname -a
Linux scw-80de9e 4.5.7-std-3 #1 SMP Tue Jul 12 09:56:30 UTC 2016 x86_64
x86_64 x86_64 GNU/Linux
$ java -version
openjdk version "1.8.0_151"
OpenJDK Runtime Environment (build 1.8.0_151-8u151-b12-0ubuntu0.1
6.04.2-b12)
OpenJDK 64-Bit Server VM (build 25.151-b12, mixed mode)
I restarted the server, so that anyone can try.
I forgot to say that the code changes are very limited:
- switch for TDB2
val dts =
if (useTDB2)
org.apache.jena.tdb2.TDB2Factory.connectDataset(database_location)
else
TDBFactory.createDataset(Paths.get(database_location).toString())
- move an iterator inside a transaction after initial problem
reported
in first mail
2017-11-15 17:46 GMT+01:00 Andy Seaborne <[email protected]>:
Jean-Marc,
Thank you for the report!
On 15/11/17 11:18, Jean-Marc Vanel wrote:
Hi
I started to test TBD2 on my application semantic_forms .
I dumped the 3 databases with the TDB (1) tool,
then loaded with the TDB 2 tool :
runMain tdb2.tdbloader --loc TDB --verbose
/home/jmv/src/semantic_forms/scala/dump.nq
there where some bad quads with a literal subject from old bugs, that I
had
to manually edit out , but not a big deal.
At runtime many things work, but I had one runtime error in
/authenticate
:
*[TransactionException: Iterator used outside its original
transaction]*
yes - it's pickier than TDB1 :-)
Indeed , this is exactly that:
https://github.com/jmvanel/semantic_forms/blob/master/scala/
forms/src/main/scala/deductions/runtime/services/Authenticat
ion.scala#L66
See Txn.calculateRead for executing a transaction and return a value.
This was on my laptop.
I'll fix the code, which seems to be changes compatible with old TDB.
Then I'll upgrade to TDB2 on the sandbox, then on the real site.
Hope I don't bother you with non complete report .
Not at all.
Andy
--
Jean-Marc Vanel
http://www.semantic-forms.cc:9111/display?displayuri=http:/
/jmvanel.free.fr/jmv.rdf%23me#subject
<http://www.semantic-forms.cc:9111/display?displayuri=http://jmvanel.free.fr/jmv.rdf%23me>
Déductions SARL - Consulting, services, training,
Rule-based programming, Semantic Web
+33 (0)6 89 16 29 52 <+33%206%2089%2016%2029%2052>
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
