Re: Followup GeoSparql + TDB 1 : freeze
Hi Jean-Marc,
If you are only interested in the GeoSPARQL predicates then perhaps try
using the no index options as outlined in the documentation:
*
No indexes setup (Query rewrite still performed but results not
stored) : |GeoSPARQLConfig.setupNoIndex()|
*
No indexes and no query rewriting: |GeoSPARQLConfig.setupNoIndex(false)|
Thanks,
Greg
On 28/03/2021 15:27, Andy Seaborne wrote:
Assuming not running in Fuseki ...
There is code not closing transactions.
There a large number of uncompleted write transactions (which may not
have made an update but did begin(WRITE). The default threshold before
"exclusiveFlushQueue" is 100. The thread doing them either dropped
them or exited with clearing up.
An uncommitted transaction blocks immediate write-back of the changes
in the current transaction.
TransactionManager.exclusiveFlushQueue means it has gone into panic mode.
Checking local logging for messages. If there are none about
uncommitted transactions, the code isn't going through
JenaDatasetStore.rw.
Andy
On 28/03/2021 09:51, Jean-Marc Vanel wrote:
For GeoSparql + TDB 1, I applied this initialization code on my site:
val res = configureLuceneIndex(dts, useTextQuery, useSpatialIndex)
if (useSpatialIndex) {
import org.apache.jena.geosparql.configuration._
GeoSPARQLConfig.setupMemoryIndex // actually registers special SPARQL
predicates!
GeoSPARQLConfig.setupSpatialIndex(res)
}
Full code:
https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/RDFStoreLocalJenaProvider.scala#L164
and I could indeed make "rectangular" queries on my geo: (
http://www.w3.org/2003/01/geo/wgs84_pos# ) data,
but even when not running any GeoSparql query,
the application web site got frozen, several times, so I was obliged to
deactivate GeoSparql initialization.
here is an example of a stack of a number of threads that run for a long
time and retain memory:
#132 daemon prio=5 os_prio=0 cpu=44.41ms elapsed=215.79s
tid=0x7f69e8012000 nid=0x2370 waiting on condition
[0x7f68826d4000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.10/Native
Method)
- parking to wait for <0xbcd9d198> (a
java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
at
java.util.concurrent.locks.LockSupport.park(java.base@11.0.10
/LockSupport.java:194)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11.0.10
/AbstractQueuedSynchronizer.java:885)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@11.0.10
/AbstractQueuedSynchronizer.java:917)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@11.0.10
/AbstractQueuedSynchronizer.java:1240)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(java.base@11.0.10
/ReentrantReadWriteLock.java:959)
at
org.apache.jena.tdb.transaction.TransactionManager.startExclusiveMode(TransactionManager.java:706)
at
org.apache.jena.tdb.transaction.TransactionManager.exclusiveFlushQueue(TransactionManager.java:661)
at
org.apache.jena.tdb.transaction.TransactionManager.notifyCommit(TransactionManager.java:567)
at
org.apache.jena.tdb.transaction.Transaction.commit(Transaction.java:143)
at
org.apache.jena.tdb.transaction.DatasetGraphTxn.commit(DatasetGraphTxn.java:61)
at
org.apache.jena.tdb.transaction.DatasetGraphTransaction.commit(DatasetGraphTransaction.java:200)
at
org.apache.jena.sparql.core.DatasetImpl.commit(DatasetImpl.java:152)
at
org.w3.banana.jena.JenaDatasetStore.$anonfun$rw$1(JenaDatasetStore.scala:26)
at
org.w3.banana.jena.JenaDatasetStore$$Lambda$1102/0x000840a8b840.apply(Unknown
Source)
at scala.util.Try$.apply(Try.scala:213)
at
org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:22)
at
org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:10)
at
deductions.runtime.sparql_cache.SPARQLHelpers.wrapInTransaction(SPARQLHelpers.scala:237)
at
deductions.runtime.sparql_cache.SPARQLHelpers.wrapInTransaction$(SPARQLHelpers.scala:235)
at
controllers.WebPagesApp$$anon$1.wrapInTransaction(WebPages.scala:51)
at
deductions.runtime.abstract_syntax.InstanceLabelsInferenceMemory.$anonfun$makeInstanceLabelFuture$1(InstanceLabelsInferenceMemory.scala:49)
This transaction just adds one literal triple to TDB.
There are several such stacks present, seen when executing kill -3
when the
web application is frozen:
http://jmvanel.free.fr/tmp/thread_dump.txt
Jean-Marc Vanel
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
Re: Followup GeoSparql + TDB 1 : freeze
Assuming not running in Fuseki ...
There is code not closing transactions.
There a large number of uncompleted write transactions (which may not
have made an update but did begin(WRITE). The default threshold before
"exclusiveFlushQueue" is 100. The thread doing them either dropped them
or exited with clearing up.
An uncommitted transaction blocks immediate write-back of the changes in
the current transaction.
TransactionManager.exclusiveFlushQueue means it has gone into panic mode.
Checking local logging for messages. If there are none about uncommitted
transactions, the code isn't going through JenaDatasetStore.rw.
Andy
On 28/03/2021 09:51, Jean-Marc Vanel wrote:
For GeoSparql + TDB 1, I applied this initialization code on my site:
val res = configureLuceneIndex(dts, useTextQuery, useSpatialIndex)
if (useSpatialIndex) {
import org.apache.jena.geosparql.configuration._
GeoSPARQLConfig.setupMemoryIndex // actually registers special SPARQL
predicates!
GeoSPARQLConfig.setupSpatialIndex(res)
}
Full code:
https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/RDFStoreLocalJenaProvider.scala#L164
and I could indeed make "rectangular" queries on my geo: (
http://www.w3.org/2003/01/geo/wgs84_pos# ) data,
but even when not running any GeoSparql query,
the application web site got frozen, several times, so I was obliged to
deactivate GeoSparql initialization.
here is an example of a stack of a number of threads that run for a long
time and retain memory:
#132 daemon prio=5 os_prio=0 cpu=44.41ms elapsed=215.79s
tid=0x7f69e8012000 nid=0x2370 waiting on condition [0x7f68826d4000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.10/Native Method)
- parking to wait for <0xbcd9d198> (a
java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.10
/LockSupport.java:194)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11.0.10
/AbstractQueuedSynchronizer.java:885)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@11.0.10
/AbstractQueuedSynchronizer.java:917)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@11.0.10
/AbstractQueuedSynchronizer.java:1240)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(java.base@11.0.10
/ReentrantReadWriteLock.java:959)
at
org.apache.jena.tdb.transaction.TransactionManager.startExclusiveMode(TransactionManager.java:706)
at
org.apache.jena.tdb.transaction.TransactionManager.exclusiveFlushQueue(TransactionManager.java:661)
at
org.apache.jena.tdb.transaction.TransactionManager.notifyCommit(TransactionManager.java:567)
at
org.apache.jena.tdb.transaction.Transaction.commit(Transaction.java:143)
at
org.apache.jena.tdb.transaction.DatasetGraphTxn.commit(DatasetGraphTxn.java:61)
at
org.apache.jena.tdb.transaction.DatasetGraphTransaction.commit(DatasetGraphTransaction.java:200)
at
org.apache.jena.sparql.core.DatasetImpl.commit(DatasetImpl.java:152)
at
org.w3.banana.jena.JenaDatasetStore.$anonfun$rw$1(JenaDatasetStore.scala:26)
at
org.w3.banana.jena.JenaDatasetStore$$Lambda$1102/0x000840a8b840.apply(Unknown
Source)
at scala.util.Try$.apply(Try.scala:213)
at org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:22)
at org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:10)
at
deductions.runtime.sparql_cache.SPARQLHelpers.wrapInTransaction(SPARQLHelpers.scala:237)
at
deductions.runtime.sparql_cache.SPARQLHelpers.wrapInTransaction$(SPARQLHelpers.scala:235)
at
controllers.WebPagesApp$$anon$1.wrapInTransaction(WebPages.scala:51)
at
deductions.runtime.abstract_syntax.InstanceLabelsInferenceMemory.$anonfun$makeInstanceLabelFuture$1(InstanceLabelsInferenceMemory.scala:49)
This transaction just adds one literal triple to TDB.
There are several such stacks present, seen when executing kill -3 when the
web application is frozen:
http://jmvanel.free.fr/tmp/thread_dump.txt
Jean-Marc Vanel
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
Followup GeoSparql + TDB 1 : freeze
For GeoSparql + TDB 1, I applied this initialization code on my site:
val res = configureLuceneIndex(dts, useTextQuery, useSpatialIndex)
if (useSpatialIndex) {
import org.apache.jena.geosparql.configuration._
GeoSPARQLConfig.setupMemoryIndex // actually registers special SPARQL
predicates!
GeoSPARQLConfig.setupSpatialIndex(res)
}
Full code:
https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/main/scala/deductions/runtime/jena/RDFStoreLocalJenaProvider.scala#L164
and I could indeed make "rectangular" queries on my geo: (
http://www.w3.org/2003/01/geo/wgs84_pos# ) data,
but even when not running any GeoSparql query,
the application web site got frozen, several times, so I was obliged to
deactivate GeoSparql initialization.
here is an example of a stack of a number of threads that run for a long
time and retain memory:
#132 daemon prio=5 os_prio=0 cpu=44.41ms elapsed=215.79s
tid=0x7f69e8012000 nid=0x2370 waiting on condition [0x7f68826d4000]
java.lang.Thread.State: WAITING (parking)
at jdk.internal.misc.Unsafe.park(java.base@11.0.10/Native Method)
- parking to wait for <0xbcd9d198> (a
java.util.concurrent.locks.ReentrantReadWriteLock$FairSync)
at java.util.concurrent.locks.LockSupport.park(java.base@11.0.10
/LockSupport.java:194)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(java.base@11.0.10
/AbstractQueuedSynchronizer.java:885)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(java.base@11.0.10
/AbstractQueuedSynchronizer.java:917)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(java.base@11.0.10
/AbstractQueuedSynchronizer.java:1240)
at
java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(java.base@11.0.10
/ReentrantReadWriteLock.java:959)
at
org.apache.jena.tdb.transaction.TransactionManager.startExclusiveMode(TransactionManager.java:706)
at
org.apache.jena.tdb.transaction.TransactionManager.exclusiveFlushQueue(TransactionManager.java:661)
at
org.apache.jena.tdb.transaction.TransactionManager.notifyCommit(TransactionManager.java:567)
at
org.apache.jena.tdb.transaction.Transaction.commit(Transaction.java:143)
at
org.apache.jena.tdb.transaction.DatasetGraphTxn.commit(DatasetGraphTxn.java:61)
at
org.apache.jena.tdb.transaction.DatasetGraphTransaction.commit(DatasetGraphTransaction.java:200)
at
org.apache.jena.sparql.core.DatasetImpl.commit(DatasetImpl.java:152)
at
org.w3.banana.jena.JenaDatasetStore.$anonfun$rw$1(JenaDatasetStore.scala:26)
at
org.w3.banana.jena.JenaDatasetStore$$Lambda$1102/0x000840a8b840.apply(Unknown
Source)
at scala.util.Try$.apply(Try.scala:213)
at org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:22)
at org.w3.banana.jena.JenaDatasetStore.rw(JenaDatasetStore.scala:10)
at
deductions.runtime.sparql_cache.SPARQLHelpers.wrapInTransaction(SPARQLHelpers.scala:237)
at
deductions.runtime.sparql_cache.SPARQLHelpers.wrapInTransaction$(SPARQLHelpers.scala:235)
at
controllers.WebPagesApp$$anon$1.wrapInTransaction(WebPages.scala:51)
at
deductions.runtime.abstract_syntax.InstanceLabelsInferenceMemory.$anonfun$makeInstanceLabelFuture$1(InstanceLabelsInferenceMemory.scala:49)
This transaction just adds one literal triple to TDB.
There are several such stacks present, seen when executing kill -3 when the
web application is frozen:
http://jmvanel.free.fr/tmp/thread_dump.txt
Jean-Marc Vanel
+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
Chroniques jardin
