date:20200325

Re: TDB blocked

2020-03-25 Thread Andy Seaborne





On 25/03/2020 14:36, Jean-Marc Vanel wrote:

Le mer. 25 mars 2020 à 13:07, Andy Seaborne  a écrit :


What's the full stack trace?
(What's the blocking thread?)



See below.


I was looking for the other thread that is blocking this one at 
being(write).  Same JVM.


There is no across different JVM locking in TDB.





Yes, another unended W transaction in the same process might be the cause.



I thought rather, another unended W transaction in *another* process, since
the tests execution was parallel.


If I understand correctly,

parallelExecution in Test := false

means parallel in the same JVM.

Andy




Test code:
https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/test/scala/deductions/runtime/sparql_cache/TestRDFCache.scala
=
Thread [pool-1-thread-1-ScalaTest-running-TestRDFCache] (Suspended)
Unsafe.park(boolean, long) line: not available [native method]
LockSupport.park(Object) line: 175
Semaphore$FairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt()
line: 836
Semaphore$FairSync(AbstractQueuedSynchronizer).doAcquireSharedInterruptibly(int)
line: 997
Semaphore$FairSync(AbstractQueuedSynchronizer).acquireSharedInterruptibly(int)
line: 1304
Semaphore.acquire() line: 312
TransactionManager.acquireWriterLock(boolean) line: 616
TransactionManager.beginInternal(TxnType, TxnType, String) line: 358
TransactionManager.begin(TxnType, String) line: 343
StoreConnection.begin(TxnType, String) line: 128
StoreConnection.begin(TxnType) line: 108
DatasetGraphTransaction.begin(TxnType) line: 169
DatasetGraphTransaction.begin(ReadWrite) line: 162
DatasetImpl.begin(ReadWrite) line: 122
JenaDatasetStore.$anonfun$rw$1(Dataset, Function0) line: 23
52027729.apply() line: not available
Try$.apply(Function0) line: 213
JenaDatasetStore.rw(Dataset, Function0) line: 22
JenaDatasetStore.rw(Object, Function0) line: 10
TestRDFCache(RDFCacheAlgo).readStoreURI(Object, Object,
DATASET) line: 368
RDFCacheAlgo.readStoreURI$(RDFCacheAlgo, Object, Object,
Object) line: 366
TestRDFCache.readStoreURI(Object, Object, Object) line: 10
TestRDFCache(RDFCacheAlgo).readStoreURIinOwnGraph(Object)
line: 334
TestRDFCache(RDFCacheAlgo).readStoreUriInNamedGraph(Object)
line: 326
RDFCacheAlgo.readStoreUriInNamedGraph$(RDFCacheAlgo, Object)
line: 325
TestRDFCache.readStoreUriInNamedGraph(Object) line: 10
TestRDFCache.$anonfun$new$4(TestRDFCache) line: 63
1708361043.apply$mcV$sp() line: not available
1708361043(JFunction0$mcV$sp).apply() line: 23
OutcomeOf$(OutcomeOf).outcomeOf(Function0) line: 85
OutcomeOf.outcomeOf$(OutcomeOf, Function0) line: 83
OutcomeOf$.outcomeOf(Function0) line: 104
Transformer.apply() line: 22
Transformer.apply() line: 20
AnyFunSuiteLike$$anon$1.apply() line: 189
TestRDFCache(TestSuite).withFixture(TestSuite$NoArgTest) line: 196
TestSuite.withFixture$(TestSuite, TestSuite$NoArgTest) line: 195
TestRDFCache(AnyFunSuite).withFixture(TestSuite$NoArgTest) line: 1562
TestRDFCache(AnyFunSuiteLike).invokeWithFixture$1(SuperEngine$TestLeaf,
Args, String) line: 187
AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike, Args, String,
SuperEngine$TestLeaf) line: 199
1593480726.apply(Object) line: not available
Engine(SuperEngine).runTestImpl(Suite, String, Args, boolean,
Function1,Outcome>) line: 306
TestRDFCache(AnyFunSuiteLike).runTest(String, Args) line: 199
AnyFunSuiteLike.runTest$(AnyFunSuiteLike, String, Args) line: 181
TestRDFCache(AnyFunSuite).runTest(String, Args) line: 1562
AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike, String, Args) line:
232
438507753.apply(Object, Object) line: not available
SuperEngine.$anonfun$runTestsInBranch$1(SuperEngine, Args, Suite,
SuperEngine$Branch, ListBuffer, Function2, boolean, SuperEngine$Node) line:
413
855709148.apply(Object) line: not available
$colon$colon(List).foreach(Function1) line: 392
Engine(SuperEngine).traverseSubNodes$1(SuperEngine$Branch, Args, Suite,
ListBuffer, Function2, boolean) line: 401
Engine(SuperEngine).runTestsInBranch(Suite, SuperEngine, Args,
boolean, Function2) line: 396
Engine(SuperEngine).runTestsImpl(Suite, Option, Args, Informer,
boolean, Function2) line: 475
TestRDFCache(AnyFunSuiteLike).runTests(Option, Args) line: 232
AnyFunSuiteLike.runTests$(AnyFunSuiteLike, Option, Args) line: 231
TestRDFCache(AnyFunSuite).runTests(Option, Args) line: 1562
TestRDFCache(Suite).run(Option, Args) line: 1112
Suite.run$(Suite, Option, Args) line: 1094
TestRDFCache(AnyFunSuite).org$scalatest$funsuite$AnyFunSuiteLike$$super$run(Option,
Args) line: 1562
AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike, Option, Args) line: 236
113821125.apply(Object, Object) line: not available
Engine(SuperEngine).runImpl(Suite, Option, Args,
Function2,Args,Status>) line: 535
TestRDFCache(AnyFunSuiteLike).run(Option, Args) line: 236
AnyFunSuiteLike.run$(AnyFunSuiteLike, Option, Args) line: 235
TestRDFCache.org$scalatest$BeforeAndAfterAll$$super$run(Option, Args) line:
10
TestRDFCache(BeforeAndAfterAll).liftedTree1$1(Args, Option) line: 213

Re: TDB blocked

2020-03-25 Thread Jean-Marc Vanel

Le mer. 25 mars 2020 à 13:07, Andy Seaborne  a écrit :

> What's the full stack trace?
> (What's the blocking thread?)
>

See below.


> Yes, another unended W transaction in the same process might be the cause.
>

I thought rather, another unended W transaction in *another* process, since
the tests execution was parallel.

Test code:
https://github.com/jmvanel/semantic_forms/blob/master/scala/forms/src/test/scala/deductions/runtime/sparql_cache/TestRDFCache.scala
=
Thread [pool-1-thread-1-ScalaTest-running-TestRDFCache] (Suspended)
Unsafe.park(boolean, long) line: not available [native method]
LockSupport.park(Object) line: 175
Semaphore$FairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt()
line: 836
Semaphore$FairSync(AbstractQueuedSynchronizer).doAcquireSharedInterruptibly(int)
line: 997
Semaphore$FairSync(AbstractQueuedSynchronizer).acquireSharedInterruptibly(int)
line: 1304
Semaphore.acquire() line: 312
TransactionManager.acquireWriterLock(boolean) line: 616
TransactionManager.beginInternal(TxnType, TxnType, String) line: 358
TransactionManager.begin(TxnType, String) line: 343
StoreConnection.begin(TxnType, String) line: 128
StoreConnection.begin(TxnType) line: 108
DatasetGraphTransaction.begin(TxnType) line: 169
DatasetGraphTransaction.begin(ReadWrite) line: 162
DatasetImpl.begin(ReadWrite) line: 122
JenaDatasetStore.$anonfun$rw$1(Dataset, Function0) line: 23
52027729.apply() line: not available
Try$.apply(Function0) line: 213
JenaDatasetStore.rw(Dataset, Function0) line: 22
JenaDatasetStore.rw(Object, Function0) line: 10
TestRDFCache(RDFCacheAlgo).readStoreURI(Object, Object,
DATASET) line: 368
RDFCacheAlgo.readStoreURI$(RDFCacheAlgo, Object, Object,
Object) line: 366
TestRDFCache.readStoreURI(Object, Object, Object) line: 10
TestRDFCache(RDFCacheAlgo).readStoreURIinOwnGraph(Object)
line: 334
TestRDFCache(RDFCacheAlgo).readStoreUriInNamedGraph(Object)
line: 326
RDFCacheAlgo.readStoreUriInNamedGraph$(RDFCacheAlgo, Object)
line: 325
TestRDFCache.readStoreUriInNamedGraph(Object) line: 10
TestRDFCache.$anonfun$new$4(TestRDFCache) line: 63
1708361043.apply$mcV$sp() line: not available
1708361043(JFunction0$mcV$sp).apply() line: 23
OutcomeOf$(OutcomeOf).outcomeOf(Function0) line: 85
OutcomeOf.outcomeOf$(OutcomeOf, Function0) line: 83
OutcomeOf$.outcomeOf(Function0) line: 104
Transformer.apply() line: 22
Transformer.apply() line: 20
AnyFunSuiteLike$$anon$1.apply() line: 189
TestRDFCache(TestSuite).withFixture(TestSuite$NoArgTest) line: 196
TestSuite.withFixture$(TestSuite, TestSuite$NoArgTest) line: 195
TestRDFCache(AnyFunSuite).withFixture(TestSuite$NoArgTest) line: 1562
TestRDFCache(AnyFunSuiteLike).invokeWithFixture$1(SuperEngine$TestLeaf,
Args, String) line: 187
AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike, Args, String,
SuperEngine$TestLeaf) line: 199
1593480726.apply(Object) line: not available
Engine(SuperEngine).runTestImpl(Suite, String, Args, boolean,
Function1,Outcome>) line: 306
TestRDFCache(AnyFunSuiteLike).runTest(String, Args) line: 199
AnyFunSuiteLike.runTest$(AnyFunSuiteLike, String, Args) line: 181
TestRDFCache(AnyFunSuite).runTest(String, Args) line: 1562
AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike, String, Args) line:
232
438507753.apply(Object, Object) line: not available
SuperEngine.$anonfun$runTestsInBranch$1(SuperEngine, Args, Suite,
SuperEngine$Branch, ListBuffer, Function2, boolean, SuperEngine$Node) line:
413
855709148.apply(Object) line: not available
$colon$colon(List).foreach(Function1) line: 392
Engine(SuperEngine).traverseSubNodes$1(SuperEngine$Branch, Args, Suite,
ListBuffer, Function2, boolean) line: 401
Engine(SuperEngine).runTestsInBranch(Suite, SuperEngine, Args,
boolean, Function2) line: 396
Engine(SuperEngine).runTestsImpl(Suite, Option, Args, Informer,
boolean, Function2) line: 475
TestRDFCache(AnyFunSuiteLike).runTests(Option, Args) line: 232
AnyFunSuiteLike.runTests$(AnyFunSuiteLike, Option, Args) line: 231
TestRDFCache(AnyFunSuite).runTests(Option, Args) line: 1562
TestRDFCache(Suite).run(Option, Args) line: 1112
Suite.run$(Suite, Option, Args) line: 1094
TestRDFCache(AnyFunSuite).org$scalatest$funsuite$AnyFunSuiteLike$$super$run(Option,
Args) line: 1562
AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike, Option, Args) line: 236
113821125.apply(Object, Object) line: not available
Engine(SuperEngine).runImpl(Suite, Option, Args,
Function2,Args,Status>) line: 535
TestRDFCache(AnyFunSuiteLike).run(Option, Args) line: 236
AnyFunSuiteLike.run$(AnyFunSuiteLike, Option, Args) line: 235
TestRDFCache.org$scalatest$BeforeAndAfterAll$$super$run(Option, Args) line:
10
TestRDFCache(BeforeAndAfterAll).liftedTree1$1(Args, Option) line: 213
TestRDFCache(BeforeAndAfterAll).run(Option, Args) line: 210
BeforeAndAfterAll.run$(BeforeAndAfterAll, Option, Args) line: 208
TestRDFCache.run(Option, Args) line: 10
Framework.org$scalatest$tools$Framework$$runSuite(TaskDef, String, Suite,
ClassLoader, SuiteSortingReporter, Tracker, EventHandler, Set,
Set, Selector[],

Re: TDB blocked

2020-03-25 Thread Andy Seaborne


What's the full stack trace?
(What's the blocking thread?)

Yes, another unended W transaction in the same process might be the cause.

Andy


On 25/03/2020 11:08, Jean-Marc Vanel wrote:

I investigated on that.
Now my test suite passes without Jena TDB trouble. I just added this in
Scala SBT :
parallelExecution in Test := false

I didn't know that parallel execution was the default.
So apparently this was an issue with 2 processes accessing the same TDB .
Normally this is detected and an explicit Jena TDB message appears, but not
this time.

Jean-Marc Vanel

+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
  Chroniques jardin



Le mar. 24 mars 2020 à 18:56, Jean-Marc Vanel  a
écrit :


Hi Jena users!

After changing my test suite in* semantic_forms
*  , I get
consistently a thread blocked forever. I upgraded to latest Jena Version =
"3.14.0" , and happily my Web application seems to be working fine on my
laptop, and the test suite sort of works, that is running tests one by one.
In my test suite I used to recursively delete the TDB directories after
each test. What changed is that instead I kept the same TDB directories for
each test, calling close() after each test.
This is with TDB 1. Is another unended transaction the issue? Then how to
discover that ?

Thread [pool-1-thread-1-ScalaTest-running-TestRDFCache] (Suspended)
Unsafe.park(boolean, long) line: not available [native method]
LockSupport.park(Object) line: 175
Semaphore$FairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt()
line: 836
Semaphore$FairSync(AbstractQueuedSynchronizer).doAcquireSharedInterruptibly(int)
line: 997
Semaphore$FairSync(AbstractQueuedSynchronizer).acquireSharedInterruptibly(int)
line: 1304
Semaphore.acquire() line: 312

* TransactionManager.acquireWriterLock(boolean) line: 616 *
TransactionManager.beginInternal(TxnType, TxnType, String) line: 358
TransactionManager.begin(TxnType, String) line: 343
StoreConnection.begin(TxnType, String) line: 128
StoreConnection.begin(TxnType) line: 108
DatasetGraphTransaction.begin(TxnType) line: 169
DatasetGraphTransaction.begin(ReadWrite) line: 162
DatasetImpl.begin(ReadWrite) line: 122
JenaDatasetStore.$anonfun$rw$1(Dataset, Function0) line: 23


Jean-Marc Vanel

+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
  Chroniques jardin

Re: TDB blocked

2020-03-25 Thread Jean-Marc Vanel

I investigated on that.
Now my test suite passes without Jena TDB trouble. I just added this in
Scala SBT :
parallelExecution in Test := false

I didn't know that parallel execution was the default.
So apparently this was an issue with 2 processes accessing the same TDB .
Normally this is detected and an explicit Jena TDB message appears, but not
this time.

Jean-Marc Vanel

+33 (0)6 89 16 29 52
Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
 Chroniques jardin



Le mar. 24 mars 2020 à 18:56, Jean-Marc Vanel  a
écrit :

> Hi Jena users!
>
> After changing my test suite in* semantic_forms
> *  , I get
> consistently a thread blocked forever. I upgraded to latest Jena Version =
> "3.14.0" , and happily my Web application seems to be working fine on my
> laptop, and the test suite sort of works, that is running tests one by one.
> In my test suite I used to recursively delete the TDB directories after
> each test. What changed is that instead I kept the same TDB directories for
> each test, calling close() after each test.
> This is with TDB 1. Is another unended transaction the issue? Then how to
> discover that ?
>
> Thread [pool-1-thread-1-ScalaTest-running-TestRDFCache] (Suspended)
> Unsafe.park(boolean, long) line: not available [native method]
> LockSupport.park(Object) line: 175
> Semaphore$FairSync(AbstractQueuedSynchronizer).parkAndCheckInterrupt()
> line: 836
> Semaphore$FairSync(AbstractQueuedSynchronizer).doAcquireSharedInterruptibly(int)
> line: 997
> Semaphore$FairSync(AbstractQueuedSynchronizer).acquireSharedInterruptibly(int)
> line: 1304
> Semaphore.acquire() line: 312
>
> * TransactionManager.acquireWriterLock(boolean) line: 616 *
> TransactionManager.beginInternal(TxnType, TxnType, String) line: 358
> TransactionManager.begin(TxnType, String) line: 343
> StoreConnection.begin(TxnType, String) line: 128
> StoreConnection.begin(TxnType) line: 108
> DatasetGraphTransaction.begin(TxnType) line: 169
> DatasetGraphTransaction.begin(ReadWrite) line: 162
> DatasetImpl.begin(ReadWrite) line: 122
> JenaDatasetStore.$anonfun$rw$1(Dataset, Function0) line: 23
>
>
> Jean-Marc Vanel
> 
> +33 (0)6 89 16 29 52
> Twitter: @jmvanel , @jmvanel_fr ; chat: irc://irc.freenode.net#eulergui
>  Chroniques jardin
> 
>

Re: Apache Jena Fuseki with text indexing

2020-03-25 Thread Andy Seaborne





On 24/03/2020 15:11, Zhenya Antić wrote:

Hi Andy,


Did you load the data before attaching the text index?


How do I do it (or not do it, wasn't sure from your post)?


Set up the Fueski system, with the text index as the Fuskei service dataset:

 fuseki:name "biology" ;
 fuseki:dataset :text_dataset ;
...

:text_dataset rdf:type text:TextDataset ;
  text:dataset <#dataset> ;



<#dataset> rdf:type tdb2:DatasetTDB2 ;
tdb2:location "db" ; #path to TDB;
.

then send the data to /biology/data (which is the SPARQl GSP write 
endpoint) or however you want to push the data to the server (SPARQL 
Update, or the UI.


For very large data:

Load the TDB2 dataset offline
Then run the "jena.textindexer" utility

https://jena.apache.org/documentation/query/text-query.html#configuration

The first way is easier.

Andy



Thanks,
Zhenya



On Sun, Mar 22, 2020, at 9:18 AM, Andy Seaborne wrote:

Just checking one point:

Did you load the data before attaching the text index?

The text index is calculated as data is added so if you first load the
dataset then setup a text index, it will miss indexing the data.

  Andy

On 21/03/2020 07:55, Lorenz Buehmann wrote:

Hi,

welcome to Semantic Web and Apache Jena.

Comments inline:

On 20.03.20 15:36, Zhenya Antić wrote:

Hello,

I am a beginner with Fuseki, knowledge graphs and SPARQL, so please forgive me 
if the questions seem obvious, the learning curve for this turned out to be 
quite steep.

No problem, nothing is simple in the beginning,


I am trying to get text indexing to work with my Fuseki knowledge graph.

Which DBpedia dataset did you load? I mean, which files?


For starters, I tried using a regular expression, but that didn't work:

Just a plain query like this:
SELECT DISTINCT * WHERE {
?s ?p ?o
}
gives 98 results such as:

1



2



3



4




That can't be the correct output of this query. rdfs:label should return
literals as object (?o) - or you loaded some really weird data


But a query with a regular expression:
SELECT DISTINCT * WHERE {
?s ?p ?o
FILTER regex(?o, "Biol", "i")
}


1. you should help the query engine and use rdfs:label as property

2. you should use str() function on the ?o values:

SELECT DISTINCT * WHERE {
?s rdfs:label ?o
FILTER regex(str(?o), "Biol", "i")
}


gives 0 results, although there are clearly results that contain "Biol".



I've to try your config or maybe others will spot the issue in the meantime.



I also tried setting up indexing with a .ttl file, however the result was "INFO 0 (0 
per second) properties indexed". .ttl file below:

@prefix :  .
@prefix tdb2:  .
@prefix rdf:  .
@prefix ja:  .
@prefix rdfs:  .
@prefix fuseki:  .
@prefix text:  .


rdfs:subClassOf ja:RDFDataset .

ja:DatasetTxnMem rdfs:subClassOf ja:RDFDataset .

tdb2:DatasetTDB2 rdfs:subClassOf ja:RDFDataset .

tdb2:GraphTDB2 rdfs:subClassOf ja:Model .


rdfs:subClassOf ja:Model .

ja:MemoryDataset rdfs:subClassOf ja:RDFDataset .

ja:RDFDatasetZero rdfs:subClassOf ja:RDFDataset .


The rdfs:subClassOf should not be necessary (recent versions of Fuseki).

If any are, let's use know so it can be fixed.




rdfs:subClassOf ja:RDFDataset .

:service_tdb_all a fuseki:Service ;
rdfs:label "TDB biology" ;
fuseki:dataset :tdb_dataset_readwrite ;
fuseki:name "biology" ;
fuseki:serviceQuery "query" , "" , "sparql" ;
fuseki:serviceReadGraphStore "get" ;
fuseki:serviceReadQuads "" ;
fuseki:serviceReadWriteGraphStore
"data" ;
fuseki:serviceReadWriteQuads "" ;
fuseki:serviceUpdate "" , "update" ;
fuseki:serviceUpload "upload" .

:tdb_dataset_readwrite
a tdb2:DatasetTDB2 ;
tdb2:location "db" .


rdfs:subClassOf ja:Model .

ja:RDFDatasetOne rdfs:subClassOf ja:RDFDataset .

ja:RDFDatasetSink rdfs:subClassOf ja:RDFDataset .


rdfs:subClassOf ja:RDFDataset .

<#dataset> rdf:type tdb2:DatasetTDB2 ;
tdb2:location "db" ; #path to TDB;
.

# Text index description
:text_dataset rdf:type text:TextDataset ;
text:dataset <#dataset> ; # <-- replace `:my_dataset` with

Re: TDB blocked

Re: TDB blocked

Re: TDB blocked

Re: TDB blocked

Re: Apache Jena Fuseki with text indexing

5 matches

Site Navigation

Mail list logo

Footer information