Re: [3.0.1] Upgrading to 3.8.0

[ajs6f]

For 

> 6. org.apache.jena.sparql.engine.http.Service.queryAuthUser/queryAuthPwd


in particular, https://jena.apache.org/documentation/query/http-auth.html 
details methods for all-Jena HTTP authentication. Let us know if that doesn't 
meet your use case.

ajs6f

> On Sep 17, 2018, at 6:04 PM, Martynas Jusevičius  
> wrote:
> 
> Hi,
> 
> now that SPIN API has done that, we are finally forced to do (a long
> overdue) upgrade from 3.0.1 to 3.8.0.
> 
> This brings up a few deprecated/removed methods that I would like to
> ask some help with:
> 1. ReaderRIOT.setErrorHandler()
> 2. ReaderRIOT.setParserProfile()
> 3. ParserProfile.setBaseURI()
> 4. RDFDataMgr.createReader()
> 5. RiotLib.profile(String baseIRI, boolean resolveIRIs, boolean
> checking, ErrorHandler handler)
> 6. org.apache.jena.sparql.engine.http.Service.queryAuthUser/queryAuthPwd
> 
> Could I get any pointers as to what these should be replaced with?
> 
> Thanks
> 
> Martynas
> atomgraph.com

[3.0.1] Upgrading to 3.8.0

[Martynas Jusevičius]

Hi,

now that SPIN API has done that, we are finally forced to do (a long
overdue) upgrade from 3.0.1 to 3.8.0.

This brings up a few deprecated/removed methods that I would like to
ask some help with:
1. ReaderRIOT.setErrorHandler()
2. ReaderRIOT.setParserProfile()
3. ParserProfile.setBaseURI()
4. RDFDataMgr.createReader()
5. RiotLib.profile(String baseIRI, boolean resolveIRIs, boolean
checking, ErrorHandler handler)
6. org.apache.jena.sparql.engine.http.Service.queryAuthUser/queryAuthPwd

Could I get any pointers as to what these should be replaced with?

Thanks

Martynas
atomgraph.com

Re: Building a spatial index with jena/fuseki

[ajs6f]

Okay, I think we've got it. Nice debugging, Markus!

If you look at the line of the spatial indexer code that is throwing an 
exception:

https://github.com/apache/jena/blob/master/jena-spatial/src/main/java/jena/spatialindexer.java#L110

indeed, it does not start a transaction against the dataset before starting to 
read quads to index them. Now, in TDB1 this isn't inherently a problem-- it's 
possible to work against a dataset without transactions. (Although it's 
generally not a good idea at all.) But IIRC, TDB2 requires transactions. If 
that's true (and Andy can verify that for us) then this is a bug and we can fix 
it in a straightforward way.

Andy, can you confirm that diagnosis? If so, I'll open a ticket and write a PR 
to use a Txn method to fix this.

ajs6f

> On Sep 17, 2018, at 10:14 AM, Markus Neumann  wrote:
> 
> It seems I made a mess with absolute and relative paths all together. So I 
> created a minimal setup for testing:
> (I include the file-contents at the bottom)
> 
> $ export JENA_HOME=/srv/linked_data_store/apache-jena-3.8.0
> $ export PATH=$PATH:$JENA_HOME/bin
> $ tdb2.tdbloader 
> --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl 
> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl 
> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl 
> 13:50:54 INFO  loader   :: Loader = LoaderPhased
> 13:50:54 INFO  loader   :: Data: 
> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl: 13 
> mm-loc-mini.ttl 0.06s (Avg: 232)
> 13:50:54 INFO  loader   :: Data: 
> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl: 50 
> mm-rdf.ttl 0.01s (Avg: 6,250)
> 13:50:55 INFO  loader   :: Finish - index SPO
> 13:50:55 INFO  loader   :: Start replay index SPO
> 13:50:55 INFO  loader   :: Index set:  SPO => SPO->POS, SPO->OSP
> 13:50:55 INFO  loader   :: Index set:  SPO => SPO->POS, SPO->OSP 
> [63 items, 0.0 seconds]
> 13:50:55 INFO  loader   :: Finish - index POS
> 13:50:55 INFO  loader   :: Finish - index OSP
> $ java -cp 
> /srv/linked_data_store/fuseki-server/fuseki-server.jar:/srv/linked_data_store/apache-jena-3.8.0/lib/*
>  jena.spatialindexer 
> --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/srv/linked_data_store/fuseki-server/fuseki-server.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/srv/linked_data_store/apache-jena-3.8.0/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an 
> explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a 
> transaction
>   at 
> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
>   at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
>   at 
> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
>   at 
> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
>   at 
> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
>   at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
>   at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
>   at 
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
>   at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
>   at 
> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104)
>   at 
> org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72)
>   at 
> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50)
>   at 
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
>   at 
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
>   at jena.spatialindexer.exec(spatialindexer.java:110)
>   at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
>   at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
>   at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
>   at jena.spatialindexer.main(spatialindexer.java:53)
> 
> I can get rid of the SLF4J warnings:
> $ java -cp /srv/linked_data_store/fuseki-server/fuseki-server.jar 
> jena.spatialindexer 
> --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a 
> transaction
>   at 
> 

Re: Building a spatial index with jena/fuseki

[Markus Neumann]

It seems I made a mess with absolute and relative paths all together. So I 
created a minimal setup for testing:
(I include the file-contents at the bottom)

$ export JENA_HOME=/srv/linked_data_store/apache-jena-3.8.0
$ export PATH=$PATH:$JENA_HOME/bin
$ tdb2.tdbloader 
--desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl 
/srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl 
/srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl 
13:50:54 INFO  loader   :: Loader = LoaderPhased
13:50:54 INFO  loader   :: Data: 
/srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl: 13 
mm-loc-mini.ttl 0.06s (Avg: 232)
13:50:54 INFO  loader   :: Data: 
/srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl: 50 
mm-rdf.ttl 0.01s (Avg: 6,250)
13:50:55 INFO  loader   :: Finish - index SPO
13:50:55 INFO  loader   :: Start replay index SPO
13:50:55 INFO  loader   :: Index set:  SPO => SPO->POS, SPO->OSP
13:50:55 INFO  loader   :: Index set:  SPO => SPO->POS, SPO->OSP 
[63 items, 0.0 seconds]
13:50:55 INFO  loader   :: Finish - index POS
13:50:55 INFO  loader   :: Finish - index OSP
$ java -cp 
/srv/linked_data_store/fuseki-server/fuseki-server.jar:/srv/linked_data_store/apache-jena-3.8.0/lib/*
 jena.spatialindexer 
--desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in 
[jar:file:/srv/linked_data_store/fuseki-server/fuseki-server.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in 
[jar:file:/srv/linked_data_store/apache-jena-3.8.0/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
org.apache.jena.dboe.transaction.txn.TransactionException: Not in a transaction
at 
org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
at 
org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
at 
org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
at 
org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
at 
org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
at 
org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104)
at 
org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72)
at 
org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50)
at 
org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
at 
org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
at jena.spatialindexer.exec(spatialindexer.java:110)
at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
at jena.spatialindexer.main(spatialindexer.java:53)

I can get rid of the SLF4J warnings:
$ java -cp /srv/linked_data_store/fuseki-server/fuseki-server.jar 
jena.spatialindexer 
--desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl
org.apache.jena.dboe.transaction.txn.TransactionException: Not in a transaction
at 
org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
at 
org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
at 
org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
at 
org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
at 
org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
at 

Re: Building a spatial index with jena/fuseki

[ajs6f]

I think you might have misunderstood the use of $JENA_HOME, or perhaps the Java 
classpath. $JENA_HOME, which should generally be an absolute filepath, allows 
the Jena scripts to find their libraries at runtime. Internally, the scripts 
use $JENA_HOME to set up the classpaths with which to start running Java code. 
But in this case, you're not using the scripts. You're calling a Java class 
directly, which means you have to give it all the libraries directly, using the 
classpath flag directly. For example, your second attempt:

$ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer 
--desc=fuseki-server/run/configuration/mm.ttl
Error: Could not find or load main class 
apache-jena-3.8.0.lib.commons-cli-1.4.jar

may have failed because whatever directory you are in probably doesn't happen 
to have a Jena distribution in it called "apache-jena-3.8.0". Perhaps try:

java -cp $JENA_HOME/lib/* jena.spatialindexer 
--desc=fuseki-server/run/configuration/mm.ttl

Your first example looks like it might be misusing the -cp flag [1]. You 
probably don't want the *.jar, just *, like

java -cp lib/* jena.spatialindexer 
--desc=../fuseki-server/run/configuration/mm.ttl

especially if you happen to be using Windows. [2] That might not be the entire 
story, but you can at least try those invocations and see if you get anything 
different/better.

ajs6f

[1] 
https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html#A1100762
[2] 
https://stackoverflow.com/questions/219585/including-all-the-jars-in-a-directory-within-the-java-classpath


> On Sep 17, 2018, at 7:07 AM, Markus Neumann  wrote:
> 
> Ok, that's exactly what I did here:
> $ java -cp lib/*.jar jena.spatialindexer 
> --desc=../fuseki-server/run/configuration/mm.ttl
> Error: Could not find or load main class lib.commons-cli-1.4.jar
> 
> Or from outside JENA_HOME:
> $ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer 
> --desc=fuseki-server/run/configuration/mm.ttl
> Error: Could not find or load main class 
> apache-jena-3.8.0.lib.commons-cli-1.4.jar
> 
> Or did I missunderstand something?
> 
>> Am 17.09.2018 um 12:02 schrieb Marco Neumann :
>> 
>> don't worry it's a common problem and frequent source of confusion not only
>> with jena but java projects. try to simply inlcude all the jars.
>> 
>> I presume you run this on a linux box with OpenJDK 8. correct?
>> 
>> now once you have set up jena just add the following
>> {replace_with_your_path}/apache-jena-3.8.0/lib/*
>> 
>> 
>> 
>> 
>> On Mon, Sep 17, 2018 at 10:56 AM Markus Neumann > >
>> wrote:
>> 
>>> This is embarrassing but I don't get it.
>>> 
>>> I've configured $JENA_HOME and adjusted the PATH.
>>> 
>>> Now what classpath should I include when invoking the indexer?
>>> What I've tried so far:
>>> 
>>> $ cd $JENA_HOME
>>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer
>>> --desc=../fuseki-server/run/configuration/mm.ttl
>>> Error: Could not find or load main class jena.spatialindexer
>>> $ java -cp lib/*.jar jena.spatialindexer
>>> --desc=../fuseki-server/run/configuration/mm.ttl
>>> Error: Could not find or load main class lib.commons-cli-1.4.jar
>>> 
>>> Thanks for your patience
>>> 
 Am 17.09.2018 um 11:38 schrieb Marco Neumann :
 
 yes correct, this looks good. now include the classpath during invocation
 of the spatialindexer
 
 in addition it's always a good idea to properly configure jena along your
 java runtime environment settings to avoid jdk / lib conflicts during
 execution.
 
 https://jena.apache.org/documentation/tools/index.html <
>>> https://jena.apache.org/documentation/tools/index.html 
>>> >
 
 
 
 On Mon, Sep 17, 2018 at 10:28 AM Markus Neumann <
>>> mneum...@meteomatics.com  
>>> >>
 wrote:
 
> 1.: what do you mean by indexer?
> Probably my biggest problem is, that I have no clue how to find out,
>>> where
> the jena.spatialindexer could be located except for asking here.
> 
> Here is the content of the apache jena I downloaded:
> $ ls -lh ../apache-jena-3.8.0/lib
> total 17M
> -rw-r--r-- 1 process process 162K Oct 26  2017 collection-0.7.jar
> -rw-r--r-- 1 process process  53K Oct 26  2017 commons-cli-1.4.jar
> -rw-r--r-- 1 process process 328K Mar  5  2018 commons-codec-1.11.jar
> -rw-r--r-- 1 process process 576K Jun 25 09:59 commons-compress-1.17.jar
> -rw-r--r-- 1 process process  41K Mar  5  2018 commons-csv-1.5.jar
> -rw-r--r-- 1 process process 210K Mar  5  2018 commons-io-2.6.jar
> -rw-r--r-- 1 process process 425K Oct 26  2017 commons-lang3-3.4.jar
> -rw-r--r-- 1 process process 749K Jun 25 10:01 httpclient-4.5.5.jar
> -rw-r--r-- 1 process process 158K Jun 25 10:01
>>> httpclient-cache-4.5.5.jar
> -rw-r--r-- 1 process process 318K 

Re: Building a spatial index with jena/fuseki

[Markus Neumann]

Ok, that's exactly what I did here:
$ java -cp lib/*.jar jena.spatialindexer 
--desc=../fuseki-server/run/configuration/mm.ttl
Error: Could not find or load main class lib.commons-cli-1.4.jar

Or from outside JENA_HOME:
$ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer 
--desc=fuseki-server/run/configuration/mm.ttl
Error: Could not find or load main class 
apache-jena-3.8.0.lib.commons-cli-1.4.jar

Or did I missunderstand something?

> Am 17.09.2018 um 12:02 schrieb Marco Neumann :
> 
> don't worry it's a common problem and frequent source of confusion not only
> with jena but java projects. try to simply inlcude all the jars.
> 
> I presume you run this on a linux box with OpenJDK 8. correct?
> 
> now once you have set up jena just add the following
> {replace_with_your_path}/apache-jena-3.8.0/lib/*
> 
> 
> 
> 
> On Mon, Sep 17, 2018 at 10:56 AM Markus Neumann  >
> wrote:
> 
>> This is embarrassing but I don't get it.
>> 
>> I've configured $JENA_HOME and adjusted the PATH.
>> 
>> Now what classpath should I include when invoking the indexer?
>> What I've tried so far:
>> 
>> $ cd $JENA_HOME
>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer
>> --desc=../fuseki-server/run/configuration/mm.ttl
>> Error: Could not find or load main class jena.spatialindexer
>> $ java -cp lib/*.jar jena.spatialindexer
>> --desc=../fuseki-server/run/configuration/mm.ttl
>> Error: Could not find or load main class lib.commons-cli-1.4.jar
>> 
>> Thanks for your patience
>> 
>>> Am 17.09.2018 um 11:38 schrieb Marco Neumann :
>>> 
>>> yes correct, this looks good. now include the classpath during invocation
>>> of the spatialindexer
>>> 
>>> in addition it's always a good idea to properly configure jena along your
>>> java runtime environment settings to avoid jdk / lib conflicts during
>>> execution.
>>> 
>>> https://jena.apache.org/documentation/tools/index.html <
>> https://jena.apache.org/documentation/tools/index.html 
>> >
>>> 
>>> 
>>> 
>>> On Mon, Sep 17, 2018 at 10:28 AM Markus Neumann <
>> mneum...@meteomatics.com  
>> >>
>>> wrote:
>>> 
 1.: what do you mean by indexer?
 Probably my biggest problem is, that I have no clue how to find out,
>> where
 the jena.spatialindexer could be located except for asking here.
 
 Here is the content of the apache jena I downloaded:
 $ ls -lh ../apache-jena-3.8.0/lib
 total 17M
 -rw-r--r-- 1 process process 162K Oct 26  2017 collection-0.7.jar
 -rw-r--r-- 1 process process  53K Oct 26  2017 commons-cli-1.4.jar
 -rw-r--r-- 1 process process 328K Mar  5  2018 commons-codec-1.11.jar
 -rw-r--r-- 1 process process 576K Jun 25 09:59 commons-compress-1.17.jar
 -rw-r--r-- 1 process process  41K Mar  5  2018 commons-csv-1.5.jar
 -rw-r--r-- 1 process process 210K Mar  5  2018 commons-io-2.6.jar
 -rw-r--r-- 1 process process 425K Oct 26  2017 commons-lang3-3.4.jar
 -rw-r--r-- 1 process process 749K Jun 25 10:01 httpclient-4.5.5.jar
 -rw-r--r-- 1 process process 158K Jun 25 10:01
>> httpclient-cache-4.5.5.jar
 -rw-r--r-- 1 process process 318K Jun 25 10:01 httpcore-4.4.9.jar
 -rw-r--r-- 1 process process  65K Oct 26  2017
 jackson-annotations-2.9.0.jar
 -rw-r--r-- 1 process process 315K Jun 25 10:01 jackson-core-2.9.5.jar
 -rw-r--r-- 1 process process 1.3M Jun 25 10:01
>> jackson-databind-2.9.5.jar
 -rw-r--r-- 1 process process  17K Oct 26  2017 jcl-over-slf4j-1.7.25.jar
 -rw-r--r-- 1 process process 2.9M Jun 25 11:03 jena-arq-3.8.0.jar
 -rw-r--r-- 1 process process 211K Jun 25 11:01 jena-base-3.8.0.jar
 -rw-r--r-- 1 process process 259K Jun 25 11:07 jena-cmds-3.8.0.jar
 -rw-r--r-- 1 process process 1.7M Jun 25 11:02 jena-core-3.8.0.jar
 -rw-r--r-- 1 process process 135K Jun 25 11:05 jena-dboe-base-3.8.0.jar
 -rw-r--r-- 1 process process  21K Jun 25 11:05 jena-dboe-index-3.8.0.jar
 -rw-r--r-- 1 process process  74K Jun 25 11:05
 jena-dboe-transaction-3.8.0.jar
 -rw-r--r-- 1 process process  95K Jun 25 11:06
 jena-dboe-trans-data-3.8.0.jar
 -rw-r--r-- 1 process process 135K Jun 25 11:01 jena-iri-3.8.0.jar
 -rw-r--r-- 1 process process  49K Jun 25 11:04
>> jena-rdfconnection-3.8.0.jar
 -rw-r--r-- 1 process process 2.8M Jun 25 11:01
>> jena-shaded-guava-3.8.0.jar
 -rw-r--r-- 1 process process 264K Jun 25 11:06 jena-tdb2-3.8.0.jar
 -rw-r--r-- 1 process process 512K Jun 25 11:04 jena-tdb-3.8.0.jar
 -rw-r--r-- 1 process process 1.6M Jun 25 10:01 jsonld-java-0.12.0.jar
 -rw-r--r-- 1 process process 232K Mar  5  2018 libthrift-0.10.0.jar
 -rw-r--r-- 1 process process 479K Oct 26  2017 log4j-1.2.17.jar
 -rw-r--r-- 1 process process  41K Oct 26  2017 slf4j-api-1.7.25.jar
 -rw-r--r-- 1 process process  12K Oct 26  2017 slf4j-log4j12-1.7.25.jar

Re: Building a spatial index with jena/fuseki

[Marco Neumann]

don't worry it's a common problem and frequent source of confusion not only
with jena but java projects. try to simply inlcude all the jars.

I presume you run this on a linux box with OpenJDK 8. correct?

now once you have set up jena just add the following
{replace_with_your_path}/apache-jena-3.8.0/lib/*




On Mon, Sep 17, 2018 at 10:56 AM Markus Neumann 
wrote:

> This is embarrassing but I don't get it.
>
> I've configured $JENA_HOME and adjusted the PATH.
>
> Now what classpath should I include when invoking the indexer?
> What I've tried so far:
>
> $ cd $JENA_HOME
> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer
> --desc=../fuseki-server/run/configuration/mm.ttl
> Error: Could not find or load main class jena.spatialindexer
> $ java -cp lib/*.jar jena.spatialindexer
> --desc=../fuseki-server/run/configuration/mm.ttl
> Error: Could not find or load main class lib.commons-cli-1.4.jar
>
> Thanks for your patience
>
> > Am 17.09.2018 um 11:38 schrieb Marco Neumann :
> >
> > yes correct, this looks good. now include the classpath during invocation
> > of the spatialindexer
> >
> > in addition it's always a good idea to properly configure jena along your
> > java runtime environment settings to avoid jdk / lib conflicts during
> > execution.
> >
> > https://jena.apache.org/documentation/tools/index.html <
> https://jena.apache.org/documentation/tools/index.html>
> >
> >
> >
> > On Mon, Sep 17, 2018 at 10:28 AM Markus Neumann <
> mneum...@meteomatics.com >
> > wrote:
> >
> >> 1.: what do you mean by indexer?
> >> Probably my biggest problem is, that I have no clue how to find out,
> where
> >> the jena.spatialindexer could be located except for asking here.
> >>
> >> Here is the content of the apache jena I downloaded:
> >> $ ls -lh ../apache-jena-3.8.0/lib
> >> total 17M
> >> -rw-r--r-- 1 process process 162K Oct 26  2017 collection-0.7.jar
> >> -rw-r--r-- 1 process process  53K Oct 26  2017 commons-cli-1.4.jar
> >> -rw-r--r-- 1 process process 328K Mar  5  2018 commons-codec-1.11.jar
> >> -rw-r--r-- 1 process process 576K Jun 25 09:59 commons-compress-1.17.jar
> >> -rw-r--r-- 1 process process  41K Mar  5  2018 commons-csv-1.5.jar
> >> -rw-r--r-- 1 process process 210K Mar  5  2018 commons-io-2.6.jar
> >> -rw-r--r-- 1 process process 425K Oct 26  2017 commons-lang3-3.4.jar
> >> -rw-r--r-- 1 process process 749K Jun 25 10:01 httpclient-4.5.5.jar
> >> -rw-r--r-- 1 process process 158K Jun 25 10:01
> httpclient-cache-4.5.5.jar
> >> -rw-r--r-- 1 process process 318K Jun 25 10:01 httpcore-4.4.9.jar
> >> -rw-r--r-- 1 process process  65K Oct 26  2017
> >> jackson-annotations-2.9.0.jar
> >> -rw-r--r-- 1 process process 315K Jun 25 10:01 jackson-core-2.9.5.jar
> >> -rw-r--r-- 1 process process 1.3M Jun 25 10:01
> jackson-databind-2.9.5.jar
> >> -rw-r--r-- 1 process process  17K Oct 26  2017 jcl-over-slf4j-1.7.25.jar
> >> -rw-r--r-- 1 process process 2.9M Jun 25 11:03 jena-arq-3.8.0.jar
> >> -rw-r--r-- 1 process process 211K Jun 25 11:01 jena-base-3.8.0.jar
> >> -rw-r--r-- 1 process process 259K Jun 25 11:07 jena-cmds-3.8.0.jar
> >> -rw-r--r-- 1 process process 1.7M Jun 25 11:02 jena-core-3.8.0.jar
> >> -rw-r--r-- 1 process process 135K Jun 25 11:05 jena-dboe-base-3.8.0.jar
> >> -rw-r--r-- 1 process process  21K Jun 25 11:05 jena-dboe-index-3.8.0.jar
> >> -rw-r--r-- 1 process process  74K Jun 25 11:05
> >> jena-dboe-transaction-3.8.0.jar
> >> -rw-r--r-- 1 process process  95K Jun 25 11:06
> >> jena-dboe-trans-data-3.8.0.jar
> >> -rw-r--r-- 1 process process 135K Jun 25 11:01 jena-iri-3.8.0.jar
> >> -rw-r--r-- 1 process process  49K Jun 25 11:04
> jena-rdfconnection-3.8.0.jar
> >> -rw-r--r-- 1 process process 2.8M Jun 25 11:01
> jena-shaded-guava-3.8.0.jar
> >> -rw-r--r-- 1 process process 264K Jun 25 11:06 jena-tdb2-3.8.0.jar
> >> -rw-r--r-- 1 process process 512K Jun 25 11:04 jena-tdb-3.8.0.jar
> >> -rw-r--r-- 1 process process 1.6M Jun 25 10:01 jsonld-java-0.12.0.jar
> >> -rw-r--r-- 1 process process 232K Mar  5  2018 libthrift-0.10.0.jar
> >> -rw-r--r-- 1 process process 479K Oct 26  2017 log4j-1.2.17.jar
> >> -rw-r--r-- 1 process process  41K Oct 26  2017 slf4j-api-1.7.25.jar
> >> -rw-r--r-- 1 process process  12K Oct 26  2017 slf4j-log4j12-1.7.25.jar
> >>
> >> 2.: Turns out, I don't need the jts. after commenting out the affected
> >> lines, the server still runs fine. (But still without spatial index)
> >>
> >> Best Markus
> >>
> >>> Am 17.09.2018 um 11:14 schrieb Marco Neumann  >:
> >>>
> >>> Ok let's take a look at this.
> >>>
> >>> 1. please include in the indexer in the classpath. It's part of the
> >> current
> >>> jena distribution.
> >>>
> >>> 2. do you need jts? it's a package that allows you to index multi point
> >>> geometries. here a sample query would be helpful. If single point
> >>> geometries are indexed and used in conjunction with MBR range queries.
> >>> there is no need for jts.
> >>>
> >>>
> >>> On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann <
> 

Re: Building a spatial index with jena/fuseki

[Markus Neumann]

This is embarrassing but I don't get it.

I've configured $JENA_HOME and adjusted the PATH.

Now what classpath should I include when invoking the indexer?
What I've tried so far:

$ cd $JENA_HOME
$ java -cp jena-spatial-3.8.0.jar jena.spatialindexer 
--desc=../fuseki-server/run/configuration/mm.ttl
Error: Could not find or load main class jena.spatialindexer
$ java -cp lib/*.jar jena.spatialindexer 
--desc=../fuseki-server/run/configuration/mm.ttl
Error: Could not find or load main class lib.commons-cli-1.4.jar

Thanks for your patience

> Am 17.09.2018 um 11:38 schrieb Marco Neumann :
> 
> yes correct, this looks good. now include the classpath during invocation
> of the spatialindexer
> 
> in addition it's always a good idea to properly configure jena along your
> java runtime environment settings to avoid jdk / lib conflicts during
> execution.
> 
> https://jena.apache.org/documentation/tools/index.html 
> 
> 
> 
> 
> On Mon, Sep 17, 2018 at 10:28 AM Markus Neumann  >
> wrote:
> 
>> 1.: what do you mean by indexer?
>> Probably my biggest problem is, that I have no clue how to find out, where
>> the jena.spatialindexer could be located except for asking here.
>> 
>> Here is the content of the apache jena I downloaded:
>> $ ls -lh ../apache-jena-3.8.0/lib
>> total 17M
>> -rw-r--r-- 1 process process 162K Oct 26  2017 collection-0.7.jar
>> -rw-r--r-- 1 process process  53K Oct 26  2017 commons-cli-1.4.jar
>> -rw-r--r-- 1 process process 328K Mar  5  2018 commons-codec-1.11.jar
>> -rw-r--r-- 1 process process 576K Jun 25 09:59 commons-compress-1.17.jar
>> -rw-r--r-- 1 process process  41K Mar  5  2018 commons-csv-1.5.jar
>> -rw-r--r-- 1 process process 210K Mar  5  2018 commons-io-2.6.jar
>> -rw-r--r-- 1 process process 425K Oct 26  2017 commons-lang3-3.4.jar
>> -rw-r--r-- 1 process process 749K Jun 25 10:01 httpclient-4.5.5.jar
>> -rw-r--r-- 1 process process 158K Jun 25 10:01 httpclient-cache-4.5.5.jar
>> -rw-r--r-- 1 process process 318K Jun 25 10:01 httpcore-4.4.9.jar
>> -rw-r--r-- 1 process process  65K Oct 26  2017
>> jackson-annotations-2.9.0.jar
>> -rw-r--r-- 1 process process 315K Jun 25 10:01 jackson-core-2.9.5.jar
>> -rw-r--r-- 1 process process 1.3M Jun 25 10:01 jackson-databind-2.9.5.jar
>> -rw-r--r-- 1 process process  17K Oct 26  2017 jcl-over-slf4j-1.7.25.jar
>> -rw-r--r-- 1 process process 2.9M Jun 25 11:03 jena-arq-3.8.0.jar
>> -rw-r--r-- 1 process process 211K Jun 25 11:01 jena-base-3.8.0.jar
>> -rw-r--r-- 1 process process 259K Jun 25 11:07 jena-cmds-3.8.0.jar
>> -rw-r--r-- 1 process process 1.7M Jun 25 11:02 jena-core-3.8.0.jar
>> -rw-r--r-- 1 process process 135K Jun 25 11:05 jena-dboe-base-3.8.0.jar
>> -rw-r--r-- 1 process process  21K Jun 25 11:05 jena-dboe-index-3.8.0.jar
>> -rw-r--r-- 1 process process  74K Jun 25 11:05
>> jena-dboe-transaction-3.8.0.jar
>> -rw-r--r-- 1 process process  95K Jun 25 11:06
>> jena-dboe-trans-data-3.8.0.jar
>> -rw-r--r-- 1 process process 135K Jun 25 11:01 jena-iri-3.8.0.jar
>> -rw-r--r-- 1 process process  49K Jun 25 11:04 jena-rdfconnection-3.8.0.jar
>> -rw-r--r-- 1 process process 2.8M Jun 25 11:01 jena-shaded-guava-3.8.0.jar
>> -rw-r--r-- 1 process process 264K Jun 25 11:06 jena-tdb2-3.8.0.jar
>> -rw-r--r-- 1 process process 512K Jun 25 11:04 jena-tdb-3.8.0.jar
>> -rw-r--r-- 1 process process 1.6M Jun 25 10:01 jsonld-java-0.12.0.jar
>> -rw-r--r-- 1 process process 232K Mar  5  2018 libthrift-0.10.0.jar
>> -rw-r--r-- 1 process process 479K Oct 26  2017 log4j-1.2.17.jar
>> -rw-r--r-- 1 process process  41K Oct 26  2017 slf4j-api-1.7.25.jar
>> -rw-r--r-- 1 process process  12K Oct 26  2017 slf4j-log4j12-1.7.25.jar
>> 
>> 2.: Turns out, I don't need the jts. after commenting out the affected
>> lines, the server still runs fine. (But still without spatial index)
>> 
>> Best Markus
>> 
>>> Am 17.09.2018 um 11:14 schrieb Marco Neumann :
>>> 
>>> Ok let's take a look at this.
>>> 
>>> 1. please include in the indexer in the classpath. It's part of the
>> current
>>> jena distribution.
>>> 
>>> 2. do you need jts? it's a package that allows you to index multi point
>>> geometries. here a sample query would be helpful. If single point
>>> geometries are indexed and used in conjunction with MBR range queries.
>>> there is no need for jts.
>>> 
>>> 
>>> On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann > >>
>>> wrote:
>>> 
 Hi,
 (sorry for the long post)
 
 I've been struggling with the spatial index for a while now.
 I've read
>> https://jena.apache.org/documentation/query/spatial-query.html 
>>  <
>> https://jena.apache.org/documentation/query/spatial-query.html 
>> > <
 https://jena.apache.org/documentation/query/spatial-query.html 
 

Re: API in binary release?

[Andy Seaborne]

Using maven/gradle/... will make things easier in the long run.


If you do want to use the distribution, the source is available in the 
distribution and if you attach that, the javadoc should be available in 
the IDE (at least, it is in Eclipse).


It is also online:
http://jena.apache.org/documentation/javadoc/

Andy

On 17/09/18 08:38, Lorenz B. wrote:

Using Maven/Gradle would fetch sources and Javadoc automatically via
IDEs like Eclipse, IntelliJ, etc.

If you really want to import the JARs manually, you can get Javadoc from
Maven repo [1]. Indeed, you have to do it for each module, e.g. "core",
"arq", etc. in your classpath

[1] http://central.maven.org/maven2/org/apache/jena/jena-core/3.8.0/


The download page says the distribution includes the API, but I cannot
find a javadoc folder in the jars.  Is there some way to get this short
of standing up a source build myself?

(I'm putting the jars in an Eclipse plugin so Jena is conveniently
available to multiple other plugins, and it is nice for developers to
attach the javadoc API.)

Thanks.

Re: Building a spatial index with jena/fuseki

[Marco Neumann]

yes correct, this looks good. now include the classpath during invocation
of the spatialindexer

in addition it's always a good idea to properly configure jena along your
java runtime environment settings to avoid jdk / lib conflicts during
execution.

https://jena.apache.org/documentation/tools/index.html



On Mon, Sep 17, 2018 at 10:28 AM Markus Neumann 
wrote:

> 1.: what do you mean by indexer?
> Probably my biggest problem is, that I have no clue how to find out, where
> the jena.spatialindexer could be located except for asking here.
>
> Here is the content of the apache jena I downloaded:
> $ ls -lh ../apache-jena-3.8.0/lib
> total 17M
> -rw-r--r-- 1 process process 162K Oct 26  2017 collection-0.7.jar
> -rw-r--r-- 1 process process  53K Oct 26  2017 commons-cli-1.4.jar
> -rw-r--r-- 1 process process 328K Mar  5  2018 commons-codec-1.11.jar
> -rw-r--r-- 1 process process 576K Jun 25 09:59 commons-compress-1.17.jar
> -rw-r--r-- 1 process process  41K Mar  5  2018 commons-csv-1.5.jar
> -rw-r--r-- 1 process process 210K Mar  5  2018 commons-io-2.6.jar
> -rw-r--r-- 1 process process 425K Oct 26  2017 commons-lang3-3.4.jar
> -rw-r--r-- 1 process process 749K Jun 25 10:01 httpclient-4.5.5.jar
> -rw-r--r-- 1 process process 158K Jun 25 10:01 httpclient-cache-4.5.5.jar
> -rw-r--r-- 1 process process 318K Jun 25 10:01 httpcore-4.4.9.jar
> -rw-r--r-- 1 process process  65K Oct 26  2017
> jackson-annotations-2.9.0.jar
> -rw-r--r-- 1 process process 315K Jun 25 10:01 jackson-core-2.9.5.jar
> -rw-r--r-- 1 process process 1.3M Jun 25 10:01 jackson-databind-2.9.5.jar
> -rw-r--r-- 1 process process  17K Oct 26  2017 jcl-over-slf4j-1.7.25.jar
> -rw-r--r-- 1 process process 2.9M Jun 25 11:03 jena-arq-3.8.0.jar
> -rw-r--r-- 1 process process 211K Jun 25 11:01 jena-base-3.8.0.jar
> -rw-r--r-- 1 process process 259K Jun 25 11:07 jena-cmds-3.8.0.jar
> -rw-r--r-- 1 process process 1.7M Jun 25 11:02 jena-core-3.8.0.jar
> -rw-r--r-- 1 process process 135K Jun 25 11:05 jena-dboe-base-3.8.0.jar
> -rw-r--r-- 1 process process  21K Jun 25 11:05 jena-dboe-index-3.8.0.jar
> -rw-r--r-- 1 process process  74K Jun 25 11:05
> jena-dboe-transaction-3.8.0.jar
> -rw-r--r-- 1 process process  95K Jun 25 11:06
> jena-dboe-trans-data-3.8.0.jar
> -rw-r--r-- 1 process process 135K Jun 25 11:01 jena-iri-3.8.0.jar
> -rw-r--r-- 1 process process  49K Jun 25 11:04 jena-rdfconnection-3.8.0.jar
> -rw-r--r-- 1 process process 2.8M Jun 25 11:01 jena-shaded-guava-3.8.0.jar
> -rw-r--r-- 1 process process 264K Jun 25 11:06 jena-tdb2-3.8.0.jar
> -rw-r--r-- 1 process process 512K Jun 25 11:04 jena-tdb-3.8.0.jar
> -rw-r--r-- 1 process process 1.6M Jun 25 10:01 jsonld-java-0.12.0.jar
> -rw-r--r-- 1 process process 232K Mar  5  2018 libthrift-0.10.0.jar
> -rw-r--r-- 1 process process 479K Oct 26  2017 log4j-1.2.17.jar
> -rw-r--r-- 1 process process  41K Oct 26  2017 slf4j-api-1.7.25.jar
> -rw-r--r-- 1 process process  12K Oct 26  2017 slf4j-log4j12-1.7.25.jar
>
> 2.: Turns out, I don't need the jts. after commenting out the affected
> lines, the server still runs fine. (But still without spatial index)
>
> Best Markus
>
> > Am 17.09.2018 um 11:14 schrieb Marco Neumann :
> >
> > Ok let's take a look at this.
> >
> > 1. please include in the indexer in the classpath. It's part of the
> current
> > jena distribution.
> >
> > 2. do you need jts? it's a package that allows you to index multi point
> > geometries. here a sample query would be helpful. If single point
> > geometries are indexed and used in conjunction with MBR range queries.
> > there is no need for jts.
> >
> >
> > On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann  >
> > wrote:
> >
> >> Hi,
> >> (sorry for the long post)
> >>
> >> I've been struggling with the spatial index for a while now.
> >> I've read
> https://jena.apache.org/documentation/query/spatial-query.html <
> https://jena.apache.org/documentation/query/spatial-query.html> <
> >> https://jena.apache.org/documentation/query/spatial-query.html <
> https://jena.apache.org/documentation/query/spatial-query.html>> and
> >> skipped through about every spatial related thread here on the list.
> >> I managed to get an in-memory version up and running at some point
> >> following this
> >>
> >>
> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
> <
> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
> >
> >> <
> >>
> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
> <
> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
> >
> >>>
> >> and this
> >> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ <
> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> <
> >> 

Re: Building a spatial index with jena/fuseki

[Markus Neumann]

Hi,

I tried that and it does not change anything in the behaviour.
Server starts fine but spatial queries don't work.

Best
Markus

> Am 17.09.2018 um 11:26 schrieb Øyvind Gjesdal :
> 
> Looking at our very similar config (based on the fuseki examples) only
> difference I see (except from the additional library) is that our config
> uses TDB, and your config uses TDB2. I believe tdbloader2 generates tdb1
> data ( I could be wrong), since I remember using it pre tdb2 release, so
> there might be a mismatch in your config and built database.
> 
> :tdb_dataset_readwrite
>a  tdb2:DatasetTDB2 ;
>tdb2:location
> "/srv/linked_data_store/fuseki-server/run/databases/mm"
> 
> Does using
> :tdb_dataset_readwrite
>a  tdb:DatasetTDB ;
>tdb:location
> "/srv/linked_data_store/fuseki-server/run/databases/mm"
> 
> in your config help?
> 
> Best regards,
> Øyvind
> 
> 
> Den man. 17. sep. 2018 kl. 11:14 skrev Marco Neumann <
> marco.neum...@gmail.com>:
> 
>> Ok let's take a look at this.
>> 
>> 1. please include in the indexer in the classpath. It's part of the current
>> jena distribution.
>> 
>> 2. do you need jts? it's a package that allows you to index multi point
>> geometries. here a sample query would be helpful. If single point
>> geometries are indexed and used in conjunction with MBR range queries.
>> there is no need for jts.
>> 
>> 
>> On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann 
>> wrote:
>> 
>>> Hi,
>>> (sorry for the long post)
>>> 
>>> I've been struggling with the spatial index for a while now.
>>> I've read https://jena.apache.org/documentation/query/spatial-query.html
>> <
>>> https://jena.apache.org/documentation/query/spatial-query.html> and
>>> skipped through about every spatial related thread here on the list.
>>> I managed to get an in-memory version up and running at some point
>>> following this
>>> 
>>> 
>> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
>>> <
>>> 
>> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
 
>>> and this
>>> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ <
>>> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/>
>>> 
>>> But that was on a small dataset that I uploaded via the web-interface of
>>> Fuseki.
>>> 
>>> Now the story is a bit different. I'm building the database using
>>> tdbloader2 and would like to create the spatial index upfront.
>>> 
>>> java -cp jena-spatial.jar jena.spatialindexer --desc=assembler_file
>>> I could not figure out, how to get this line to work. I have the
>> following
>>> possible scenarios:
>>> 
>>> 1. I download the jena-spatial-3.8.0.jar and specify that in the
>> classpath:
>>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer
>>> --desc=../fuseki-server/run/configuration/mm.ttl
>>> Error: Could not find or load main class jena.spatialindexer
>>> 
>>> 2. I use the fuseki-server jar:
>>> $ java -cp fuseki-server.jar jena.spatialindexer
>>> --desc=./run/configuration/mm.ttl
>>> WARN  Custom SpatialContextFactory lib is not ready in
>>> classpath:com/vividsolutions/jts/geom/CoordinateSequenceFactory
>>> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a
>>> transaction
>>> 
>>> I managed to eliminate the warning about jts by including that in the
>>> classpath, but the error remains the same (with full trace):
>>> java -cp fuseki-server.jar:jts-1.13.jar jena.spatialindexer
>>> --desc=./run/configuration/mm.ttl
>>> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a
>>> transaction
>>>at
>>> 
>> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
>>>at
>>> 
>> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
>>>at
>>> 
>> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
>>>at
>>> 
>> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
>>>at
>>> 
>> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
>>>at
>>> 
>> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
>>>at
>>> 
>> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
>>>at
>>> 
>> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
>>>at
>> org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
>>>at
>>> 
>> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104)
>>>at
>>> 
>> org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72)
>>>at
>>> 
>> 

Re: Building a spatial index with jena/fuseki

[Markus Neumann]

1.: what do you mean by indexer?
Probably my biggest problem is, that I have no clue how to find out, where the 
jena.spatialindexer could be located except for asking here.

Here is the content of the apache jena I downloaded:
$ ls -lh ../apache-jena-3.8.0/lib
total 17M
-rw-r--r-- 1 process process 162K Oct 26  2017 collection-0.7.jar
-rw-r--r-- 1 process process  53K Oct 26  2017 commons-cli-1.4.jar
-rw-r--r-- 1 process process 328K Mar  5  2018 commons-codec-1.11.jar
-rw-r--r-- 1 process process 576K Jun 25 09:59 commons-compress-1.17.jar
-rw-r--r-- 1 process process  41K Mar  5  2018 commons-csv-1.5.jar
-rw-r--r-- 1 process process 210K Mar  5  2018 commons-io-2.6.jar
-rw-r--r-- 1 process process 425K Oct 26  2017 commons-lang3-3.4.jar
-rw-r--r-- 1 process process 749K Jun 25 10:01 httpclient-4.5.5.jar
-rw-r--r-- 1 process process 158K Jun 25 10:01 httpclient-cache-4.5.5.jar
-rw-r--r-- 1 process process 318K Jun 25 10:01 httpcore-4.4.9.jar
-rw-r--r-- 1 process process  65K Oct 26  2017 jackson-annotations-2.9.0.jar
-rw-r--r-- 1 process process 315K Jun 25 10:01 jackson-core-2.9.5.jar
-rw-r--r-- 1 process process 1.3M Jun 25 10:01 jackson-databind-2.9.5.jar
-rw-r--r-- 1 process process  17K Oct 26  2017 jcl-over-slf4j-1.7.25.jar
-rw-r--r-- 1 process process 2.9M Jun 25 11:03 jena-arq-3.8.0.jar
-rw-r--r-- 1 process process 211K Jun 25 11:01 jena-base-3.8.0.jar
-rw-r--r-- 1 process process 259K Jun 25 11:07 jena-cmds-3.8.0.jar
-rw-r--r-- 1 process process 1.7M Jun 25 11:02 jena-core-3.8.0.jar
-rw-r--r-- 1 process process 135K Jun 25 11:05 jena-dboe-base-3.8.0.jar
-rw-r--r-- 1 process process  21K Jun 25 11:05 jena-dboe-index-3.8.0.jar
-rw-r--r-- 1 process process  74K Jun 25 11:05 jena-dboe-transaction-3.8.0.jar
-rw-r--r-- 1 process process  95K Jun 25 11:06 jena-dboe-trans-data-3.8.0.jar
-rw-r--r-- 1 process process 135K Jun 25 11:01 jena-iri-3.8.0.jar
-rw-r--r-- 1 process process  49K Jun 25 11:04 jena-rdfconnection-3.8.0.jar
-rw-r--r-- 1 process process 2.8M Jun 25 11:01 jena-shaded-guava-3.8.0.jar
-rw-r--r-- 1 process process 264K Jun 25 11:06 jena-tdb2-3.8.0.jar
-rw-r--r-- 1 process process 512K Jun 25 11:04 jena-tdb-3.8.0.jar
-rw-r--r-- 1 process process 1.6M Jun 25 10:01 jsonld-java-0.12.0.jar
-rw-r--r-- 1 process process 232K Mar  5  2018 libthrift-0.10.0.jar
-rw-r--r-- 1 process process 479K Oct 26  2017 log4j-1.2.17.jar
-rw-r--r-- 1 process process  41K Oct 26  2017 slf4j-api-1.7.25.jar
-rw-r--r-- 1 process process  12K Oct 26  2017 slf4j-log4j12-1.7.25.jar

2.: Turns out, I don't need the jts. after commenting out the affected lines, 
the server still runs fine. (But still without spatial index)

Best Markus

> Am 17.09.2018 um 11:14 schrieb Marco Neumann :
> 
> Ok let's take a look at this.
> 
> 1. please include in the indexer in the classpath. It's part of the current
> jena distribution.
> 
> 2. do you need jts? it's a package that allows you to index multi point
> geometries. here a sample query would be helpful. If single point
> geometries are indexed and used in conjunction with MBR range queries.
> there is no need for jts.
> 
> 
> On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann  >
> wrote:
> 
>> Hi,
>> (sorry for the long post)
>> 
>> I've been struggling with the spatial index for a while now.
>> I've read https://jena.apache.org/documentation/query/spatial-query.html 
>>  <
>> https://jena.apache.org/documentation/query/spatial-query.html 
>> > and
>> skipped through about every spatial related thread here on the list.
>> I managed to get an in-memory version up and running at some point
>> following this
>> 
>> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
>>  
>> 
>> <
>> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
>>  
>> 
>>> 
>> and this
>> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ 
>>  <
>> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/>
>> 
>> But that was on a small dataset that I uploaded via the web-interface of
>> Fuseki.
>> 
>> Now the story is a bit different. I'm building the database using
>> tdbloader2 and would like to create the spatial index upfront.
>> 
>> java -cp jena-spatial.jar jena.spatialindexer --desc=assembler_file
>> I could not figure out, how to get this line to work. I have the following
>> possible scenarios:
>> 
>> 1. I download the jena-spatial-3.8.0.jar and specify that in the classpath:
>> $ 

Re: Building a spatial index with jena/fuseki

[Øyvind Gjesdal]

Looking at our very similar config (based on the fuseki examples) only
difference I see (except from the additional library) is that our config
uses TDB, and your config uses TDB2. I believe tdbloader2 generates tdb1
data ( I could be wrong), since I remember using it pre tdb2 release, so
there might be a mismatch in your config and built database.

:tdb_dataset_readwrite
a  tdb2:DatasetTDB2 ;
tdb2:location
"/srv/linked_data_store/fuseki-server/run/databases/mm"

Does using
:tdb_dataset_readwrite
a  tdb:DatasetTDB ;
tdb:location
"/srv/linked_data_store/fuseki-server/run/databases/mm"

in your config help?

Best regards,
Øyvind


Den man. 17. sep. 2018 kl. 11:14 skrev Marco Neumann <
marco.neum...@gmail.com>:

> Ok let's take a look at this.
>
> 1. please include in the indexer in the classpath. It's part of the current
> jena distribution.
>
> 2. do you need jts? it's a package that allows you to index multi point
> geometries. here a sample query would be helpful. If single point
> geometries are indexed and used in conjunction with MBR range queries.
> there is no need for jts.
>
>
> On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann 
> wrote:
>
> > Hi,
> > (sorry for the long post)
> >
> > I've been struggling with the spatial index for a while now.
> > I've read https://jena.apache.org/documentation/query/spatial-query.html
> <
> > https://jena.apache.org/documentation/query/spatial-query.html> and
> > skipped through about every spatial related thread here on the list.
> > I managed to get an in-memory version up and running at some point
> > following this
> >
> >
> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
> > <
> >
> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
> > >
> > and this
> > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ <
> > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/>
> >
> > But that was on a small dataset that I uploaded via the web-interface of
> > Fuseki.
> >
> > Now the story is a bit different. I'm building the database using
> > tdbloader2 and would like to create the spatial index upfront.
> >
> > java -cp jena-spatial.jar jena.spatialindexer --desc=assembler_file
> > I could not figure out, how to get this line to work. I have the
> following
> > possible scenarios:
> >
> > 1. I download the jena-spatial-3.8.0.jar and specify that in the
> classpath:
> > $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer
> > --desc=../fuseki-server/run/configuration/mm.ttl
> > Error: Could not find or load main class jena.spatialindexer
> >
> > 2. I use the fuseki-server jar:
> > $ java -cp fuseki-server.jar jena.spatialindexer
> > --desc=./run/configuration/mm.ttl
> > WARN  Custom SpatialContextFactory lib is not ready in
> > classpath:com/vividsolutions/jts/geom/CoordinateSequenceFactory
> > org.apache.jena.dboe.transaction.txn.TransactionException: Not in a
> > transaction
> >
> > I managed to eliminate the warning about jts by including that in the
> > classpath, but the error remains the same (with full trace):
> > java -cp fuseki-server.jar:jts-1.13.jar jena.spatialindexer
> > --desc=./run/configuration/mm.ttl
> > org.apache.jena.dboe.transaction.txn.TransactionException: Not in a
> > transaction
> > at
> >
> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
> > at
> >
> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
> > at
> >
> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
> > at
> >
> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
> > at
> >
> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
> > at
> >
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
> > at
> >
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
> > at
> >
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
> > at
> org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
> > at
> >
> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104)
> > at
> >
> org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72)
> > at
> >
> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50)
> > at
> >
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
> > at
> >
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
> > at 

Re: Building a spatial index with jena/fuseki

[Lorenz B.]

> 1. I download the jena-spatial-3.8.0.jar and specify that in the classpath:
> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer 
> --desc=../fuseki-server/run/configuration/mm.ttl 
> Error: Could not find or load main class jena.spatialindexer
The documentation is a bit misleading I'd say. Just having
jena-spatial-3.8.0.jar in the classpath isn't enough and it fails -
indeed with a not helpful message - because other libs like jena-core,
jena-arq, etc are missing. Either you you add the Jena distribution libs
to the default classpath, or you just use the Fuseki server jar as you
already did

>
> 2. I use the fuseki-server jar:
> $ java -cp fuseki-server.jar jena.spatialindexer 
> --desc=./run/configuration/mm.ttl
> WARN  Custom SpatialContextFactory lib is not ready in 
> classpath:com/vividsolutions/jts/geom/CoordinateSequenceFactory
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a 
> transaction
Well, Andy knows better. Maybe, the docs are a bit outdated or something
is different when using it in combination with TDB2. Lets wait for a
correct answer.

-- 
Lorenz Bühmann
AKSW group, University of Leipzig
Group: http://aksw.org - semantic web research center

Re: Building a spatial index with jena/fuseki

[Marco Neumann]

Ok let's take a look at this.

1. please include in the indexer in the classpath. It's part of the current
jena distribution.

2. do you need jts? it's a package that allows you to index multi point
geometries. here a sample query would be helpful. If single point
geometries are indexed and used in conjunction with MBR range queries.
there is no need for jts.


On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann 
wrote:

> Hi,
> (sorry for the long post)
>
> I've been struggling with the spatial index for a while now.
> I've read https://jena.apache.org/documentation/query/spatial-query.html <
> https://jena.apache.org/documentation/query/spatial-query.html> and
> skipped through about every spatial related thread here on the list.
> I managed to get an in-memory version up and running at some point
> following this
>
> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
> <
> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
> >
> and this
> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ <
> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/>
>
> But that was on a small dataset that I uploaded via the web-interface of
> Fuseki.
>
> Now the story is a bit different. I'm building the database using
> tdbloader2 and would like to create the spatial index upfront.
>
> java -cp jena-spatial.jar jena.spatialindexer --desc=assembler_file
> I could not figure out, how to get this line to work. I have the following
> possible scenarios:
>
> 1. I download the jena-spatial-3.8.0.jar and specify that in the classpath:
> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer
> --desc=../fuseki-server/run/configuration/mm.ttl
> Error: Could not find or load main class jena.spatialindexer
>
> 2. I use the fuseki-server jar:
> $ java -cp fuseki-server.jar jena.spatialindexer
> --desc=./run/configuration/mm.ttl
> WARN  Custom SpatialContextFactory lib is not ready in
> classpath:com/vividsolutions/jts/geom/CoordinateSequenceFactory
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a
> transaction
>
> I managed to eliminate the warning about jts by including that in the
> classpath, but the error remains the same (with full trace):
> java -cp fuseki-server.jar:jts-1.13.jar jena.spatialindexer
> --desc=./run/configuration/mm.ttl
> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a
> transaction
> at
> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
> at
> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
> at
> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
> at
> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
> at
> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
> at
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
> at
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
> at
> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
> at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
> at
> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104)
> at
> org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72)
> at
> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50)
> at
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
> at
> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
> at jena.spatialindexer.exec(spatialindexer.java:110)
> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
> at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
> at jena.spatialindexer.main(spatialindexer.java:53)
>
> My configuration is the following:
> @prefix :   .
> @prefix rdf:    .
> @prefix tdb2:   .
> @prefix ja: .
> @prefix rdfs:   .
> @prefix fuseki:  .
> @prefix spatial:  .
> @prefix geo:  .
> @prefix geosparql:  .
> # Spatial
> [] ja:loadClass "org.apache.jena.query.spatial.SpatialQuery" .
> spatial:SpatialtDataset rdfs:subClassOf ja:RDFDataset .
> spatial:SpatialIndexLucene rdfs:subClassOf 

Building a spatial index with jena/fuseki

[Markus Neumann]

Hi, 
(sorry for the long post)

I've been struggling with the spatial index for a while now.
I've read https://jena.apache.org/documentation/query/spatial-query.html 
 and skipped 
through about every spatial related thread here on the list.
I managed to get an in-memory version up and running at some point following 
this
https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066
 

and this
https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ 


But that was on a small dataset that I uploaded via the web-interface of Fuseki.

Now the story is a bit different. I'm building the database using tdbloader2 
and would like to create the spatial index upfront.

java -cp jena-spatial.jar jena.spatialindexer --desc=assembler_file
I could not figure out, how to get this line to work. I have the following 
possible scenarios:

1. I download the jena-spatial-3.8.0.jar and specify that in the classpath:
$ java -cp jena-spatial-3.8.0.jar jena.spatialindexer 
--desc=../fuseki-server/run/configuration/mm.ttl 
Error: Could not find or load main class jena.spatialindexer

2. I use the fuseki-server jar:
$ java -cp fuseki-server.jar jena.spatialindexer 
--desc=./run/configuration/mm.ttl
WARN  Custom SpatialContextFactory lib is not ready in 
classpath:com/vividsolutions/jts/geom/CoordinateSequenceFactory
org.apache.jena.dboe.transaction.txn.TransactionException: Not in a transaction

I managed to eliminate the warning about jts by including that in the 
classpath, but the error remains the same (with full trace):
java -cp fuseki-server.jar:jts-1.13.jar jena.spatialindexer 
--desc=./run/configuration/mm.ttl
org.apache.jena.dboe.transaction.txn.TransactionException: Not in a transaction
at 
org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417)
at 
org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159)
at 
org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348)
at 
org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251)
at 
org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150)
at 
org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141)
at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64)
at 
org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104)
at 
org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72)
at 
org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50)
at 
org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
at 
org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152)
at jena.spatialindexer.exec(spatialindexer.java:110)
at jena.cmd.CmdMain.mainMethod(CmdMain.java:93)
at jena.cmd.CmdMain.mainRun(CmdMain.java:58)
at jena.cmd.CmdMain.mainRun(CmdMain.java:45)
at jena.spatialindexer.main(spatialindexer.java:53)

My configuration is the following:
@prefix :   .
@prefix rdf:    .
@prefix tdb2:   .
@prefix ja: .
@prefix rdfs:   .
@prefix fuseki:  .
@prefix spatial:  .
@prefix geo:  .
@prefix geosparql:  .
# Spatial
[] ja:loadClass "org.apache.jena.query.spatial.SpatialQuery" .
spatial:SpatialtDataset rdfs:subClassOf ja:RDFDataset .
spatial:SpatialIndexLucene rdfs:subClassOf spatial:SpatialIndex .

:service_tdb_all  a   fuseki:Service ;
rdfs:label"TDB2 mm" ;
fuseki:dataset:spatial_dataset ;
fuseki:name   "mm" ;
fuseki:serviceQuery   "query" , "sparql" ;
fuseki:serviceReadGraphStore  "get" ;
fuseki:serviceReadWriteGraphStore
"data" ;
fuseki:serviceUpdate  "update" ;
fuseki:serviceUpload  "upload" .

:spatial_dataset a spatial:SpatialDataset ;
spatial:dataset   

Re: API in binary release?

[Lorenz B.]

Using Maven/Gradle would fetch sources and Javadoc automatically via
IDEs like Eclipse, IntelliJ, etc.

If you really want to import the JARs manually, you can get Javadoc from
Maven repo [1]. Indeed, you have to do it for each module, e.g. "core",
"arq", etc. in your classpath

[1] http://central.maven.org/maven2/org/apache/jena/jena-core/3.8.0/

> The download page says the distribution includes the API, but I cannot
> find a javadoc folder in the jars.  Is there some way to get this short
> of standing up a source build myself?
>
> (I'm putting the jars in an Eclipse plugin so Jena is conveniently
> available to multiple other plugins, and it is nice for developers to
> attach the javadoc API.)
>
> Thanks.
>
>
>
-- 
Lorenz Bühmann
AKSW group, University of Leipzig
Group: http://aksw.org - semantic web research center