well spotted Adam, I think that nails it. I guess I was lucky with my setup to avoid this glitch so far. Don't want to anticipates Andy's call here but what's your plan? wrap the call into a transaction to fix it? If so I'd like to see it as an option in the code to work with tdb2 access methods if required in addition the current tdb support.
On Mon, Sep 17, 2018 at 3:26 PM ajs6f <[email protected]> wrote: > Okay, I think we've got it. Nice debugging, Markus! > > If you look at the line of the spatial indexer code that is throwing an > exception: > > > https://github.com/apache/jena/blob/master/jena-spatial/src/main/java/jena/spatialindexer.java#L110 > > indeed, it does not start a transaction against the dataset before > starting to read quads to index them. Now, in TDB1 this isn't inherently a > problem-- it's possible to work against a dataset without transactions. > (Although it's generally not a good idea at all.) But IIRC, TDB2 requires > transactions. If that's true (and Andy can verify that for us) then this is > a bug and we can fix it in a straightforward way. > > Andy, can you confirm that diagnosis? If so, I'll open a ticket and write > a PR to use a Txn method to fix this. > > ajs6f > > > On Sep 17, 2018, at 10:14 AM, Markus Neumann <[email protected]> > wrote: > > > > It seems I made a mess with absolute and relative paths all together. So > I created a minimal setup for testing: > > (I include the file-contents at the bottom) > > > > $ export JENA_HOME=/srv/linked_data_store/apache-jena-3.8.0 > > $ export PATH=$PATH:$JENA_HOME/bin > > $ tdb2.tdbloader > --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl > > 13:50:54 INFO loader :: Loader = LoaderPhased > > 13:50:54 INFO loader :: Data: > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl: > 13 mm-loc-mini.ttl 0.06s (Avg: 232) > > 13:50:54 INFO loader :: Data: > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl: 50 > mm-rdf.ttl 0.01s (Avg: 6,250) > > 13:50:55 INFO loader :: Finish - index SPO > > 13:50:55 INFO loader :: Start replay index SPO > > 13:50:55 INFO loader :: Index set: SPO => SPO->POS, > SPO->OSP > > 13:50:55 INFO loader :: Index set: SPO => SPO->POS, > SPO->OSP [63 items, 0.0 seconds] > > 13:50:55 INFO loader :: Finish - index POS > > 13:50:55 INFO loader :: Finish - index OSP > > $ java -cp > /srv/linked_data_store/fuseki-server/fuseki-server.jar:/srv/linked_data_store/apache-jena-3.8.0/lib/* > jena.spatialindexer > --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl > > SLF4J: Class path contains multiple SLF4J bindings. > > SLF4J: Found binding in > [jar:file:/srv/linked_data_store/fuseki-server/fuseki-server.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: Found binding in > [jar:file:/srv/linked_data_store/apache-jena-3.8.0/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] > > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > > SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > > org.apache.jena.dboe.transaction.txn.TransactionException: Not in a > transaction > > at > org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) > > at > org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) > > at > org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) > > at > org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) > > at > org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) > > at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) > > at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) > > at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) > > at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) > > at > org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) > > at > org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) > > at > org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) > > at > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > > at > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > > at jena.spatialindexer.exec(spatialindexer.java:110) > > at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) > > at jena.cmd.CmdMain.mainRun(CmdMain.java:58) > > at jena.cmd.CmdMain.mainRun(CmdMain.java:45) > > at jena.spatialindexer.main(spatialindexer.java:53) > > > > I can get rid of the SLF4J warnings: > > $ java -cp /srv/linked_data_store/fuseki-server/fuseki-server.jar > jena.spatialindexer > --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl > > org.apache.jena.dboe.transaction.txn.TransactionException: Not in a > transaction > > at > org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) > > at > org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) > > at > org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) > > at > org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) > > at > org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) > > at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) > > at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) > > at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) > > at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) > > at > org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) > > at > org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) > > at > org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) > > at > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > > at > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > > at jena.spatialindexer.exec(spatialindexer.java:110) > > at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) > > at jena.cmd.CmdMain.mainRun(CmdMain.java:58) > > at jena.cmd.CmdMain.mainRun(CmdMain.java:45) > > at jena.spatialindexer.main(spatialindexer.java:53) > > > > There is no other java instance running at the time when I run those > commands. > > All paths are fully qualified and double checked. The destination > directory and the Lucene directory were cleared just before. > > > > The server does start and serve the data without errors or warnings in > the log, but spatial queries do not work. > > > > Does anyone see a reason for the TransactionException? > > Thanks and best regards > > Markus > > > > Attachments: > > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl: > > @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . > > @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . > > @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > > @prefix owl: <http://www.w3.org/2002/07/owl#> . > > @prefix dc: <http://purl.org/dc/elements/1.1/> . > > @prefix dcterms: <http://purl.org/dc/terms/> . > > @prefix xsd: <http://www.w3.org/2001/XMLSchema#> > > @prefix mm: <http://rdf.meteomatics.com/mm/> . > > > > <http://rdf.meteomatics.com/mm/> > > dcterms:publisher <http://api.meteomatics.com/Imprint.html> ; > > dcterms:title "Meteomatics RDF Vocabulary"@en . > > > > mm:location a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "location" ; > > dcterms:description "Location of a measurement"@en . > > > > mm:elevation a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "elevation" ; > > dcterms:description "Elevation of a location"@en . > > > > mm:validdate a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "validdate" ; > > dcterms:description "Temporal information describing when a > Measurement is/was valid"@en . > > > > mm:value a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "value" ; > > dcterms:description "The (observed or predicted) value of a > Measurement"@en . > > > > mm:parameter a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "parameter" ; > > dcterms:description "what was measured"@en . > > > > mm:station_name a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "station_name" ; > > dcterms:description "Name of a station"@en . > > > > mm:abbreviation a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "abbreviation" ; > > dcterms:description "Abbreviation of a station name"@en . > > > > mm:didok_id a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "didok_id" ; > > dcterms:description "didok id of a station"@en . > > > > mm:unit_symbol a rdf:Property ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "unit symbol" ; > > dcterms:description "unit symbol of a parameter"@en . > > > > mm:Measurement a rdf:Class ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "Measurement" ; > > dcterms:description "Measurement of a specific weather parameter at > specific location at a specific date (both future or past)" . > > > > mm:Location a rdf:Class ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "Location" ; > > dcterms:description "A Point on the globe where we observe > weather"@en . > > > > mm:Parameter a rdf:Class ; > > rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > > rdfs:label "Parameter" ; > > dcterms:description "A parameter describing an observable aspect of > the weather"@en . > > > > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl > > @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . > > @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . > > @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . > > @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > > @prefix owl: <http://www.w3.org/2002/07/owl#> . > > @prefix dc: <http://purl.org/dc/elements/1.1/> . > > @prefix dcterms: <http://purl.org/dc/terms/> . > > @prefix xsd: <http://www.w3.org/2001/XMLSchema#> . > > @prefix mm: <http://rdf.meteomatics.com/mm/> . > > @prefix mm-measurements: <http://rdf.meteomatics.com/mm/measurements/> . > > @prefix mm-locations: <http://rdf.meteomatics.com/mm/locations/> . > > > > > > mm-locations:8500015 a mm:Location ; > > a geosparql:Geometry ; > > mm:didok_id 8500015 ; > > geo:lat 47.54259 ; > > geo:long 7.61574 ; > > mm:elevation 273 . > > > > > > mm-locations:8504238 a mm:Location ; > > a geosparql:Geometry ; > > geosparql:asWKT "POINT(6.86466645231 > 47.0840196485)"^^geosparql:wktLiteral ; > > mm:didok_id 8504238 ; > > geo:lat 47.08402 ; > > geo:long 6.86467 ; > > mm:elevation 1086 . > > > > /srv/linked_data_store/fuseki-server/run/configuration/mm.ttl > > @prefix : <http://base/#> . > > @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . > > @prefix tdb2: <http://jena.apache.org/2016/tdb#> . > > @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> . > > @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > > @prefix fuseki: <http://jena.apache.org/fuseki#> . > > @prefix spatial: <http://jena.apache.org/spatial#> . > > @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . > > @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . > > # Spatial > > #[] ja:loadClass "org.apache.jena.query.spatial.SpatialQuery" . > > #spatial:SpatialtDataset rdfs:subClassOf ja:RDFDataset . > > #spatial:SpatialIndexLucene rdfs:subClassOf spatial:SpatialIndex . > > > > :service_tdb_all a fuseki:Service ; > > rdfs:label "TDB2 mm" ; > > fuseki:dataset :spatial_dataset ; > > fuseki:name "mm" ; > > fuseki:serviceQuery "query" , "sparql" ; > > fuseki:serviceReadGraphStore "get" ; > > fuseki:serviceReadWriteGraphStore > > "data" ; > > fuseki:serviceUpdate "update" ; > > fuseki:serviceUpload "upload" . > > > > :spatial_dataset a spatial:SpatialDataset ; > > spatial:dataset :tdb_dataset_readwrite ; > > spatial:index <#indexLucene> ; > > . > > > > <#indexLucene> a spatial:SpatialIndexLucene ; > > spatial:directory <file:Lucene> ; > > #spatial:directory "mem" ; > > spatial:definition <#definition> ; > > . > > > > <#definition> a spatial:EntityDefinition ; > > spatial:entityField "uri" ; > > spatial:geoField "geo" ; > > # custom geo predicates for 1) Latitude/Longitude Format > > spatial:hasSpatialPredicatePairs ( > > [ spatial:latitude geo:lat ; spatial:longitude geo:long ] > > ) ; > > # custom geo predicates for 2) Well Known Text (WKT) Literal > > spatial:hasWKTPredicates (geosparql:asWKT) ; > > #spatial:spatialContextFactory > > # "com.spatial4j.core.context.jts.JtsSpatialContextFactory" > > # "org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory" > > . > > > > :tdb_dataset_readwrite > > a tdb2:DatasetTDB2 ; > > tdb2:location > "/srv/linked_data_store/fuseki-server/run/databases/mm" . > > > > > >> Am 17.09.2018 um 15:15 schrieb ajs6f <[email protected]>: > >> > >> I think you might have misunderstood the use of $JENA_HOME, or perhaps > the Java classpath. $JENA_HOME, which should generally be an absolute > filepath, allows the Jena scripts to find their libraries at runtime. > Internally, the scripts use $JENA_HOME to set up the classpaths with which > to start running Java code. But in this case, you're not using the scripts. > You're calling a Java class directly, which means you have to give it all > the libraries directly, using the classpath flag directly. For example, > your second attempt: > >> > >> $ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer > --desc=fuseki-server/run/configuration/mm.ttl > >> Error: Could not find or load main class > apache-jena-3.8.0.lib.commons-cli-1.4.jar > >> > >> may have failed because whatever directory you are in probably doesn't > happen to have a Jena distribution in it called "apache-jena-3.8.0". > Perhaps try: > >> > >> java -cp $JENA_HOME/lib/* jena.spatialindexer > --desc=fuseki-server/run/configuration/mm.ttl > >> > >> Your first example looks like it might be misusing the -cp flag [1]. > You probably don't want the *.jar, just *, like > >> > >> java -cp lib/* jena.spatialindexer > --desc=../fuseki-server/run/configuration/mm.ttl > >> > >> especially if you happen to be using Windows. [2] That might not be the > entire story, but you can at least try those invocations and see if you get > anything different/better. > >> > >> ajs6f > >> > >> [1] > https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html#A1100762 > < > https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html#A1100762 > > > >> [2] > https://stackoverflow.com/questions/219585/including-all-the-jars-in-a-directory-within-the-java-classpath > < > https://stackoverflow.com/questions/219585/including-all-the-jars-in-a-directory-within-the-java-classpath > > > >> > >> > >>> On Sep 17, 2018, at 7:07 AM, Markus Neumann <[email protected] > <mailto:[email protected]>> wrote: > >>> > >>> Ok, that's exactly what I did here: > >>> $ java -cp lib/*.jar jena.spatialindexer > --desc=../fuseki-server/run/configuration/mm.ttl > >>> Error: Could not find or load main class lib.commons-cli-1.4.jar > >>> > >>> Or from outside JENA_HOME: > >>> $ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer > --desc=fuseki-server/run/configuration/mm.ttl > >>> Error: Could not find or load main class > apache-jena-3.8.0.lib.commons-cli-1.4.jar > >>> > >>> Or did I missunderstand something? > >>> > >>>> Am 17.09.2018 um 12:02 schrieb Marco Neumann <[email protected] > >: > >>>> > >>>> don't worry it's a common problem and frequent source of confusion > not only > >>>> with jena but java projects. try to simply inlcude all the jars. > >>>> > >>>> I presume you run this on a linux box with OpenJDK 8. correct? > >>>> > >>>> now once you have set up jena just add the following > >>>> {replace_with_your_path}/apache-jena-3.8.0/lib/* > >>>> > >>>> > >>>> > >>>> > >>>> On Mon, Sep 17, 2018 at 10:56 AM Markus Neumann < > [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>>> > >>>> wrote: > >>>> > >>>>> This is embarrassing but I don't get it. > >>>>> > >>>>> I've configured $JENA_HOME and adjusted the PATH. > >>>>> > >>>>> Now what classpath should I include when invoking the indexer? > >>>>> What I've tried so far: > >>>>> > >>>>> $ cd $JENA_HOME > >>>>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer > >>>>> --desc=../fuseki-server/run/configuration/mm.ttl > >>>>> Error: Could not find or load main class jena.spatialindexer > >>>>> $ java -cp lib/*.jar jena.spatialindexer > >>>>> --desc=../fuseki-server/run/configuration/mm.ttl > >>>>> Error: Could not find or load main class lib.commons-cli-1.4.jar > >>>>> > >>>>> Thanks for your patience > >>>>> > >>>>>> Am 17.09.2018 um 11:38 schrieb Marco Neumann < > [email protected] <mailto:[email protected]>>: > >>>>>> > >>>>>> yes correct, this looks good. now include the classpath during > invocation > >>>>>> of the spatialindexer > >>>>>> > >>>>>> in addition it's always a good idea to properly configure jena > along your > >>>>>> java runtime environment settings to avoid jdk / lib conflicts > during > >>>>>> execution. > >>>>>> > >>>>>> https://jena.apache.org/documentation/tools/index.html < > https://jena.apache.org/documentation/tools/index.html> < > >>>>> https://jena.apache.org/documentation/tools/index.html < > https://jena.apache.org/documentation/tools/index.html> < > https://jena.apache.org/documentation/tools/index.html < > https://jena.apache.org/documentation/tools/index.html>>> > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Mon, Sep 17, 2018 at 10:28 AM Markus Neumann < > >>>>> [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>> <mailto: > [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>>>> > >>>>>> wrote: > >>>>>> > >>>>>>> 1.: what do you mean by indexer? > >>>>>>> Probably my biggest problem is, that I have no clue how to find > out, > >>>>> where > >>>>>>> the jena.spatialindexer could be located except for asking here. > >>>>>>> > >>>>>>> Here is the content of the apache jena I downloaded: > >>>>>>> $ ls -lh ../apache-jena-3.8.0/lib > >>>>>>> total 17M > >>>>>>> -rw-r--r-- 1 process process 162K Oct 26 2017 collection-0.7.jar > >>>>>>> -rw-r--r-- 1 process process 53K Oct 26 2017 commons-cli-1.4.jar > >>>>>>> -rw-r--r-- 1 process process 328K Mar 5 2018 > commons-codec-1.11.jar > >>>>>>> -rw-r--r-- 1 process process 576K Jun 25 09:59 > commons-compress-1.17.jar > >>>>>>> -rw-r--r-- 1 process process 41K Mar 5 2018 commons-csv-1.5.jar > >>>>>>> -rw-r--r-- 1 process process 210K Mar 5 2018 commons-io-2.6.jar > >>>>>>> -rw-r--r-- 1 process process 425K Oct 26 2017 > commons-lang3-3.4.jar > >>>>>>> -rw-r--r-- 1 process process 749K Jun 25 10:01 httpclient-4.5.5.jar > >>>>>>> -rw-r--r-- 1 process process 158K Jun 25 10:01 > >>>>> httpclient-cache-4.5.5.jar > >>>>>>> -rw-r--r-- 1 process process 318K Jun 25 10:01 httpcore-4.4.9.jar > >>>>>>> -rw-r--r-- 1 process process 65K Oct 26 2017 > >>>>>>> jackson-annotations-2.9.0.jar > >>>>>>> -rw-r--r-- 1 process process 315K Jun 25 10:01 > jackson-core-2.9.5.jar > >>>>>>> -rw-r--r-- 1 process process 1.3M Jun 25 10:01 > >>>>> jackson-databind-2.9.5.jar > >>>>>>> -rw-r--r-- 1 process process 17K Oct 26 2017 > jcl-over-slf4j-1.7.25.jar > >>>>>>> -rw-r--r-- 1 process process 2.9M Jun 25 11:03 jena-arq-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 211K Jun 25 11:01 jena-base-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 259K Jun 25 11:07 jena-cmds-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 1.7M Jun 25 11:02 jena-core-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 135K Jun 25 11:05 > jena-dboe-base-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 21K Jun 25 11:05 > jena-dboe-index-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 74K Jun 25 11:05 > >>>>>>> jena-dboe-transaction-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 95K Jun 25 11:06 > >>>>>>> jena-dboe-trans-data-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 135K Jun 25 11:01 jena-iri-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 49K Jun 25 11:04 > >>>>> jena-rdfconnection-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 2.8M Jun 25 11:01 > >>>>> jena-shaded-guava-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 264K Jun 25 11:06 jena-tdb2-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 512K Jun 25 11:04 jena-tdb-3.8.0.jar > >>>>>>> -rw-r--r-- 1 process process 1.6M Jun 25 10:01 > jsonld-java-0.12.0.jar > >>>>>>> -rw-r--r-- 1 process process 232K Mar 5 2018 libthrift-0.10.0.jar > >>>>>>> -rw-r--r-- 1 process process 479K Oct 26 2017 log4j-1.2.17.jar > >>>>>>> -rw-r--r-- 1 process process 41K Oct 26 2017 slf4j-api-1.7.25.jar > >>>>>>> -rw-r--r-- 1 process process 12K Oct 26 2017 > slf4j-log4j12-1.7.25.jar > >>>>>>> > >>>>>>> 2.: Turns out, I don't need the jts. after commenting out the > affected > >>>>>>> lines, the server still runs fine. (But still without spatial > index) > >>>>>>> > >>>>>>> Best Markus > >>>>>>> > >>>>>>>> Am 17.09.2018 um 11:14 schrieb Marco Neumann < > [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>> > >>>>>> : > >>>>>>>> > >>>>>>>> Ok let's take a look at this. > >>>>>>>> > >>>>>>>> 1. please include in the indexer in the classpath. It's part of > the > >>>>>>> current > >>>>>>>> jena distribution. > >>>>>>>> > >>>>>>>> 2. do you need jts? it's a package that allows you to index multi > point > >>>>>>>> geometries. here a sample query would be helpful. If single point > >>>>>>>> geometries are indexed and used in conjunction with MBR range > queries. > >>>>>>>> there is no need for jts. > >>>>>>>> > >>>>>>>> > >>>>>>>> On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann < > >>>>> [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>> > >>>>>>> <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>>>> > >>>>>>>> wrote: > >>>>>>>> > >>>>>>>>> Hi, > >>>>>>>>> (sorry for the long post) > >>>>>>>>> > >>>>>>>>> I've been struggling with the spatial index for a while now. > >>>>>>>>> I've read > >>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>> < > >>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>>> < > >>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>> < > >>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>>>> < > >>>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>> < > >>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>>> < > >>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>> < > >>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>>>>> and > >>>>>>>>> skipped through about every spatial related thread here on the > list. > >>>>>>>>> I managed to get an in-memory version up and running at some > point > >>>>>>>>> following this > >>>>>>>>> > >>>>>>>>> > >>>>>>> > >>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>> < > >>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>> > >>>>>>> < > >>>>>>> > >>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>> < > >>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>> > >>>>>>>> > >>>>>>>>> < > >>>>>>>>> > >>>>>>> > >>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>> < > >>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>> > >>>>>>> < > >>>>>>> > >>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>> < > >>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>> > >>>>>>>> > >>>>>>>>>> > >>>>>>>>> and this > >>>>>>>>> > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ < > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ < > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/>> > >>>>> < > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < > >>>>>>> > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < > >>>>>>>>> > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> > >>>>>>>>> > >>>>>>>>> But that was on a small dataset that I uploaded via the > web-interface > >>>>> of > >>>>>>>>> Fuseki. > >>>>>>>>> > >>>>>>>>> Now the story is a bit different. I'm building the database using > >>>>>>>>> tdbloader2 and would like to create the spatial index upfront. > >>>>>>>>> > >>>>>>>>> java -cp jena-spatial.jar jena.spatialindexer > --desc=assembler_file > >>>>>>>>> I could not figure out, how to get this line to work. I have the > >>>>>>> following > >>>>>>>>> possible scenarios: > >>>>>>>>> > >>>>>>>>> 1. I download the jena-spatial-3.8.0.jar and specify that in the > >>>>>>> classpath: > >>>>>>>>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer > >>>>>>>>> --desc=../fuseki-server/run/configuration/mm.ttl > >>>>>>>>> Error: Could not find or load main class jena.spatialindexer > >>>>>>>>> > >>>>>>>>> 2. I use the fuseki-server jar: > >>>>>>>>> $ java -cp fuseki-server.jar jena.spatialindexer > >>>>>>>>> --desc=./run/configuration/mm.ttl > >>>>>>>>> WARN Custom SpatialContextFactory lib is not ready in > >>>>>>>>> classpath:com/vividsolutions/jts/geom/CoordinateSequenceFactory > >>>>>>>>> org.apache.jena.dboe.transaction.txn.TransactionException: Not > in a > >>>>>>>>> transaction > >>>>>>>>> > >>>>>>>>> I managed to eliminate the warning about jts by including that > in the > >>>>>>>>> classpath, but the error remains the same (with full trace): > >>>>>>>>> java -cp fuseki-server.jar:jts-1.13.jar jena.spatialindexer > >>>>>>>>> --desc=./run/configuration/mm.ttl > >>>>>>>>> org.apache.jena.dboe.transaction.txn.TransactionException: Not > in a > >>>>>>>>> transaction > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) > >>>>>>>>> at > >>>>>>> org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > >>>>>>>>> at > >>>>>>>>> > >>>>>>> > >>>>> > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > >>>>>>>>> at jena.spatialindexer.exec(spatialindexer.java:110) > >>>>>>>>> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) > >>>>>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) > >>>>>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) > >>>>>>>>> at jena.spatialindexer.main(spatialindexer.java:53) > >>>>>>>>> > >>>>>>>>> My configuration is the following: > >>>>>>>>> @prefix : <http://base/#> . > >>>>>>>>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . > >>>>>>>>> @prefix tdb2: <http://jena.apache.org/2016/tdb#> . > >>>>>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> . > >>>>>>>>> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > >>>>>>>>> @prefix fuseki: <http://jena.apache.org/fuseki#> . > >>>>>>>>> @prefix spatial: <http://jena.apache.org/spatial#> . > >>>>>>>>> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . > >>>>>>>>> @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . > >>>>>>>>> # Spatial > >>>>>>>>> [] ja:loadClass "org.apache.jena.query.spatial.SpatialQuery" . > >>>>>>>>> spatial:SpatialtDataset rdfs:subClassOf ja:RDFDataset . > >>>>>>>>> spatial:SpatialIndexLucene rdfs:subClassOf spatial:SpatialIndex . > >>>>>>>>> > >>>>>>>>> :service_tdb_all a fuseki:Service ; > >>>>>>>>> rdfs:label "TDB2 mm" ; > >>>>>>>>> fuseki:dataset :spatial_dataset ; > >>>>>>>>> fuseki:name "mm" ; > >>>>>>>>> fuseki:serviceQuery "query" , "sparql" ; > >>>>>>>>> fuseki:serviceReadGraphStore "get" ; > >>>>>>>>> fuseki:serviceReadWriteGraphStore > >>>>>>>>> "data" ; > >>>>>>>>> fuseki:serviceUpdate "update" ; > >>>>>>>>> fuseki:serviceUpload "upload" . > >>>>>>>>> > >>>>>>>>> :spatial_dataset a spatial:SpatialDataset ; > >>>>>>>>> spatial:dataset :tdb_dataset_readwrite ; > >>>>>>>>> spatial:index <#indexLucene> ; > >>>>>>>>> . > >>>>>>>>> > >>>>>>>>> <#indexLucene> a spatial:SpatialIndexLucene ; > >>>>>>>>> spatial:directory <file:Lucene> ; > >>>>>>>>> #spatial:directory "mem" ; > >>>>>>>>> spatial:definition <#definition> ; > >>>>>>>>> . > >>>>>>>>> > >>>>>>>>> <#definition> a spatial:EntityDefinition ; > >>>>>>>>> spatial:entityField "uri" ; > >>>>>>>>> spatial:geoField "geo" ; > >>>>>>>>> # custom geo predicates for 1) Latitude/Longitude Format > >>>>>>>>> spatial:hasSpatialPredicatePairs ( > >>>>>>>>> [ spatial:latitude geo:lat ; spatial:longitude geo:long ] > >>>>>>>>> ) ; > >>>>>>>>> # custom geo predicates for 2) Well Known Text (WKT) Literal > >>>>>>>>> spatial:hasWKTPredicates (geosparql:asWKT) ; > >>>>>>>>> # custom SpatialContextFactory for 2) Well Known Text (WKT) > Literal > >>>>>>>>> spatial:spatialContextFactory > >>>>>>>>> # > "com.spatial4j.core.context.jts.JtsSpatialContextFactory" > >>>>>>>>> > >>>>> "org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory" > >>>>>>>>> . > >>>>>>>>> > >>>>>>>>> :tdb_dataset_readwrite > >>>>>>>>> a tdb2:DatasetTDB2 ; > >>>>>>>>> tdb2:location > >>>>>>>>> "/srv/linked_data_store/fuseki-server/run/databases/mm" . > >>>>>>>>> > >>>>>>>>> And I've changed the fuseki script to include the jts jar: > >>>>>>>>> FUSEKI_CLASSES=$FUSEKI_HOME/jts-1.13.jar > >>>>>>>>> > >>>>>>>>> I see no errors or warnings in the server log, when starting > with that > >>>>>>>>> configuration, the Lucene folder is created and populated with > >>>>>>> something: > >>>>>>>>> $ ll run/configuration/Lucene/ > >>>>>>>>> total 4 > >>>>>>>>> -rw-rw-r-- 1 process process 71 Sep 14 09:28 segments_1 > >>>>>>>>> -rw-rw-r-- 1 process process 0 Sep 14 09:28 write.lock > >>>>>>>>> > >>>>>>>>> But any spatial query does not return any results. > >>>>>>>>> Any help on how to tackle this would be highly appreciated. > >>>>>>>>> > >>>>>>>>> Best > >>>>>>>>> Markus Neumann > >>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> -- > >>>>>>>> > >>>>>>>> > >>>>>>>> --- > >>>>>>>> Marco Neumann > >>>>>>>> KONA > >>>>>>> > >>>>>>> > >>>>>> > >>>>>> -- > >>>>>> > >>>>>> > >>>>>> --- > >>>>>> Marco Neumann > >>>>>> KONA > >>>>> > >>>>> > >>>> > >>>> -- > >>>> > >>>> > >>>> --- > >>>> Marco Neumann > >>>> KONA > > > > > -- --- Marco Neumann KONA
