Marco-- I've got a PR at https://github.com/apache/jena/pull/471. Until we hear from Andy, could you perhaps try cloning that PR/branch and trying the new code out? If that is not a reasonable piece of work for you, no problem, but it would help greatly in validating that we've found and fixed the problem.
ajs6f > On Sep 18, 2018, at 4:51 AM, Marco Neumann <[email protected]> wrote: > > well spotted Adam, I think that nails it. I guess I was lucky with my setup > to avoid this glitch so far. Don't want to anticipates Andy's call here but > what's your plan? wrap the call into a transaction to fix it? If so I'd > like to see it as an option in the code to work with tdb2 access methods if > required in addition the current tdb support. > > > On Mon, Sep 17, 2018 at 3:26 PM ajs6f <[email protected]> wrote: > >> Okay, I think we've got it. Nice debugging, Markus! >> >> If you look at the line of the spatial indexer code that is throwing an >> exception: >> >> >> https://github.com/apache/jena/blob/master/jena-spatial/src/main/java/jena/spatialindexer.java#L110 >> >> indeed, it does not start a transaction against the dataset before >> starting to read quads to index them. Now, in TDB1 this isn't inherently a >> problem-- it's possible to work against a dataset without transactions. >> (Although it's generally not a good idea at all.) But IIRC, TDB2 requires >> transactions. If that's true (and Andy can verify that for us) then this is >> a bug and we can fix it in a straightforward way. >> >> Andy, can you confirm that diagnosis? If so, I'll open a ticket and write >> a PR to use a Txn method to fix this. >> >> ajs6f >> >>> On Sep 17, 2018, at 10:14 AM, Markus Neumann <[email protected]> >> wrote: >>> >>> It seems I made a mess with absolute and relative paths all together. So >> I created a minimal setup for testing: >>> (I include the file-contents at the bottom) >>> >>> $ export JENA_HOME=/srv/linked_data_store/apache-jena-3.8.0 >>> $ export PATH=$PATH:$JENA_HOME/bin >>> $ tdb2.tdbloader >> --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl >> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl >> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl >>> 13:50:54 INFO loader :: Loader = LoaderPhased >>> 13:50:54 INFO loader :: Data: >> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl: >> 13 mm-loc-mini.ttl 0.06s (Avg: 232) >>> 13:50:54 INFO loader :: Data: >> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl: 50 >> mm-rdf.ttl 0.01s (Avg: 6,250) >>> 13:50:55 INFO loader :: Finish - index SPO >>> 13:50:55 INFO loader :: Start replay index SPO >>> 13:50:55 INFO loader :: Index set: SPO => SPO->POS, >> SPO->OSP >>> 13:50:55 INFO loader :: Index set: SPO => SPO->POS, >> SPO->OSP [63 items, 0.0 seconds] >>> 13:50:55 INFO loader :: Finish - index POS >>> 13:50:55 INFO loader :: Finish - index OSP >>> $ java -cp >> /srv/linked_data_store/fuseki-server/fuseki-server.jar:/srv/linked_data_store/apache-jena-3.8.0/lib/* >> jena.spatialindexer >> --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl >>> SLF4J: Class path contains multiple SLF4J bindings. >>> SLF4J: Found binding in >> [jar:file:/srv/linked_data_store/fuseki-server/fuseki-server.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>> SLF4J: Found binding in >> [jar:file:/srv/linked_data_store/apache-jena-3.8.0/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] >>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an >> explanation. >>> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] >>> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a >> transaction >>> at >> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) >>> at >> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) >>> at >> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) >>> at >> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) >>> at >> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) >>> at >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) >>> at >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) >>> at >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) >>> at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) >>> at >> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) >>> at >> org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) >>> at >> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) >>> at >> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) >>> at >> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) >>> at jena.spatialindexer.exec(spatialindexer.java:110) >>> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) >>> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) >>> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) >>> at jena.spatialindexer.main(spatialindexer.java:53) >>> >>> I can get rid of the SLF4J warnings: >>> $ java -cp /srv/linked_data_store/fuseki-server/fuseki-server.jar >> jena.spatialindexer >> --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl >>> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a >> transaction >>> at >> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) >>> at >> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) >>> at >> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) >>> at >> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) >>> at >> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) >>> at >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) >>> at >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) >>> at >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) >>> at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) >>> at >> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) >>> at >> org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) >>> at >> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) >>> at >> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) >>> at >> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) >>> at jena.spatialindexer.exec(spatialindexer.java:110) >>> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) >>> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) >>> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) >>> at jena.spatialindexer.main(spatialindexer.java:53) >>> >>> There is no other java instance running at the time when I run those >> commands. >>> All paths are fully qualified and double checked. The destination >> directory and the Lucene directory were cleared just before. >>> >>> The server does start and serve the data without errors or warnings in >> the log, but spatial queries do not work. >>> >>> Does anyone see a reason for the TransactionException? >>> Thanks and best regards >>> Markus >>> >>> Attachments: >>> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl: >>> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . >>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . >>> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . >>> @prefix owl: <http://www.w3.org/2002/07/owl#> . >>> @prefix dc: <http://purl.org/dc/elements/1.1/> . >>> @prefix dcterms: <http://purl.org/dc/terms/> . >>> @prefix xsd: <http://www.w3.org/2001/XMLSchema#> >>> @prefix mm: <http://rdf.meteomatics.com/mm/> . >>> >>> <http://rdf.meteomatics.com/mm/> >>> dcterms:publisher <http://api.meteomatics.com/Imprint.html> ; >>> dcterms:title "Meteomatics RDF Vocabulary"@en . >>> >>> mm:location a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "location" ; >>> dcterms:description "Location of a measurement"@en . >>> >>> mm:elevation a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "elevation" ; >>> dcterms:description "Elevation of a location"@en . >>> >>> mm:validdate a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "validdate" ; >>> dcterms:description "Temporal information describing when a >> Measurement is/was valid"@en . >>> >>> mm:value a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "value" ; >>> dcterms:description "The (observed or predicted) value of a >> Measurement"@en . >>> >>> mm:parameter a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "parameter" ; >>> dcterms:description "what was measured"@en . >>> >>> mm:station_name a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "station_name" ; >>> dcterms:description "Name of a station"@en . >>> >>> mm:abbreviation a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "abbreviation" ; >>> dcterms:description "Abbreviation of a station name"@en . >>> >>> mm:didok_id a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "didok_id" ; >>> dcterms:description "didok id of a station"@en . >>> >>> mm:unit_symbol a rdf:Property ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "unit symbol" ; >>> dcterms:description "unit symbol of a parameter"@en . >>> >>> mm:Measurement a rdf:Class ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "Measurement" ; >>> dcterms:description "Measurement of a specific weather parameter at >> specific location at a specific date (both future or past)" . >>> >>> mm:Location a rdf:Class ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "Location" ; >>> dcterms:description "A Point on the globe where we observe >> weather"@en . >>> >>> mm:Parameter a rdf:Class ; >>> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; >>> rdfs:label "Parameter" ; >>> dcterms:description "A parameter describing an observable aspect of >> the weather"@en . >>> >>> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl >>> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . >>> @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . >>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . >>> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . >>> @prefix owl: <http://www.w3.org/2002/07/owl#> . >>> @prefix dc: <http://purl.org/dc/elements/1.1/> . >>> @prefix dcterms: <http://purl.org/dc/terms/> . >>> @prefix xsd: <http://www.w3.org/2001/XMLSchema#> . >>> @prefix mm: <http://rdf.meteomatics.com/mm/> . >>> @prefix mm-measurements: <http://rdf.meteomatics.com/mm/measurements/> . >>> @prefix mm-locations: <http://rdf.meteomatics.com/mm/locations/> . >>> >>> >>> mm-locations:8500015 a mm:Location ; >>> a geosparql:Geometry ; >>> mm:didok_id 8500015 ; >>> geo:lat 47.54259 ; >>> geo:long 7.61574 ; >>> mm:elevation 273 . >>> >>> >>> mm-locations:8504238 a mm:Location ; >>> a geosparql:Geometry ; >>> geosparql:asWKT "POINT(6.86466645231 >> 47.0840196485)"^^geosparql:wktLiteral ; >>> mm:didok_id 8504238 ; >>> geo:lat 47.08402 ; >>> geo:long 6.86467 ; >>> mm:elevation 1086 . >>> >>> /srv/linked_data_store/fuseki-server/run/configuration/mm.ttl >>> @prefix : <http://base/#> . >>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . >>> @prefix tdb2: <http://jena.apache.org/2016/tdb#> . >>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> . >>> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . >>> @prefix fuseki: <http://jena.apache.org/fuseki#> . >>> @prefix spatial: <http://jena.apache.org/spatial#> . >>> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . >>> @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . >>> # Spatial >>> #[] ja:loadClass "org.apache.jena.query.spatial.SpatialQuery" . >>> #spatial:SpatialtDataset rdfs:subClassOf ja:RDFDataset . >>> #spatial:SpatialIndexLucene rdfs:subClassOf spatial:SpatialIndex . >>> >>> :service_tdb_all a fuseki:Service ; >>> rdfs:label "TDB2 mm" ; >>> fuseki:dataset :spatial_dataset ; >>> fuseki:name "mm" ; >>> fuseki:serviceQuery "query" , "sparql" ; >>> fuseki:serviceReadGraphStore "get" ; >>> fuseki:serviceReadWriteGraphStore >>> "data" ; >>> fuseki:serviceUpdate "update" ; >>> fuseki:serviceUpload "upload" . >>> >>> :spatial_dataset a spatial:SpatialDataset ; >>> spatial:dataset :tdb_dataset_readwrite ; >>> spatial:index <#indexLucene> ; >>> . >>> >>> <#indexLucene> a spatial:SpatialIndexLucene ; >>> spatial:directory <file:Lucene> ; >>> #spatial:directory "mem" ; >>> spatial:definition <#definition> ; >>> . >>> >>> <#definition> a spatial:EntityDefinition ; >>> spatial:entityField "uri" ; >>> spatial:geoField "geo" ; >>> # custom geo predicates for 1) Latitude/Longitude Format >>> spatial:hasSpatialPredicatePairs ( >>> [ spatial:latitude geo:lat ; spatial:longitude geo:long ] >>> ) ; >>> # custom geo predicates for 2) Well Known Text (WKT) Literal >>> spatial:hasWKTPredicates (geosparql:asWKT) ; >>> #spatial:spatialContextFactory >>> # "com.spatial4j.core.context.jts.JtsSpatialContextFactory" >>> # "org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory" >>> . >>> >>> :tdb_dataset_readwrite >>> a tdb2:DatasetTDB2 ; >>> tdb2:location >> "/srv/linked_data_store/fuseki-server/run/databases/mm" . >>> >>> >>>> Am 17.09.2018 um 15:15 schrieb ajs6f <[email protected]>: >>>> >>>> I think you might have misunderstood the use of $JENA_HOME, or perhaps >> the Java classpath. $JENA_HOME, which should generally be an absolute >> filepath, allows the Jena scripts to find their libraries at runtime. >> Internally, the scripts use $JENA_HOME to set up the classpaths with which >> to start running Java code. But in this case, you're not using the scripts. >> You're calling a Java class directly, which means you have to give it all >> the libraries directly, using the classpath flag directly. For example, >> your second attempt: >>>> >>>> $ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer >> --desc=fuseki-server/run/configuration/mm.ttl >>>> Error: Could not find or load main class >> apache-jena-3.8.0.lib.commons-cli-1.4.jar >>>> >>>> may have failed because whatever directory you are in probably doesn't >> happen to have a Jena distribution in it called "apache-jena-3.8.0". >> Perhaps try: >>>> >>>> java -cp $JENA_HOME/lib/* jena.spatialindexer >> --desc=fuseki-server/run/configuration/mm.ttl >>>> >>>> Your first example looks like it might be misusing the -cp flag [1]. >> You probably don't want the *.jar, just *, like >>>> >>>> java -cp lib/* jena.spatialindexer >> --desc=../fuseki-server/run/configuration/mm.ttl >>>> >>>> especially if you happen to be using Windows. [2] That might not be the >> entire story, but you can at least try those invocations and see if you get >> anything different/better. >>>> >>>> ajs6f >>>> >>>> [1] >> https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html#A1100762 >> < >> https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html#A1100762 >>> >>>> [2] >> https://stackoverflow.com/questions/219585/including-all-the-jars-in-a-directory-within-the-java-classpath >> < >> https://stackoverflow.com/questions/219585/including-all-the-jars-in-a-directory-within-the-java-classpath >>> >>>> >>>> >>>>> On Sep 17, 2018, at 7:07 AM, Markus Neumann <[email protected] >> <mailto:[email protected]>> wrote: >>>>> >>>>> Ok, that's exactly what I did here: >>>>> $ java -cp lib/*.jar jena.spatialindexer >> --desc=../fuseki-server/run/configuration/mm.ttl >>>>> Error: Could not find or load main class lib.commons-cli-1.4.jar >>>>> >>>>> Or from outside JENA_HOME: >>>>> $ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer >> --desc=fuseki-server/run/configuration/mm.ttl >>>>> Error: Could not find or load main class >> apache-jena-3.8.0.lib.commons-cli-1.4.jar >>>>> >>>>> Or did I missunderstand something? >>>>> >>>>>> Am 17.09.2018 um 12:02 schrieb Marco Neumann <[email protected] >>> : >>>>>> >>>>>> don't worry it's a common problem and frequent source of confusion >> not only >>>>>> with jena but java projects. try to simply inlcude all the jars. >>>>>> >>>>>> I presume you run this on a linux box with OpenJDK 8. correct? >>>>>> >>>>>> now once you have set up jena just add the following >>>>>> {replace_with_your_path}/apache-jena-3.8.0/lib/* >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Sep 17, 2018 at 10:56 AM Markus Neumann < >> [email protected] <mailto:[email protected]> <mailto: >> [email protected] <mailto:[email protected]>>> >>>>>> wrote: >>>>>> >>>>>>> This is embarrassing but I don't get it. >>>>>>> >>>>>>> I've configured $JENA_HOME and adjusted the PATH. >>>>>>> >>>>>>> Now what classpath should I include when invoking the indexer? >>>>>>> What I've tried so far: >>>>>>> >>>>>>> $ cd $JENA_HOME >>>>>>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer >>>>>>> --desc=../fuseki-server/run/configuration/mm.ttl >>>>>>> Error: Could not find or load main class jena.spatialindexer >>>>>>> $ java -cp lib/*.jar jena.spatialindexer >>>>>>> --desc=../fuseki-server/run/configuration/mm.ttl >>>>>>> Error: Could not find or load main class lib.commons-cli-1.4.jar >>>>>>> >>>>>>> Thanks for your patience >>>>>>> >>>>>>>> Am 17.09.2018 um 11:38 schrieb Marco Neumann < >> [email protected] <mailto:[email protected]>>: >>>>>>>> >>>>>>>> yes correct, this looks good. now include the classpath during >> invocation >>>>>>>> of the spatialindexer >>>>>>>> >>>>>>>> in addition it's always a good idea to properly configure jena >> along your >>>>>>>> java runtime environment settings to avoid jdk / lib conflicts >> during >>>>>>>> execution. >>>>>>>> >>>>>>>> https://jena.apache.org/documentation/tools/index.html < >> https://jena.apache.org/documentation/tools/index.html> < >>>>>>> https://jena.apache.org/documentation/tools/index.html < >> https://jena.apache.org/documentation/tools/index.html> < >> https://jena.apache.org/documentation/tools/index.html < >> https://jena.apache.org/documentation/tools/index.html>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Mon, Sep 17, 2018 at 10:28 AM Markus Neumann < >>>>>>> [email protected] <mailto:[email protected]> <mailto: >> [email protected] <mailto:[email protected]>> <mailto: >> [email protected] <mailto:[email protected]> <mailto: >> [email protected] <mailto:[email protected]>>>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> 1.: what do you mean by indexer? >>>>>>>>> Probably my biggest problem is, that I have no clue how to find >> out, >>>>>>> where >>>>>>>>> the jena.spatialindexer could be located except for asking here. >>>>>>>>> >>>>>>>>> Here is the content of the apache jena I downloaded: >>>>>>>>> $ ls -lh ../apache-jena-3.8.0/lib >>>>>>>>> total 17M >>>>>>>>> -rw-r--r-- 1 process process 162K Oct 26 2017 collection-0.7.jar >>>>>>>>> -rw-r--r-- 1 process process 53K Oct 26 2017 commons-cli-1.4.jar >>>>>>>>> -rw-r--r-- 1 process process 328K Mar 5 2018 >> commons-codec-1.11.jar >>>>>>>>> -rw-r--r-- 1 process process 576K Jun 25 09:59 >> commons-compress-1.17.jar >>>>>>>>> -rw-r--r-- 1 process process 41K Mar 5 2018 commons-csv-1.5.jar >>>>>>>>> -rw-r--r-- 1 process process 210K Mar 5 2018 commons-io-2.6.jar >>>>>>>>> -rw-r--r-- 1 process process 425K Oct 26 2017 >> commons-lang3-3.4.jar >>>>>>>>> -rw-r--r-- 1 process process 749K Jun 25 10:01 httpclient-4.5.5.jar >>>>>>>>> -rw-r--r-- 1 process process 158K Jun 25 10:01 >>>>>>> httpclient-cache-4.5.5.jar >>>>>>>>> -rw-r--r-- 1 process process 318K Jun 25 10:01 httpcore-4.4.9.jar >>>>>>>>> -rw-r--r-- 1 process process 65K Oct 26 2017 >>>>>>>>> jackson-annotations-2.9.0.jar >>>>>>>>> -rw-r--r-- 1 process process 315K Jun 25 10:01 >> jackson-core-2.9.5.jar >>>>>>>>> -rw-r--r-- 1 process process 1.3M Jun 25 10:01 >>>>>>> jackson-databind-2.9.5.jar >>>>>>>>> -rw-r--r-- 1 process process 17K Oct 26 2017 >> jcl-over-slf4j-1.7.25.jar >>>>>>>>> -rw-r--r-- 1 process process 2.9M Jun 25 11:03 jena-arq-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 211K Jun 25 11:01 jena-base-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 259K Jun 25 11:07 jena-cmds-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 1.7M Jun 25 11:02 jena-core-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 135K Jun 25 11:05 >> jena-dboe-base-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 21K Jun 25 11:05 >> jena-dboe-index-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 74K Jun 25 11:05 >>>>>>>>> jena-dboe-transaction-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 95K Jun 25 11:06 >>>>>>>>> jena-dboe-trans-data-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 135K Jun 25 11:01 jena-iri-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 49K Jun 25 11:04 >>>>>>> jena-rdfconnection-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 2.8M Jun 25 11:01 >>>>>>> jena-shaded-guava-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 264K Jun 25 11:06 jena-tdb2-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 512K Jun 25 11:04 jena-tdb-3.8.0.jar >>>>>>>>> -rw-r--r-- 1 process process 1.6M Jun 25 10:01 >> jsonld-java-0.12.0.jar >>>>>>>>> -rw-r--r-- 1 process process 232K Mar 5 2018 libthrift-0.10.0.jar >>>>>>>>> -rw-r--r-- 1 process process 479K Oct 26 2017 log4j-1.2.17.jar >>>>>>>>> -rw-r--r-- 1 process process 41K Oct 26 2017 slf4j-api-1.7.25.jar >>>>>>>>> -rw-r--r-- 1 process process 12K Oct 26 2017 >> slf4j-log4j12-1.7.25.jar >>>>>>>>> >>>>>>>>> 2.: Turns out, I don't need the jts. after commenting out the >> affected >>>>>>>>> lines, the server still runs fine. (But still without spatial >> index) >>>>>>>>> >>>>>>>>> Best Markus >>>>>>>>> >>>>>>>>>> Am 17.09.2018 um 11:14 schrieb Marco Neumann < >> [email protected] <mailto:[email protected]> <mailto: >> [email protected] <mailto:[email protected]>> >>>>>>>> : >>>>>>>>>> >>>>>>>>>> Ok let's take a look at this. >>>>>>>>>> >>>>>>>>>> 1. please include in the indexer in the classpath. It's part of >> the >>>>>>>>> current >>>>>>>>>> jena distribution. >>>>>>>>>> >>>>>>>>>> 2. do you need jts? it's a package that allows you to index multi >> point >>>>>>>>>> geometries. here a sample query would be helpful. If single point >>>>>>>>>> geometries are indexed and used in conjunction with MBR range >> queries. >>>>>>>>>> there is no need for jts. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann < >>>>>>> [email protected] <mailto:[email protected]> <mailto: >> [email protected] <mailto:[email protected]>> >>>>>>>>> <mailto:[email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>> >> <mailto:[email protected] <mailto:[email protected]> >> <mailto:[email protected] <mailto:[email protected]>>>>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> (sorry for the long post) >>>>>>>>>>> >>>>>>>>>>> I've been struggling with the spatial index for a while now. >>>>>>>>>>> I've read >>>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html> < >> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html>> < >>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html> < >> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html>>> < >>>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html> < >> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html>> < >>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html> < >> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html>>>> < >>>>>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html> < >> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html>> < >>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html> < >> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html>>> < >>>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html> < >> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html>> < >>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html> < >> https://jena.apache.org/documentation/query/spatial-query.html < >> https://jena.apache.org/documentation/query/spatial-query.html>>>>> and >>>>>>>>>>> skipped through about every spatial related thread here on the >> list. >>>>>>>>>>> I managed to get an in-memory version up and running at some >> point >>>>>>>>>>> following this >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>> >>>>>>> >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>>> >>>>>>> < >>>>>>> >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>>> >>>>>>>> >>>>>>>>> < >>>>>>>>> >>>>>>> >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>>> >>>>>>> < >>>>>>> >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>>> >>>>>>>> >>>>>>>>>> >>>>>>>>>>> < >>>>>>>>>>> >>>>>>>>> >>>>>>> >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>>> >>>>>>> < >>>>>>> >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>>> >>>>>>>> >>>>>>>>> < >>>>>>>>> >>>>>>> >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>>> >>>>>>> < >>>>>>> >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >> < >> https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 >>>> >>>>>>>> >>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> and this >>>>>>>>>>> >> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ < >> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < >> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ < >> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/>> >>>>>>> < >> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < >>>>>>>>> >> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < >>>>>>>>>>> >> https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> >>>>>>>>>>> >>>>>>>>>>> But that was on a small dataset that I uploaded via the >> web-interface >>>>>>> of >>>>>>>>>>> Fuseki. >>>>>>>>>>> >>>>>>>>>>> Now the story is a bit different. I'm building the database using >>>>>>>>>>> tdbloader2 and would like to create the spatial index upfront. >>>>>>>>>>> >>>>>>>>>>> java -cp jena-spatial.jar jena.spatialindexer >> --desc=assembler_file >>>>>>>>>>> I could not figure out, how to get this line to work. I have the >>>>>>>>> following >>>>>>>>>>> possible scenarios: >>>>>>>>>>> >>>>>>>>>>> 1. I download the jena-spatial-3.8.0.jar and specify that in the >>>>>>>>> classpath: >>>>>>>>>>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer >>>>>>>>>>> --desc=../fuseki-server/run/configuration/mm.ttl >>>>>>>>>>> Error: Could not find or load main class jena.spatialindexer >>>>>>>>>>> >>>>>>>>>>> 2. I use the fuseki-server jar: >>>>>>>>>>> $ java -cp fuseki-server.jar jena.spatialindexer >>>>>>>>>>> --desc=./run/configuration/mm.ttl >>>>>>>>>>> WARN Custom SpatialContextFactory lib is not ready in >>>>>>>>>>> classpath:com/vividsolutions/jts/geom/CoordinateSequenceFactory >>>>>>>>>>> org.apache.jena.dboe.transaction.txn.TransactionException: Not >> in a >>>>>>>>>>> transaction >>>>>>>>>>> >>>>>>>>>>> I managed to eliminate the warning about jts by including that >> in the >>>>>>>>>>> classpath, but the error remains the same (with full trace): >>>>>>>>>>> java -cp fuseki-server.jar:jts-1.13.jar jena.spatialindexer >>>>>>>>>>> --desc=./run/configuration/mm.ttl >>>>>>>>>>> org.apache.jena.dboe.transaction.txn.TransactionException: Not >> in a >>>>>>>>>>> transaction >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) >>>>>>>>>>> at >>>>>>>>> org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) >>>>>>>>>>> at >>>>>>>>>>> >>>>>>>>> >>>>>>> >> org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) >>>>>>>>>>> at jena.spatialindexer.exec(spatialindexer.java:110) >>>>>>>>>>> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) >>>>>>>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) >>>>>>>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) >>>>>>>>>>> at jena.spatialindexer.main(spatialindexer.java:53) >>>>>>>>>>> >>>>>>>>>>> My configuration is the following: >>>>>>>>>>> @prefix : <http://base/#> . >>>>>>>>>>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . >>>>>>>>>>> @prefix tdb2: <http://jena.apache.org/2016/tdb#> . >>>>>>>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> . >>>>>>>>>>> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . >>>>>>>>>>> @prefix fuseki: <http://jena.apache.org/fuseki#> . >>>>>>>>>>> @prefix spatial: <http://jena.apache.org/spatial#> . >>>>>>>>>>> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . >>>>>>>>>>> @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . >>>>>>>>>>> # Spatial >>>>>>>>>>> [] ja:loadClass "org.apache.jena.query.spatial.SpatialQuery" . >>>>>>>>>>> spatial:SpatialtDataset rdfs:subClassOf ja:RDFDataset . >>>>>>>>>>> spatial:SpatialIndexLucene rdfs:subClassOf spatial:SpatialIndex . >>>>>>>>>>> >>>>>>>>>>> :service_tdb_all a fuseki:Service ; >>>>>>>>>>> rdfs:label "TDB2 mm" ; >>>>>>>>>>> fuseki:dataset :spatial_dataset ; >>>>>>>>>>> fuseki:name "mm" ; >>>>>>>>>>> fuseki:serviceQuery "query" , "sparql" ; >>>>>>>>>>> fuseki:serviceReadGraphStore "get" ; >>>>>>>>>>> fuseki:serviceReadWriteGraphStore >>>>>>>>>>> "data" ; >>>>>>>>>>> fuseki:serviceUpdate "update" ; >>>>>>>>>>> fuseki:serviceUpload "upload" . >>>>>>>>>>> >>>>>>>>>>> :spatial_dataset a spatial:SpatialDataset ; >>>>>>>>>>> spatial:dataset :tdb_dataset_readwrite ; >>>>>>>>>>> spatial:index <#indexLucene> ; >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> <#indexLucene> a spatial:SpatialIndexLucene ; >>>>>>>>>>> spatial:directory <file:Lucene> ; >>>>>>>>>>> #spatial:directory "mem" ; >>>>>>>>>>> spatial:definition <#definition> ; >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> <#definition> a spatial:EntityDefinition ; >>>>>>>>>>> spatial:entityField "uri" ; >>>>>>>>>>> spatial:geoField "geo" ; >>>>>>>>>>> # custom geo predicates for 1) Latitude/Longitude Format >>>>>>>>>>> spatial:hasSpatialPredicatePairs ( >>>>>>>>>>> [ spatial:latitude geo:lat ; spatial:longitude geo:long ] >>>>>>>>>>> ) ; >>>>>>>>>>> # custom geo predicates for 2) Well Known Text (WKT) Literal >>>>>>>>>>> spatial:hasWKTPredicates (geosparql:asWKT) ; >>>>>>>>>>> # custom SpatialContextFactory for 2) Well Known Text (WKT) >> Literal >>>>>>>>>>> spatial:spatialContextFactory >>>>>>>>>>> # >> "com.spatial4j.core.context.jts.JtsSpatialContextFactory" >>>>>>>>>>> >>>>>>> "org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory" >>>>>>>>>>> . >>>>>>>>>>> >>>>>>>>>>> :tdb_dataset_readwrite >>>>>>>>>>> a tdb2:DatasetTDB2 ; >>>>>>>>>>> tdb2:location >>>>>>>>>>> "/srv/linked_data_store/fuseki-server/run/databases/mm" . >>>>>>>>>>> >>>>>>>>>>> And I've changed the fuseki script to include the jts jar: >>>>>>>>>>> FUSEKI_CLASSES=$FUSEKI_HOME/jts-1.13.jar >>>>>>>>>>> >>>>>>>>>>> I see no errors or warnings in the server log, when starting >> with that >>>>>>>>>>> configuration, the Lucene folder is created and populated with >>>>>>>>> something: >>>>>>>>>>> $ ll run/configuration/Lucene/ >>>>>>>>>>> total 4 >>>>>>>>>>> -rw-rw-r-- 1 process process 71 Sep 14 09:28 segments_1 >>>>>>>>>>> -rw-rw-r-- 1 process process 0 Sep 14 09:28 write.lock >>>>>>>>>>> >>>>>>>>>>> But any spatial query does not return any results. >>>>>>>>>>> Any help on how to tackle this would be highly appreciated. >>>>>>>>>>> >>>>>>>>>>> Best >>>>>>>>>>> Markus Neumann >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> --- >>>>>>>>>> Marco Neumann >>>>>>>>>> KONA >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> >>>>>>>> >>>>>>>> --- >>>>>>>> Marco Neumann >>>>>>>> KONA >>>>>>> >>>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> >>>>>> --- >>>>>> Marco Neumann >>>>>> KONA >>> >> >> >> > > -- > > > --- > Marco Neumann > KONA
