On Wed, Sep 19, 2018 at 11:04 AM Andy Seaborne <[email protected]> wrote:
> > > On 17/09/18 15:26, ajs6f wrote: > > Okay, I think we've got it. Nice debugging, Markus! > > > > If you look at the line of the spatial indexer code that is throwing an > exception: > > > > > https://github.com/apache/jena/blob/master/jena-spatial/src/main/java/jena/spatialindexer.java#L110 > > > > indeed, it does not start a transaction against the dataset before > starting to read quads to index them. Now, in TDB1 this isn't inherently a > problem-- it's possible to work against a dataset without transactions. > (Although it's generally not a good idea at all.) But IIRC, TDB2 requires > transactions. If that's true (and Andy can verify that for us) then this is > a bug and we can fix it in a straightforward way. > > > > Andy, can you confirm that diagnosis? > > Yes - TDB2 only works transactionally. > > Andy-- I am just looking at the file system, I noticed that there is no binary difference between tdb and tdb2 indexes. would it be fair to describe tdb2 as a tdb+txn with different access methods? > > > If so, I'll open a ticket and write a PR to use a Txn method to fix this. > > which ajs6f has merged into the development system. > > > > > ajs6f > > > >> On Sep 17, 2018, at 10:14 AM, Markus Neumann <[email protected]> > wrote: > >> > >> It seems I made a mess with absolute and relative paths all together. > So I created a minimal setup for testing: > >> (I include the file-contents at the bottom) > >> > >> $ export JENA_HOME=/srv/linked_data_store/apache-jena-3.8.0 > >> $ export PATH=$PATH:$JENA_HOME/bin > >> $ tdb2.tdbloader > --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl > >> 13:50:54 INFO loader :: Loader = LoaderPhased > >> 13:50:54 INFO loader :: Data: > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl: > 13 mm-loc-mini.ttl 0.06s (Avg: 232) > >> 13:50:54 INFO loader :: Data: > /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl: 50 > mm-rdf.ttl 0.01s (Avg: 6,250) > >> 13:50:55 INFO loader :: Finish - index SPO > >> 13:50:55 INFO loader :: Start replay index SPO > >> 13:50:55 INFO loader :: Index set: SPO => SPO->POS, > SPO->OSP > >> 13:50:55 INFO loader :: Index set: SPO => SPO->POS, > SPO->OSP [63 items, 0.0 seconds] > >> 13:50:55 INFO loader :: Finish - index POS > >> 13:50:55 INFO loader :: Finish - index OSP > >> $ java -cp > /srv/linked_data_store/fuseki-server/fuseki-server.jar:/srv/linked_data_store/apache-jena-3.8.0/lib/* > jena.spatialindexer > --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl > >> SLF4J: Class path contains multiple SLF4J bindings. > >> SLF4J: Found binding in > [jar:file:/srv/linked_data_store/fuseki-server/fuseki-server.jar!/org/slf4j/impl/StaticLoggerBinder.class] > >> SLF4J: Found binding in > [jar:file:/srv/linked_data_store/apache-jena-3.8.0/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class] > >> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > >> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] > >> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a > transaction > >> at > org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) > >> at > org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) > >> at > org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) > >> at > org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) > >> at > org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) > >> at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) > >> at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) > >> at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) > >> at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) > >> at > org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) > >> at > org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) > >> at > org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) > >> at > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > >> at > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > >> at jena.spatialindexer.exec(spatialindexer.java:110) > >> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) > >> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) > >> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) > >> at jena.spatialindexer.main(spatialindexer.java:53) > >> > >> I can get rid of the SLF4J warnings: > >> $ java -cp /srv/linked_data_store/fuseki-server/fuseki-server.jar > jena.spatialindexer > --desc=/srv/linked_data_store/fuseki-server/run/configuration/mm.ttl > >> org.apache.jena.dboe.transaction.txn.TransactionException: Not in a > transaction > >> at > org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) > >> at > org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) > >> at > org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) > >> at > org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) > >> at > org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) > >> at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) > >> at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) > >> at > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) > >> at org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) > >> at > org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) > >> at > org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) > >> at > org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) > >> at > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > >> at > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > >> at jena.spatialindexer.exec(spatialindexer.java:110) > >> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) > >> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) > >> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) > >> at jena.spatialindexer.main(spatialindexer.java:53) > >> > >> There is no other java instance running at the time when I run those > commands. > >> All paths are fully qualified and double checked. The destination > directory and the Lucene directory were cleared just before. > >> > >> The server does start and serve the data without errors or warnings in > the log, but spatial queries do not work. > >> > >> Does anyone see a reason for the TransactionException? > >> Thanks and best regards > >> Markus > >> > >> Attachments: > >> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-rdf.ttl: > >> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . > >> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . > >> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > >> @prefix owl: <http://www.w3.org/2002/07/owl#> . > >> @prefix dc: <http://purl.org/dc/elements/1.1/> . > >> @prefix dcterms: <http://purl.org/dc/terms/> . > >> @prefix xsd: <http://www.w3.org/2001/XMLSchema#> > >> @prefix mm: <http://rdf.meteomatics.com/mm/> . > >> > >> <http://rdf.meteomatics.com/mm/> > >> dcterms:publisher <http://api.meteomatics.com/Imprint.html> ; > >> dcterms:title "Meteomatics RDF Vocabulary"@en . > >> > >> mm:location a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "location" ; > >> dcterms:description "Location of a measurement"@en . > >> > >> mm:elevation a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "elevation" ; > >> dcterms:description "Elevation of a location"@en . > >> > >> mm:validdate a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "validdate" ; > >> dcterms:description "Temporal information describing when a > Measurement is/was valid"@en . > >> > >> mm:value a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "value" ; > >> dcterms:description "The (observed or predicted) value of a > Measurement"@en . > >> > >> mm:parameter a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "parameter" ; > >> dcterms:description "what was measured"@en . > >> > >> mm:station_name a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "station_name" ; > >> dcterms:description "Name of a station"@en . > >> > >> mm:abbreviation a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "abbreviation" ; > >> dcterms:description "Abbreviation of a station name"@en . > >> > >> mm:didok_id a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "didok_id" ; > >> dcterms:description "didok id of a station"@en . > >> > >> mm:unit_symbol a rdf:Property ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "unit symbol" ; > >> dcterms:description "unit symbol of a parameter"@en . > >> > >> mm:Measurement a rdf:Class ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "Measurement" ; > >> dcterms:description "Measurement of a specific weather parameter at > specific location at a specific date (both future or past)" . > >> > >> mm:Location a rdf:Class ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "Location" ; > >> dcterms:description "A Point on the globe where we observe > weather"@en . > >> > >> mm:Parameter a rdf:Class ; > >> rdfs:isDefinedBy <http://rdf.meteomatics.com/mm/> ; > >> rdfs:label "Parameter" ; > >> dcterms:description "A parameter describing an observable aspect of > the weather"@en . > >> > >> /srv/linked_data_store/rdf_python_connector/turtle_files/mm-loc-mini.ttl > >> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . > >> @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . > >> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . > >> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > >> @prefix owl: <http://www.w3.org/2002/07/owl#> . > >> @prefix dc: <http://purl.org/dc/elements/1.1/> . > >> @prefix dcterms: <http://purl.org/dc/terms/> . > >> @prefix xsd: <http://www.w3.org/2001/XMLSchema#> . > >> @prefix mm: <http://rdf.meteomatics.com/mm/> . > >> @prefix mm-measurements: <http://rdf.meteomatics.com/mm/measurements/> > . > >> @prefix mm-locations: <http://rdf.meteomatics.com/mm/locations/> . > >> > >> > >> mm-locations:8500015 a mm:Location ; > >> a geosparql:Geometry ; > >> mm:didok_id 8500015 ; > >> geo:lat 47.54259 ; > >> geo:long 7.61574 ; > >> mm:elevation 273 . > >> > >> > >> mm-locations:8504238 a mm:Location ; > >> a geosparql:Geometry ; > >> geosparql:asWKT "POINT(6.86466645231 > 47.0840196485)"^^geosparql:wktLiteral ; > >> mm:didok_id 8504238 ; > >> geo:lat 47.08402 ; > >> geo:long 6.86467 ; > >> mm:elevation 1086 . > >> > >> /srv/linked_data_store/fuseki-server/run/configuration/mm.ttl > >> @prefix : <http://base/#> . > >> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . > >> @prefix tdb2: <http://jena.apache.org/2016/tdb#> . > >> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> . > >> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > >> @prefix fuseki: <http://jena.apache.org/fuseki#> . > >> @prefix spatial: <http://jena.apache.org/spatial#> . > >> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . > >> @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . > >> # Spatial > >> #[] ja:loadClass "org.apache.jena.query.spatial.SpatialQuery" . > >> #spatial:SpatialtDataset rdfs:subClassOf ja:RDFDataset . > >> #spatial:SpatialIndexLucene rdfs:subClassOf spatial:SpatialIndex . > >> > >> :service_tdb_all a fuseki:Service ; > >> rdfs:label "TDB2 mm" ; > >> fuseki:dataset :spatial_dataset ; > >> fuseki:name "mm" ; > >> fuseki:serviceQuery "query" , "sparql" ; > >> fuseki:serviceReadGraphStore "get" ; > >> fuseki:serviceReadWriteGraphStore > >> "data" ; > >> fuseki:serviceUpdate "update" ; > >> fuseki:serviceUpload "upload" . > >> > >> :spatial_dataset a spatial:SpatialDataset ; > >> spatial:dataset :tdb_dataset_readwrite ; > >> spatial:index <#indexLucene> ; > >> . > >> > >> <#indexLucene> a spatial:SpatialIndexLucene ; > >> spatial:directory <file:Lucene> ; > >> #spatial:directory "mem" ; > >> spatial:definition <#definition> ; > >> . > >> > >> <#definition> a spatial:EntityDefinition ; > >> spatial:entityField "uri" ; > >> spatial:geoField "geo" ; > >> # custom geo predicates for 1) Latitude/Longitude Format > >> spatial:hasSpatialPredicatePairs ( > >> [ spatial:latitude geo:lat ; spatial:longitude geo:long ] > >> ) ; > >> # custom geo predicates for 2) Well Known Text (WKT) Literal > >> spatial:hasWKTPredicates (geosparql:asWKT) ; > >> #spatial:spatialContextFactory > >> # "com.spatial4j.core.context.jts.JtsSpatialContextFactory" > >> # "org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory" > >> . > >> > >> :tdb_dataset_readwrite > >> a tdb2:DatasetTDB2 ; > >> tdb2:location > "/srv/linked_data_store/fuseki-server/run/databases/mm" . > >> > >> > >>> Am 17.09.2018 um 15:15 schrieb ajs6f <[email protected]>: > >>> > >>> I think you might have misunderstood the use of $JENA_HOME, or perhaps > the Java classpath. $JENA_HOME, which should generally be an absolute > filepath, allows the Jena scripts to find their libraries at runtime. > Internally, the scripts use $JENA_HOME to set up the classpaths with which > to start running Java code. But in this case, you're not using the scripts. > You're calling a Java class directly, which means you have to give it all > the libraries directly, using the classpath flag directly. For example, > your second attempt: > >>> > >>> $ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer > --desc=fuseki-server/run/configuration/mm.ttl > >>> Error: Could not find or load main class > apache-jena-3.8.0.lib.commons-cli-1.4.jar > >>> > >>> may have failed because whatever directory you are in probably doesn't > happen to have a Jena distribution in it called "apache-jena-3.8.0". > Perhaps try: > >>> > >>> java -cp $JENA_HOME/lib/* jena.spatialindexer > --desc=fuseki-server/run/configuration/mm.ttl > >>> > >>> Your first example looks like it might be misusing the -cp flag [1]. > You probably don't want the *.jar, just *, like > >>> > >>> java -cp lib/* jena.spatialindexer > --desc=../fuseki-server/run/configuration/mm.ttl > >>> > >>> especially if you happen to be using Windows. [2] That might not be > the entire story, but you can at least try those invocations and see if you > get anything different/better. > >>> > >>> ajs6f > >>> > >>> [1] > https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html#A1100762 > < > https://docs.oracle.com/javase/8/docs/technotes/tools/windows/classpath.html#A1100762 > > > >>> [2] > https://stackoverflow.com/questions/219585/including-all-the-jars-in-a-directory-within-the-java-classpath > < > https://stackoverflow.com/questions/219585/including-all-the-jars-in-a-directory-within-the-java-classpath > > > >>> > >>> > >>>> On Sep 17, 2018, at 7:07 AM, Markus Neumann <[email protected] > <mailto:[email protected]>> wrote: > >>>> > >>>> Ok, that's exactly what I did here: > >>>> $ java -cp lib/*.jar jena.spatialindexer > --desc=../fuseki-server/run/configuration/mm.ttl > >>>> Error: Could not find or load main class lib.commons-cli-1.4.jar > >>>> > >>>> Or from outside JENA_HOME: > >>>> $ java -cp apache-jena-3.8.0/lib/* jena.spatialindexer > --desc=fuseki-server/run/configuration/mm.ttl > >>>> Error: Could not find or load main class > apache-jena-3.8.0.lib.commons-cli-1.4.jar > >>>> > >>>> Or did I missunderstand something? > >>>> > >>>>> Am 17.09.2018 um 12:02 schrieb Marco Neumann < > [email protected]>: > >>>>> > >>>>> don't worry it's a common problem and frequent source of confusion > not only > >>>>> with jena but java projects. try to simply inlcude all the jars. > >>>>> > >>>>> I presume you run this on a linux box with OpenJDK 8. correct? > >>>>> > >>>>> now once you have set up jena just add the following > >>>>> {replace_with_your_path}/apache-jena-3.8.0/lib/* > >>>>> > >>>>> > >>>>> > >>>>> > >>>>> On Mon, Sep 17, 2018 at 10:56 AM Markus Neumann < > [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>>> > >>>>> wrote: > >>>>> > >>>>>> This is embarrassing but I don't get it. > >>>>>> > >>>>>> I've configured $JENA_HOME and adjusted the PATH. > >>>>>> > >>>>>> Now what classpath should I include when invoking the indexer? > >>>>>> What I've tried so far: > >>>>>> > >>>>>> $ cd $JENA_HOME > >>>>>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer > >>>>>> --desc=../fuseki-server/run/configuration/mm.ttl > >>>>>> Error: Could not find or load main class jena.spatialindexer > >>>>>> $ java -cp lib/*.jar jena.spatialindexer > >>>>>> --desc=../fuseki-server/run/configuration/mm.ttl > >>>>>> Error: Could not find or load main class lib.commons-cli-1.4.jar > >>>>>> > >>>>>> Thanks for your patience > >>>>>> > >>>>>>> Am 17.09.2018 um 11:38 schrieb Marco Neumann < > [email protected] <mailto:[email protected]>>: > >>>>>>> > >>>>>>> yes correct, this looks good. now include the classpath during > invocation > >>>>>>> of the spatialindexer > >>>>>>> > >>>>>>> in addition it's always a good idea to properly configure jena > along your > >>>>>>> java runtime environment settings to avoid jdk / lib conflicts > during > >>>>>>> execution. > >>>>>>> > >>>>>>> https://jena.apache.org/documentation/tools/index.html < > https://jena.apache.org/documentation/tools/index.html> < > >>>>>> https://jena.apache.org/documentation/tools/index.html < > https://jena.apache.org/documentation/tools/index.html> < > https://jena.apache.org/documentation/tools/index.html < > https://jena.apache.org/documentation/tools/index.html>>> > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> On Mon, Sep 17, 2018 at 10:28 AM Markus Neumann < > >>>>>> [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>> <mailto: > [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> 1.: what do you mean by indexer? > >>>>>>>> Probably my biggest problem is, that I have no clue how to find > out, > >>>>>> where > >>>>>>>> the jena.spatialindexer could be located except for asking here. > >>>>>>>> > >>>>>>>> Here is the content of the apache jena I downloaded: > >>>>>>>> $ ls -lh ../apache-jena-3.8.0/lib > >>>>>>>> total 17M > >>>>>>>> -rw-r--r-- 1 process process 162K Oct 26 2017 collection-0.7.jar > >>>>>>>> -rw-r--r-- 1 process process 53K Oct 26 2017 commons-cli-1.4.jar > >>>>>>>> -rw-r--r-- 1 process process 328K Mar 5 2018 > commons-codec-1.11.jar > >>>>>>>> -rw-r--r-- 1 process process 576K Jun 25 09:59 > commons-compress-1.17.jar > >>>>>>>> -rw-r--r-- 1 process process 41K Mar 5 2018 commons-csv-1.5.jar > >>>>>>>> -rw-r--r-- 1 process process 210K Mar 5 2018 commons-io-2.6.jar > >>>>>>>> -rw-r--r-- 1 process process 425K Oct 26 2017 > commons-lang3-3.4.jar > >>>>>>>> -rw-r--r-- 1 process process 749K Jun 25 10:01 > httpclient-4.5.5.jar > >>>>>>>> -rw-r--r-- 1 process process 158K Jun 25 10:01 > >>>>>> httpclient-cache-4.5.5.jar > >>>>>>>> -rw-r--r-- 1 process process 318K Jun 25 10:01 httpcore-4.4.9.jar > >>>>>>>> -rw-r--r-- 1 process process 65K Oct 26 2017 > >>>>>>>> jackson-annotations-2.9.0.jar > >>>>>>>> -rw-r--r-- 1 process process 315K Jun 25 10:01 > jackson-core-2.9.5.jar > >>>>>>>> -rw-r--r-- 1 process process 1.3M Jun 25 10:01 > >>>>>> jackson-databind-2.9.5.jar > >>>>>>>> -rw-r--r-- 1 process process 17K Oct 26 2017 > jcl-over-slf4j-1.7.25.jar > >>>>>>>> -rw-r--r-- 1 process process 2.9M Jun 25 11:03 jena-arq-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 211K Jun 25 11:01 jena-base-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 259K Jun 25 11:07 jena-cmds-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 1.7M Jun 25 11:02 jena-core-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 135K Jun 25 11:05 > jena-dboe-base-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 21K Jun 25 11:05 > jena-dboe-index-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 74K Jun 25 11:05 > >>>>>>>> jena-dboe-transaction-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 95K Jun 25 11:06 > >>>>>>>> jena-dboe-trans-data-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 135K Jun 25 11:01 jena-iri-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 49K Jun 25 11:04 > >>>>>> jena-rdfconnection-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 2.8M Jun 25 11:01 > >>>>>> jena-shaded-guava-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 264K Jun 25 11:06 jena-tdb2-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 512K Jun 25 11:04 jena-tdb-3.8.0.jar > >>>>>>>> -rw-r--r-- 1 process process 1.6M Jun 25 10:01 > jsonld-java-0.12.0.jar > >>>>>>>> -rw-r--r-- 1 process process 232K Mar 5 2018 > libthrift-0.10.0.jar > >>>>>>>> -rw-r--r-- 1 process process 479K Oct 26 2017 log4j-1.2.17.jar > >>>>>>>> -rw-r--r-- 1 process process 41K Oct 26 2017 > slf4j-api-1.7.25.jar > >>>>>>>> -rw-r--r-- 1 process process 12K Oct 26 2017 > slf4j-log4j12-1.7.25.jar > >>>>>>>> > >>>>>>>> 2.: Turns out, I don't need the jts. after commenting out the > affected > >>>>>>>> lines, the server still runs fine. (But still without spatial > index) > >>>>>>>> > >>>>>>>> Best Markus > >>>>>>>> > >>>>>>>>> Am 17.09.2018 um 11:14 schrieb Marco Neumann < > [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>> > >>>>>>> : > >>>>>>>>> > >>>>>>>>> Ok let's take a look at this. > >>>>>>>>> > >>>>>>>>> 1. please include in the indexer in the classpath. It's part of > the > >>>>>>>> current > >>>>>>>>> jena distribution. > >>>>>>>>> > >>>>>>>>> 2. do you need jts? it's a package that allows you to index > multi point > >>>>>>>>> geometries. here a sample query would be helpful. If single point > >>>>>>>>> geometries are indexed and used in conjunction with MBR range > queries. > >>>>>>>>> there is no need for jts. > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> On Mon, Sep 17, 2018 at 9:14 AM Markus Neumann < > >>>>>> [email protected] <mailto:[email protected]> <mailto: > [email protected] <mailto:[email protected]>> > >>>>>>>> <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>> > <mailto:[email protected] <mailto:[email protected]> > <mailto:[email protected] <mailto:[email protected]>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Hi, > >>>>>>>>>> (sorry for the long post) > >>>>>>>>>> > >>>>>>>>>> I've been struggling with the spatial index for a while now. > >>>>>>>>>> I've read > >>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>> < > >>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>>> < > >>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>> < > >>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>>>> < > >>>>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html > <https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>> < > >>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>>> < > >>>>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>> < > >>>>>> https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html> < > https://jena.apache.org/documentation/query/spatial-query.html < > https://jena.apache.org/documentation/query/spatial-query.html>>>>> and > >>>>>>>>>> skipped through about every spatial related thread here on the > list. > >>>>>>>>>> I managed to get an in-memory version up and running at some > point > >>>>>>>>>> following this > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>> > >>>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>> < > >>>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>>> > >>>>>>>> < > >>>>>>>> > >>>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>> < > >>>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>>> > >>>>>>>>> > >>>>>>>>>> < > >>>>>>>>>> > >>>>>>>> > >>>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>> < > >>>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>>> > >>>>>>>> < > >>>>>>>> > >>>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>> < > >>>>>> > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >< > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > < > https://stackoverflow.com/questions/45814956/building-geospatial-index-when-working-with-jena-fuseki/52099066#52099066 > >> > >>>>>>> > >>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>> and this > >>>>>>>>>> > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ < > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/ < > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/>> > >>>>>> < > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < > >>>>>>>> > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> < > >>>>>>>>>> > https://ceyxstudios.com/2017/11/25/rdf-datenbank-mit-geo-funktionen/> > >>>>>>>>>> > >>>>>>>>>> But that was on a small dataset that I uploaded via the > web-interface > >>>>>> of > >>>>>>>>>> Fuseki. > >>>>>>>>>> > >>>>>>>>>> Now the story is a bit different. I'm building the database > using > >>>>>>>>>> tdbloader2 and would like to create the spatial index upfront. > >>>>>>>>>> > >>>>>>>>>> java -cp jena-spatial.jar jena.spatialindexer > --desc=assembler_file > >>>>>>>>>> I could not figure out, how to get this line to work. I have the > >>>>>>>> following > >>>>>>>>>> possible scenarios: > >>>>>>>>>> > >>>>>>>>>> 1. I download the jena-spatial-3.8.0.jar and specify that in the > >>>>>>>> classpath: > >>>>>>>>>> $ java -cp jena-spatial-3.8.0.jar jena.spatialindexer > >>>>>>>>>> --desc=../fuseki-server/run/configuration/mm.ttl > >>>>>>>>>> Error: Could not find or load main class jena.spatialindexer > >>>>>>>>>> > >>>>>>>>>> 2. I use the fuseki-server jar: > >>>>>>>>>> $ java -cp fuseki-server.jar jena.spatialindexer > >>>>>>>>>> --desc=./run/configuration/mm.ttl > >>>>>>>>>> WARN Custom SpatialContextFactory lib is not ready in > >>>>>>>>>> classpath:com/vividsolutions/jts/geom/CoordinateSequenceFactory > >>>>>>>>>> org.apache.jena.dboe.transaction.txn.TransactionException: Not > in a > >>>>>>>>>> transaction > >>>>>>>>>> > >>>>>>>>>> I managed to eliminate the warning about jts by including that > in the > >>>>>>>>>> classpath, but the error remains the same (with full trace): > >>>>>>>>>> java -cp fuseki-server.jar:jts-1.13.jar jena.spatialindexer > >>>>>>>>>> --desc=./run/configuration/mm.ttl > >>>>>>>>>> org.apache.jena.dboe.transaction.txn.TransactionException: Not > in a > >>>>>>>>>> transaction > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.dboe.transaction.txn.TransactionalComponentLifecycle.checkTxn(TransactionalComponentLifecycle.java:417) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.dboe.trans.bplustree.BPlusTree.getRootRead(BPlusTree.java:159) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.dboe.trans.bplustree.BPlusTree.iterator(BPlusTree.java:348) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.tdb2.store.tupletable.TupleIndexRecord.all(TupleIndexRecord.java:251) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.tdb2.store.tupletable.TupleTable.find(TupleTable.java:148) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:161) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.find(NodeTupleTableConcrete.java:150) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.tdb2.store.nodetupletable.NodeTupleTableConcrete.findAsNodeIds(NodeTupleTableConcrete.java:141) > >>>>>>>>>> at > >>>>>>>> org.apache.jena.tdb2.store.TripleTable.find(TripleTable.java:64) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.tdb2.store.DatasetGraphTDB.findInDftGraph(DatasetGraphTDB.java:104) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.sparql.core.DatasetGraphBaseFind.findAny(DatasetGraphBaseFind.java:72) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.sparql.core.DatasetGraphBaseFind.find(DatasetGraphBaseFind.java:50) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > >>>>>>>>>> at > >>>>>>>>>> > >>>>>>>> > >>>>>> > org.apache.jena.sparql.core.DatasetGraphWrapper.find(DatasetGraphWrapper.java:152) > >>>>>>>>>> at jena.spatialindexer.exec(spatialindexer.java:110) > >>>>>>>>>> at jena.cmd.CmdMain.mainMethod(CmdMain.java:93) > >>>>>>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:58) > >>>>>>>>>> at jena.cmd.CmdMain.mainRun(CmdMain.java:45) > >>>>>>>>>> at jena.spatialindexer.main(spatialindexer.java:53) > >>>>>>>>>> > >>>>>>>>>> My configuration is the following: > >>>>>>>>>> @prefix : <http://base/#> . > >>>>>>>>>> @prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#> . > >>>>>>>>>> @prefix tdb2: <http://jena.apache.org/2016/tdb#> . > >>>>>>>>>> @prefix ja: <http://jena.hpl.hp.com/2005/11/Assembler#> . > >>>>>>>>>> @prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> . > >>>>>>>>>> @prefix fuseki: <http://jena.apache.org/fuseki#> . > >>>>>>>>>> @prefix spatial: <http://jena.apache.org/spatial#> . > >>>>>>>>>> @prefix geo: <http://www.w3.org/2003/01/geo/wgs84_pos#> . > >>>>>>>>>> @prefix geosparql: <http://www.opengis.net/ont/geosparql#> . > >>>>>>>>>> # Spatial > >>>>>>>>>> [] ja:loadClass "org.apache.jena.query.spatial.SpatialQuery" . > >>>>>>>>>> spatial:SpatialtDataset rdfs:subClassOf ja:RDFDataset . > >>>>>>>>>> spatial:SpatialIndexLucene rdfs:subClassOf spatial:SpatialIndex > . > >>>>>>>>>> > >>>>>>>>>> :service_tdb_all a fuseki:Service ; > >>>>>>>>>> rdfs:label "TDB2 mm" ; > >>>>>>>>>> fuseki:dataset :spatial_dataset ; > >>>>>>>>>> fuseki:name "mm" ; > >>>>>>>>>> fuseki:serviceQuery "query" , "sparql" ; > >>>>>>>>>> fuseki:serviceReadGraphStore "get" ; > >>>>>>>>>> fuseki:serviceReadWriteGraphStore > >>>>>>>>>> "data" ; > >>>>>>>>>> fuseki:serviceUpdate "update" ; > >>>>>>>>>> fuseki:serviceUpload "upload" . > >>>>>>>>>> > >>>>>>>>>> :spatial_dataset a spatial:SpatialDataset ; > >>>>>>>>>> spatial:dataset :tdb_dataset_readwrite ; > >>>>>>>>>> spatial:index <#indexLucene> ; > >>>>>>>>>> . > >>>>>>>>>> > >>>>>>>>>> <#indexLucene> a spatial:SpatialIndexLucene ; > >>>>>>>>>> spatial:directory <file:Lucene> ; > >>>>>>>>>> #spatial:directory "mem" ; > >>>>>>>>>> spatial:definition <#definition> ; > >>>>>>>>>> . > >>>>>>>>>> > >>>>>>>>>> <#definition> a spatial:EntityDefinition ; > >>>>>>>>>> spatial:entityField "uri" ; > >>>>>>>>>> spatial:geoField "geo" ; > >>>>>>>>>> # custom geo predicates for 1) Latitude/Longitude Format > >>>>>>>>>> spatial:hasSpatialPredicatePairs ( > >>>>>>>>>> [ spatial:latitude geo:lat ; spatial:longitude geo:long ] > >>>>>>>>>> ) ; > >>>>>>>>>> # custom geo predicates for 2) Well Known Text (WKT) Literal > >>>>>>>>>> spatial:hasWKTPredicates (geosparql:asWKT) ; > >>>>>>>>>> # custom SpatialContextFactory for 2) Well Known Text (WKT) > Literal > >>>>>>>>>> spatial:spatialContextFactory > >>>>>>>>>> # > "com.spatial4j.core.context.jts.JtsSpatialContextFactory" > >>>>>>>>>> > >>>>>> "org.locationtech.spatial4j.context.jts.JtsSpatialContextFactory" > >>>>>>>>>> . > >>>>>>>>>> > >>>>>>>>>> :tdb_dataset_readwrite > >>>>>>>>>> a tdb2:DatasetTDB2 ; > >>>>>>>>>> tdb2:location > >>>>>>>>>> "/srv/linked_data_store/fuseki-server/run/databases/mm" . > >>>>>>>>>> > >>>>>>>>>> And I've changed the fuseki script to include the jts jar: > >>>>>>>>>> FUSEKI_CLASSES=$FUSEKI_HOME/jts-1.13.jar > >>>>>>>>>> > >>>>>>>>>> I see no errors or warnings in the server log, when starting > with that > >>>>>>>>>> configuration, the Lucene folder is created and populated with > >>>>>>>> something: > >>>>>>>>>> $ ll run/configuration/Lucene/ > >>>>>>>>>> total 4 > >>>>>>>>>> -rw-rw-r-- 1 process process 71 Sep 14 09:28 segments_1 > >>>>>>>>>> -rw-rw-r-- 1 process process 0 Sep 14 09:28 write.lock > >>>>>>>>>> > >>>>>>>>>> But any spatial query does not return any results. > >>>>>>>>>> Any help on how to tackle this would be highly appreciated. > >>>>>>>>>> > >>>>>>>>>> Best > >>>>>>>>>> Markus Neumann > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>>> -- > >>>>>>>>> > >>>>>>>>> > >>>>>>>>> --- > >>>>>>>>> Marco Neumann > >>>>>>>>> KONA > >>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> > >>>>>>> > >>>>>>> --- > >>>>>>> Marco Neumann > >>>>>>> KONA > >>>>>> > >>>>>> > >>>>> > >>>>> -- > >>>>> > >>>>> > >>>>> --- > >>>>> Marco Neumann > >>>>> KONA > >> > > > > > > -- --- Marco Neumann KONA
