By default geotools does not merge properly into an uber-jar as some files
override others in the meta-inf directory. This is explained somewhere on the
geotools site and could be relevant to your problem? You can use the maven
shade plugin to do some automagic renaming to get round this. I do it here
https://github.com/PGWelch/com.opendoorlogistics/blob/master/odl-geotools-fat-jar/pom.xml
based on one of their examples.
Phil Welch
Sent from Samsung Mobile
-------- Original message --------
From: Stuart Adam <[email protected]>
Date:
To: [email protected]
Subject: Re: [GraphHopper] Ordnance Survey Loader
Hi Peter
Yes as I suspected the uberjar causes problems. The
/META-INF/services/org.opengis.referencing.crs.CRSAuthorityFactory ,
/META-INF/services/org.opengis.referencing.crs.CRSFactory,
/META-INF/services/org.opengis.referencing.datum.DatumAuthorityFactory and
/META-INF/services/org.opengis.referencing.datum.DatumFactory files stored are
only the versions from gt-referencing not the ones from gt-epsg-hsql (or
gt-epsg-wkt if you chose that implementation)
Not sure how we merge the content of these service registry files as I believe
we really want the content from all supplied versions.
Sincerely
Stuart Adam
Date: Tue, 25 Nov 2014 22:05:07 +0000
From: [email protected]
To: [email protected]
Subject: Re: [GraphHopper] Ordnance Survey Loader
Hi Peter
I think I might have an idea what is happening when you use the graphhopper.sh
script (I have updated to something closer to the version in upstream). The
geotools libs use a service provider pattern whereby properties files in the
META-INF/services folder of the implementing jar are read for the name of the
implementing class. It may be that the repacked jar does not have the right
layout for that to work. I will try to investigate that a bit further.
Sincerely
Stuart Adam
Sent from my BlackBerry 10 smartphone on the EE network.
From: Peter
Sent: Tuesday, 25 November 2014 13:27
To: GraphHopper Java routing engine
Reply To: GraphHopper Java routing engine
Subject: Re: [GraphHopper] Ordnance Survey Loader
Hi Stuart,
with your latest changes, which already include all dependencies as you wrote I
still get the exception. Using a clean repo and following my instructions in
the issue.
Do you have an idea how to fix this? I looked into the created jar which the
graphhopper.sh script is using (tools/target/"gh-with-deps") and the content
looks fine: with hsqldb etc
Regards,
Peter
On 24.11.2014 16:04, Stuart Adam wrote:
Hi Guys
I was using the Import class to pre prep the graph. See itngen.sh for a
command line. The pom in my fork of core should already contain the following
dependency info which would ensure the correct jars are downloaded to your
local mvn repo. gotools.version is 12.1 and you may need to add the repo
<repository>
<id>osgeo</id>
<name>Open Source Geospatial Foundation Repository</name>
<url>http://download.osgeo.org/webdav/geotools/</url>
</repository>
Dependencies as follows (you might be able to use gt-epsg-wkt in place of
gt-epsg-hsql and thereby remove the jdbc dependency as well, this worked for me
before but wasn't working when I was readying to commit hence the more
heavyweight dependency)
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-metadata</artifactId>
<version>${geotools.version}</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-opengis</artifactId>
<version>${geotools.version}</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-referencing</artifactId>
<version>${geotools.version}</version>
</dependency>
<dependency>
<groupId>net.java.dev.jsr-275</groupId>
<artifactId>jsr-275</artifactId>
<version>1.0-beta-2</version>
</dependency>
<dependency>
<groupId>java3d</groupId>
<artifactId>vecmath</artifactId>
<version>1.3.2</version>
</dependency>
<dependency>
<groupId>org.geotools</groupId>
<artifactId>gt-epsg-hsql</artifactId>
<version>${geotools.version}</version>
</dependency>
<dependency>
<groupId>org.hsqldb</groupId>
<artifactId>hsqldb</artifactId>
<version>2.3.2</version>
</dependency>
I have also removed the reflective code.
Sincerely
Stuart Adam
Date: Mon, 24 Nov 2014 15:49:27 +0100
From: [email protected]
To: [email protected]
Subject: Re: [GraphHopper] Ordnance Survey Loader
Hi Stuart, hi Philip,
thanks for the insights! Also thanks Philip for the suggestion!
> Since once the import is done the graph is the same as one imported from osm
> the query time code is unchanged nd so should still be Java 6 compatible.
I see. Then we should probably just make the module separation and put all
import stuff into another module or tools. E.g. the web module is also java 7
Also I'm not 100% sure if java 7 is that problematic as e.g. Android
development should be possible with java7 too I think.
> As commented by Philip Welch on here the simplest one's to provide at runtime
> is either the hsqldb (remember to add the hsql-jdbc driver lib as well)
> or the wkt based one.
Do you have a pom snippet at hand :) ?
> (OSITN = too much block caps for my liking)
me too but no problem - just wanted to mention it
> The sample data actually requires a license agreement so people should be
> directed to the website rather than trying to automate via wget.
> If it were not for that stage I would have placed the sample data in git
> repo. I will try and negotiate for permission to include it which would
> simplify that stage.
We can also do this license agreement on the (bash) shell.
> At the moment execution time without contraction hierarchies (in order to
> support turn restrictions)
> is very slow and so we would be interested in what work is going on to
> combine the two and if there is any way we can help
What do you mean with 'very slow'? With turn restrictions it should be like 2
times slower than without.
Combining the both is no magic but requires some time to work out the bits
(although all the necessary steps are already defined -> #270). And of course
this is important for us too but no starting date defined yet.
Regards,
Peter
On 24.11.2014 14:52, Stuart Adam wrote:
HI Peter
The logging is so heavy as we are still learning our way around the api.
Absolutely this should be reduced as time goes on and there is already a number
of test cases specifically to do with os data. Unfortunately due to a late
change in coordinate convertor (ITN uses British National Grid) some of these
are currently broken. The tests are almost certainly not as clean as they
could be. Only recently found the GraphInfo class that might help some with
that :-)
I used reflection as that would enable us to more dynamically extend the
available readers but if it causes problems with the iOS port then absolutely
happy to change that. iOS and android are important for us. The import code at
the moment takes advantage of String based switches so is Java 7 specific. We
can work on removing that but for now have been relying on using desktop class
machines to pre import and then making the graph folder available to client
machines. Since once the import is done the graph is the same as one imported
from osm the query time code is unchanged nd so should still be Java 6
compatible.
Yes we will standardise the naming scheme. Relatively new to git so was
leaving such refactorings for now but was already unhappy with my initial
naming (OSITN = too much block caps for my liking)
The geotools supplies multiple different implementations of the CRS lookup. As
commented by Philip Welch on here the simplest one's to provide at runtime is
either the hsqldb (remember to add the hsql-jdbc driver lib as well) or the wkt
based one. Only provide a single implementation at runtime. Given the
preferences of deployment teams the actual implementation used wasn't hardwired
in to the build.
The sample data actually requires a license agreement so people should be
directed to the website rather than trying to automate via wget. If it were
not for that stage I would have placed the sample data in git repo. I will try
and negotiate for permission to include it which would simplify that stage.
At the moment execution time without contraction hierarchies (in order to
support turn restrictions) is very slow and so we would be interested in what
work is going on to combine the two and if there is any way we can help.
Sincerely
Stuart Adam
Date: Mon, 24 Nov 2014 12:26:31 +0100
From: [email protected]
To: [email protected]
Subject: Re: [GraphHopper] Ordnance Survey Loader
Hi Stuart,
this is nice - thanks!
I took a closer look at it. There seems to be an old graphhopper.sh script?
Also I had a problem with the import when using the sample data you mentioned**
What I did I described in the 'current instructions' of the issue #227 and let
us move the discussion to it:
https://github.com/graphhopper/graphhopper/issues/277
Let me know when you think that this is stable enough, then we can work
together on improving the abstraction and refactoring mentioned in the notes,
if you like.
Kind Regards,
Peter
**
org.opengis.referencing.NoSuchAuthorityCodeException: No code "EPSG:27700" from
authority "EPSG" found for object of type "EngineeringCRS".
at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.noSuchAuthorityException(CartesianAuthorityFactory.java:136)
at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createEngineeringCRS(CartesianAuthorityFactory.java:130)
at
org.geotools.referencing.factory.epsg.CartesianAuthorityFactory.createCoordinateReferenceSystem(CartesianAuthorityFactory.java:121)
at
org.geotools.referencing.factory.AuthorityFactoryAdapter.createCoordinateReferenceSystem(AuthorityFactoryAdapter.java:801)
at
org.geotools.referencing.factory.ThreadedAuthorityFactory.createCoordinateReferenceSystem(ThreadedAuthorityFactory.java:731)
at
org.geotools.referencing.DefaultAuthorityFactory.createCoordinateReferenceSystem(DefaultAuthorityFactory.java:179)
at org.geotools.referencing.CRS.decode(CRS.java:519)
at org.geotools.referencing.CRS.decode(CRS.java:447)
at
uk.co.ordnancesurvey.api.srs.OpenCoordConverter.<clinit>(OpenCoordConverter.java:18)
at
com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)
at com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)
at
com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)
at com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)
at com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)
at
com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)
at
com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)
at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)
at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)
at
com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)
at com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)
at com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)
at com.graphhopper.GraphHopper.importData(GraphHopper.java:634)
at com.graphhopper.GraphHopper.process(GraphHopper.java:603)
at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)
at com.graphhopper.tools.Import.main(Import.java:16)
Exception in thread "main" java.lang.RuntimeException: Problem while parsing
file
at com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:262)
at com.graphhopper.reader.osgb.OsItnReader.readGraph(OsItnReader.java:244)
at com.graphhopper.GraphHopper.importData(GraphHopper.java:634)
at com.graphhopper.GraphHopper.process(GraphHopper.java:603)
at com.graphhopper.GraphHopper.importOrLoad(GraphHopper.java:576)
at com.graphhopper.tools.Import.main(Import.java:16)
Caused by: java.lang.NullPointerException
at org.geotools.referencing.CRS.findMathTransform(CRS.java:1201)
at
uk.co.ordnancesurvey.api.srs.OpenCoordConverter.toWGS84(OpenCoordConverter.java:30)
at
com.graphhopper.reader.osgb.OSITNNode.parseCoordinateString(OSITNNode.java:143)
at com.graphhopper.reader.osgb.OSITNNode.parseCoords(OSITNNode.java:131)
at
com.graphhopper.reader.osgb.OSITNElement.handleCoordinates(OSITNElement.java:289)
at com.graphhopper.reader.osgb.OSITNElement.readTags(OSITNElement.java:89)
at com.graphhopper.reader.osgb.OSITNNode.create(OSITNNode.java:55)
at
com.graphhopper.reader.osgb.OsItnInputFile.getNextXML(OsItnInputFile.java:209)
at
com.graphhopper.reader.osgb.OsItnInputFile.getNext(OsItnInputFile.java:174)
at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:294)
at
com.graphhopper.reader.osgb.OsItnReader.preProcessSingleFile(OsItnReader.java:284)
at
com.graphhopper.reader.osgb.OsItnReader.preProcessDirOrFile(OsItnReader.java:275)
at com.graphhopper.reader.osgb.OsItnReader.preProcess(OsItnReader.java:260)
... 5 more
On 19.11.2014 00:08, Stuart Adam wrote:
Hi All
The dependency on internal code has been removed from the OS ITN data loader by
using geotools.
This means you should be able to try it out now. Some of the tests currently
fail but the basics work. To build you will want to skip the acceptancetesting
module, since that is an internal test which will probably be moved out of this
repository as it is data dependant. For now skip tests whilst I figure out what
has broken the unit tests, hopefully tomorrow.
To run graphhopper loading itn files supply the following configuration options
osmreader.osm=<path to data file as normal but this should be either a
directory or itn xml file rather than osm>
reader.implementation=com.graphhopper.reader.osgb.OsItnReader
the reader.implementation setting changes which DataReader implementation
GraphHopper uses with com.graphhopper.reader.osgb.OsItnReader and
com.graphhopper.reader.OSMReader now implementing an interface DataReader. if
reader.implementation is not set com.graphhopper.reader.OSMReader is used by
default.
osmreader.osm setting should probably change to datareader.src with this change
but for now I have left that alone for compatibility reasons.
Once the data is imported it can be used in a standard GraphHopper server
without having to switch reader implementation which means ingested data should
work in a GraphHopper instance from the main repo without consuming the
engaric/graphhopper branch though I believe at the moment I need to resync with
graphhopper/graphhopper repo as I suspect the graph version has been changing
rapidly recently.
Sincerely
Stuart Adam
_______________________________________________ GraphHopper mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/graphhopper
_______________________________________________
GraphHopper mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/graphhopper
_______________________________________________ GraphHopper mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/graphhopper
_______________________________________________ GraphHopper mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/graphhopper_______________________________________________
GraphHopper mailing list
[email protected]
https://lists.openstreetmap.org/listinfo/graphhopper
