Re: [Neo4j] More spatial questions
Mmh,
you are looking not for the next node on a way but the nearest? In
that case, you maybe coudl index all OSM nodes in a new layer
ALL_NODES, then fish out the next node on the way, and do a search
for nodes closer that this and check if they are on your way?
Cheers,
/peter neubauer
GTalk: neubauer.peter
Skype peter.neubauer
Phone +46 704 106975
LinkedIn http://www.linkedin.com/in/neubauer
Twitter http://twitter.com/peterneubauer
http://www.neo4j.org - Your high performance graph database.
http://startupbootcamp.org/ - Öresund - Innovation happens HERE.
http://www.thoughtmade.com - Scandinavia's coolest Bring-a-Thing party.
On Fri, Jul 22, 2011 at 5:45 PM, Nolan Darilek no...@thewordnerd.info wrote:
OK, thanks for this. Unfortunately my project is getting less attention
now than it ever has, but I finally sat down and reworked my
architecture. Instead of working with Neo4J Nodes, I've reworked my
library to use SpatialDatabaseRecord and am now having a bit more success.
I have three more query types to implement, and I'm not immediately
seeing how to do this using the OSM API. I'm not sure if I need to drop
to the lower-level JTS searches for these, so any guidance would be
appreciated. Note that, in all that follows, node means OSM node, not
Neo4J node:
def nearestWay(lat:Double, lon:Double,
allowedTypes:Option[List[String]] = None) = {
Finds the nearest way to a given lat, lon. The list of allowed types
lets me filter out power lines, footways, etc. by specifying values for
the highway= tag.
def nearestNodeWithWay(lat:Double, lon:Double,
allowedTypes:Option[List[String]]) = {
Finds the nearest node on any way.
def waysForNode(n:Node) = {
Lists the ways on which a given node is found. OSMDataset lets me do the
opposite and returns the nodes for a way, but I don't immediately see a
way to do this.
Thanks.
On 06/19/2011 04:47 PM, Craig Taverner wrote:
Hi Nolan,
I think I can answer a few of your questions. Firstly, some background. The
graph model of the OSM data is based largely on the XML formated OSM
documents, and there you will find 'nodes', 'ways', 'relations' and 'tags'
each as their own xml-tag, and as a consequence each will also have their
own neo4j-node in the graph. Another point is that the geometry can be based
on one or more nodes or ways, and so we always create another node for the
geometry, and link it to the osm-node, way or relation that represents that
geometry.
What all this boils down to is that you cannot find the tags on the geometry
node itself. You cannot even find the location on that node. If you want to
use the graph model in a direct way, as you have been trying, you really do
need to know how the OSM data is modeled. For example, for a LineString
geometry, you would need to traverse from the geometry node to the way node
and finally to the tags node (to get the tags). To get to the locations is
even more complex. Rather than do that, I would suggest that you work with
the OSM API we provided with the OSMLayer, OSMDataset and OSMGeometryEncoder
classes. Then you do not need to know the graph model at all.
For example, OSMDataset has a method for getting a Way object from a node,
and the returned object can be queried for its nodes, geometry, etc.
Currently we provide methods for returning neo4j-nodes as well as objects
that make spatial sense. One minor issue here is the ambiguity inherent in
the fact that both neo4j and OSM make use of the term 'node', but for
different things. We have various solutions to this, sometimes replacing
'node' with 'point' and sometimes prefixing with 'osm'. The unit tests in
TestsForDocs includes some tests for the OSM API.
My first goal is to find the nearest OSM node to a given lat, lon. My
attempts seem to be made of fail thus far, however. Here's my code:
Most of the OSM dataset is converted into LineStrings, and what you really
want to do is find the closest vertex of the closest LineString. We have a
utility function 'findClosestEdges' in the SpatialTopologyUtils class for
that. The unit tests in TestSpatialUtils, and the testSnapping() method in
particular, show use of this.
My thinking is that nodes should be represented as points, so I can't
see why this fails. When I run this in a REPL, I do get a node back. So
far so good. Next, I want to get the node's tags. So I run:
The spatial search will return 'geometries', which are spatial objects. In
neo4j-spatial every geometry is represented by a unique node, but it is not
required that that node contain coordinates or tags. That is up to the
GeometryEncoder. In the case of the OSM model, this information is
elsewhere, because of the nature of the OSM graph, which is a highly
interconnected network of points, most of which do not represent Point
geometries, but are part of much more complex geometries (streets, regions,
buildings, etc.).
Re: [Neo4j] More spatial questions
OK, thanks for this. Unfortunately my project is getting less attention
now than it ever has, but I finally sat down and reworked my
architecture. Instead of working with Neo4J Nodes, I've reworked my
library to use SpatialDatabaseRecord and am now having a bit more success.
I have three more query types to implement, and I'm not immediately
seeing how to do this using the OSM API. I'm not sure if I need to drop
to the lower-level JTS searches for these, so any guidance would be
appreciated. Note that, in all that follows, node means OSM node, not
Neo4J node:
def nearestWay(lat:Double, lon:Double,
allowedTypes:Option[List[String]] = None) = {
Finds the nearest way to a given lat, lon. The list of allowed types
lets me filter out power lines, footways, etc. by specifying values for
the highway= tag.
def nearestNodeWithWay(lat:Double, lon:Double,
allowedTypes:Option[List[String]]) = {
Finds the nearest node on any way.
def waysForNode(n:Node) = {
Lists the ways on which a given node is found. OSMDataset lets me do the
opposite and returns the nodes for a way, but I don't immediately see a
way to do this.
Thanks.
On 06/19/2011 04:47 PM, Craig Taverner wrote:
Hi Nolan,
I think I can answer a few of your questions. Firstly, some background. The
graph model of the OSM data is based largely on the XML formated OSM
documents, and there you will find 'nodes', 'ways', 'relations' and 'tags'
each as their own xml-tag, and as a consequence each will also have their
own neo4j-node in the graph. Another point is that the geometry can be based
on one or more nodes or ways, and so we always create another node for the
geometry, and link it to the osm-node, way or relation that represents that
geometry.
What all this boils down to is that you cannot find the tags on the geometry
node itself. You cannot even find the location on that node. If you want to
use the graph model in a direct way, as you have been trying, you really do
need to know how the OSM data is modeled. For example, for a LineString
geometry, you would need to traverse from the geometry node to the way node
and finally to the tags node (to get the tags). To get to the locations is
even more complex. Rather than do that, I would suggest that you work with
the OSM API we provided with the OSMLayer, OSMDataset and OSMGeometryEncoder
classes. Then you do not need to know the graph model at all.
For example, OSMDataset has a method for getting a Way object from a node,
and the returned object can be queried for its nodes, geometry, etc.
Currently we provide methods for returning neo4j-nodes as well as objects
that make spatial sense. One minor issue here is the ambiguity inherent in
the fact that both neo4j and OSM make use of the term 'node', but for
different things. We have various solutions to this, sometimes replacing
'node' with 'point' and sometimes prefixing with 'osm'. The unit tests in
TestsForDocs includes some tests for the OSM API.
My first goal is to find the nearest OSM node to a given lat, lon. My
attempts seem to be made of fail thus far, however. Here's my code:
Most of the OSM dataset is converted into LineStrings, and what you really
want to do is find the closest vertex of the closest LineString. We have a
utility function 'findClosestEdges' in the SpatialTopologyUtils class for
that. The unit tests in TestSpatialUtils, and the testSnapping() method in
particular, show use of this.
My thinking is that nodes should be represented as points, so I can't
see why this fails. When I run this in a REPL, I do get a node back. So
far so good. Next, I want to get the node's tags. So I run:
The spatial search will return 'geometries', which are spatial objects. In
neo4j-spatial every geometry is represented by a unique node, but it is not
required that that node contain coordinates or tags. That is up to the
GeometryEncoder. In the case of the OSM model, this information is
elsewhere, because of the nature of the OSM graph, which is a highly
interconnected network of points, most of which do not represent Point
geometries, but are part of much more complex geometries (streets, regions,
buildings, etc.).
n.getSingleRelationship(OSMRelation.TAGS, Direction.INCOMING)
The geometry node is not connected directly to the tags node. You need two
steps to get there. But again, rather than figure out the graph yourself,
use the API. In this case, instead of getting the geometry node from the
SpatialDatabaseRecord, rather just get the properties using getPropertyNames
and getProperty(String). This API works the same on all kinds of spatial
data, and in the case of OSM data will return the TAGS, since those are
interpreted as attributes of the geometries.
n.getSingleRelationship(OSMRelationship.GEOM,
Direction.INCOMING).getOtherNode(n).getPropertyKeys
I see what appears to be a series of tags (oneway, name, etc.) Why are
these being returned for OSMRelation.GEOM
[Neo4j] More spatial questions
Fortunately, recent changes seem to have made the memory leaks I was
experiencing a few weeks ago to vanish. Apologies for not playing a more
active part in these discussions, but I'm finding there to be a quite
steep learning curve here, and I don't have the time to make a major
push to overcome it at once. But I finally have an import that I can
play with in a Scala REPL, though I'm encountering things about it that
just don't make sense.
My first goal is to find the nearest OSM node to a given lat, lon. My
attempts seem to be made of fail thus far, however. Here's my code:
def layer = spatialService.getLayer(map).asInstanceOf[OSMLayer]
private def pointSearch(lat:Double, lon:Double) = new SearchClosest(
layer.getGeometryFactory.createPoint(
new Coordinate(lat, lon)
)
)
def nearestNode(lat:Double, lon:Double) = {
val query = pointSearch(lat, lon)
val l = layer
l.addSimpleDynamicLayer(Constants.GTYPE_POINT)
l.getIndex.executeSearch(query)
val results = query.getResults
results.headOption.map { r =
r.getGeomNode
}
}
My thinking is that nodes should be represented as points, so I can't
see why this fails. When I run this in a REPL, I do get a node back. So
far so good. Next, I want to get the node's tags. So I run:
n.getSingleRelationship(OSMRelation.TAGS, Direction.INCOMING)
I get null. OK, so maybe this doesn't have tags. Yet, when I run:
n.getSingleRelationship(OSMRelationship.GEOM,
Direction.INCOMING).getOtherNode(n).getPropertyKeys
I see what appears to be a series of tags (oneway, name, etc.) Why are
these being returned for OSMRelation.GEOM rather than OSMRelation.TAGS?
Additionally, I see the property way_osm_id, which clearly isn't a tag.
It would also seem to indicate that this query returned a way rather
than a node like I'd hoped. This conclusion is further born out by the
tag names. So clearly I'm not getting the search correct. But beyond
that, the way being returned by this search isn't close to the lat,lon I
provided. What am I missing?
As an aside, in looking at the latest OSM import testcase, it seems like
the batch inserter may now be optional. Is this true, and what
benefits/disadvantages are there to its use? I tried importing the Texas
OSM data on my fairly powerful laptop, but gave up after 12 hours and
17 way imports (I think there are over a million in that dataset.)
Other geospatial formats seem to do the import in a matter of hours, but
this import seemed like it'd go on for days if I let it.
Thanks.
___
Neo4j mailing list
User@lists.neo4j.org
https://lists.neo4j.org/mailman/listinfo/user
Re: [Neo4j] More spatial questions
Hi Nolan, I think I can answer a few of your questions. Firstly, some background. The graph model of the OSM data is based largely on the XML formated OSM documents, and there you will find 'nodes', 'ways', 'relations' and 'tags' each as their own xml-tag, and as a consequence each will also have their own neo4j-node in the graph. Another point is that the geometry can be based on one or more nodes or ways, and so we always create another node for the geometry, and link it to the osm-node, way or relation that represents that geometry. What all this boils down to is that you cannot find the tags on the geometry node itself. You cannot even find the location on that node. If you want to use the graph model in a direct way, as you have been trying, you really do need to know how the OSM data is modeled. For example, for a LineString geometry, you would need to traverse from the geometry node to the way node and finally to the tags node (to get the tags). To get to the locations is even more complex. Rather than do that, I would suggest that you work with the OSM API we provided with the OSMLayer, OSMDataset and OSMGeometryEncoder classes. Then you do not need to know the graph model at all. For example, OSMDataset has a method for getting a Way object from a node, and the returned object can be queried for its nodes, geometry, etc. Currently we provide methods for returning neo4j-nodes as well as objects that make spatial sense. One minor issue here is the ambiguity inherent in the fact that both neo4j and OSM make use of the term 'node', but for different things. We have various solutions to this, sometimes replacing 'node' with 'point' and sometimes prefixing with 'osm'. The unit tests in TestsForDocs includes some tests for the OSM API. My first goal is to find the nearest OSM node to a given lat, lon. My attempts seem to be made of fail thus far, however. Here's my code: Most of the OSM dataset is converted into LineStrings, and what you really want to do is find the closest vertex of the closest LineString. We have a utility function 'findClosestEdges' in the SpatialTopologyUtils class for that. The unit tests in TestSpatialUtils, and the testSnapping() method in particular, show use of this. My thinking is that nodes should be represented as points, so I can't see why this fails. When I run this in a REPL, I do get a node back. So far so good. Next, I want to get the node's tags. So I run: The spatial search will return 'geometries', which are spatial objects. In neo4j-spatial every geometry is represented by a unique node, but it is not required that that node contain coordinates or tags. That is up to the GeometryEncoder. In the case of the OSM model, this information is elsewhere, because of the nature of the OSM graph, which is a highly interconnected network of points, most of which do not represent Point geometries, but are part of much more complex geometries (streets, regions, buildings, etc.). n.getSingleRelationship(OSMRelation.TAGS, Direction.INCOMING) The geometry node is not connected directly to the tags node. You need two steps to get there. But again, rather than figure out the graph yourself, use the API. In this case, instead of getting the geometry node from the SpatialDatabaseRecord, rather just get the properties using getPropertyNames and getProperty(String). This API works the same on all kinds of spatial data, and in the case of OSM data will return the TAGS, since those are interpreted as attributes of the geometries. n.getSingleRelationship(OSMRelationship.GEOM, Direction.INCOMING).getOtherNode(n).getPropertyKeys I see what appears to be a series of tags (oneway, name, etc.) Why are these being returned for OSMRelation.GEOM rather than OSMRelation.TAGS? These are not the tags. Now you have found the node representing an OSM 'Way'. This has a few properties on it that are relevant to the way, the name, whether the street is oneway or not, etc. Sometimes these are based on values in the tags, but they are not the tags themselves. This node is connected to the geometry node and the tags node, so you were half-way there (to the tags that is). You started at the geometry node, and stepped over to the way node, and one more step (this time with the TAGS relationship) would have got you to the tags. But again, I advise against trying to explore the OSM graph by itself. As you have already found, it is not completely trivial. What you should have done is access the attributes directly from the search results. Additionally, I see the property way_osm_id, which clearly isn't a tag. It would also seem to indicate that this query returned a way rather than a node like I'd hoped. This conclusion is further born out by the tag names. So clearly I'm not getting the search correct. But beyond that, the way being returned by this search isn't close to the lat,lon I provided. What am I missing? The lat/long values are quite a bit deeper in the graph. In the case
