[OSM-dev] Simpler binary OSM formats

2015-04-28 Thread andrew byrd
decode. This work is presented in an article at http://conveyal.com/blog/2015/04/27/osm-formats/. I welcome any comments you may have on this article or on the potential for a shift to simpler binary OSM formats. Regards, Andrew Byrd ___ dev mailing list dev

Re: [OSM-dev] Simpler binary OSM formats

2015-04-29 Thread andrew byrd
On Wed, Apr 29, 2015, at 07:03, Paul Norman wrote: > How does it work with diffs and history files? These features are not implemented, as this is a prototype format whose main purpose so far is to seed a discussion. The way OSM diffs are usually handled (full information is given for modified ent

Re: [OSM-dev] Simpler binary OSM formats

2015-04-29 Thread andrew byrd
Thanks for your comments Jochen. Responses in-line below. On Wed, Apr 29, 2015, at 11:18, Jochen Topf wrote: > On Wed, Apr 29, 2015 at 01:35:29AM +0200, andrew byrd wrote: ... it > has some nice properties we shouldn't forget. First and foremost that > is the block structur

Re: [OSM-dev] Simpler binary OSM formats

2015-04-29 Thread andrew byrd
On Wed, Apr 29, 2015, at 11:45, François Battail wrote: > with liprotobuf-c.so, default allocator, without assembly support: > 799s (bandwidth: 33.9 MiB/s) with liprotobuf-c.so, sw_pool_t > allocator, little assembly support: 629s (bandwidth: 43.1 MiB/s)" Point taken -- in future comparisons I wil

Re: [OSM-dev] [osmosis-dev] Proposal for a multithreaded PBF reader

2015-06-04 Thread Andrew Byrd
data once the PBF is decoded can be quite CPU intensive, but that would imply buffering decoded data and parallelizing geometric operations for example, not the reading. I’d appreciate any data points and example use cases you might have, as I’m currently working on related tooling. Andrew Byrd

[OSM-dev] new Java OSM loading / storage library

2015-06-30 Thread Andrew Byrd
geographic extracts on demand. Spatial indexing is currently limited to ways and the nodes referenced by those ways, i.e. “loose” POI nodes are not indexed. This is simply because our main use case is routing along ways. Regards, Andrew Byrd [1] https://lists.openstreetmap.org/pipermail/dev

Re: [OSM-dev] new Java OSM loading / storage library

2015-06-30 Thread Andrew Byrd
/com/conveyal/osmlib/main/Converter.java -Andrew > On 01 Jul 2015, at 00:08, Andrew Byrd wrote: > Conveyal has created an open source Java OSM library for use in our own > projects (including OpenTripPlanner). It is now in a usable pre-release > state, and some of you might be i

Re: [OSM-dev] setting up mirror

2015-11-18 Thread Andrew Byrd
Hello, I think we should clarify some terms here. The word “mirror” usually refers to a secondary copy of some very large or very heavily used resource, generally at a different physical location or network link than the primary “official” copy. Its purpose is to reduce load on the main server

Re: [OSM-dev] OSM PBF and spatial characteristics of blocks

2016-01-05 Thread Andrew Byrd
yal/vanilla-extract> https://github.com/conveyal/osm-lib/blob/master/src/main/java/com/conveyal/osmlib/VanillaExtract.java <https://github.com/conveyal/osm-lib/blob/master/src/main/java/com/conveyal/osmlib/VanillaExtract.java> Regards, Andrew Byrd > On 05 Jan 2016, at 18:40, Stadin, Ben

Re: [OSM-dev] OSM PBF and spatial characteristics of blocks

2016-01-05 Thread Andrew Byrd
> On 05 Jan 2016, at 19:18, Paul Norman wrote: > > Both of these seem to be software rather than file format definitions. I had > a quick look around the repo, but I couldn't find a format definition except > that for standard OSM PBF. Hi Paul, They are indeed software, both of which store pl

Re: [OSM-dev] OSM PBF and spatial characteristics of blocks

2016-01-06 Thread Andrew Byrd
> On 06 Jan 2016, at 03:40, Stadin, Benjamin > wrote: > Does your Vanilla Extract consider overlapping polygons? Like if you export a > small area within a country, does it add the country's polygon that overlaps > the area? > It looks pretty interesting though. I'm not sure where to start at

Re: [OSM-dev] OSM PBF and spatial characteristics of blocks

2016-01-06 Thread Andrew Byrd
> On 06 Jan 2016, at 13:13, Andrew Byrd wrote: > Obviously in the long term we’ll want to improve the index to handle these > cases. Both of these limitations should be straightforward to overcome. To > index large polygons as areas and you’d either need some kind of multi-le

Re: [OSM-dev] OSM PBF and spatial characteristics of blocks

2016-01-06 Thread Andrew Byrd
On 06 Jan 2016, at 13:56, Stadin, Benjamin wrote: > The problem I have to solve is that I need to be able to export also to other > projections. For our indoor navigation and accurate site maps we have to use > a proper projection. Thus my idea to index WGS84 datum, and enble to index > oberla

Re: [OSM-dev] OSM PBF and spatial characteristics of blocks

2016-01-06 Thread Andrew Byrd
> On 06 Jan 2016, at 14:10, Stadin, Benjamin > wrote: > > And about the cell data: I'm considering to just reuse OSM pbf format, > without preserving sort and size attributes. When exporting the data from > individual grid cells, all data items will be streamed to the output ordered > by typ

Re: [OSM-dev] Simpler binary OSM formats

2016-02-06 Thread Andrew Byrd
mments you may have on the revised format and the potential for a shift to simpler binary OSM formats. Regards, Andrew Byrd > On 29 Apr 2015, at 01:35, andrew byrd wrote: > > Hello OSM developers, > > Over the last few years I have worked on several pieces of software that >

Re: [OSM-dev] Simpler binary OSM formats

2016-02-07 Thread Andrew Byrd
a high adoption rate already. > > I think that it's well worth to reconsider creating an own file format and > parser for several reasons. Your concept looks well thought, it should be > possible to implement a lighweight parser using FlatBuffers for your data > scheme. >

Re: [OSM-dev] Simpler binary OSM formats

2016-02-08 Thread Andrew Byrd
> On 07 Feb 2016, at 20:10, Дмитрий Киселев wrote: > > As for fixed sized blocks in vex, I did consider that option but couldn’t > come up with a compelling reason for it. I can see the case for a maximum > block size (so we know what the maximum size of allocation will be), but can > you giv

Re: [OSM-dev] Simpler binary OSM formats

2016-02-08 Thread Andrew Byrd
> On 08 Feb 2016, at 10:57, Andrew Byrd wrote: > To me, it seems much more advantageous to provide a table of file offsets > stating where each entity type begins. I have already considered adding this > to vex after the basic format is established (like author metadata and map &

Re: [OSM-dev] Simpler binary OSM formats

2016-02-08 Thread Andrew Byrd
> > I think that it's well worth to reconsider creating an own file format and > parser for several reasons. Your concept looks well thought, it should be > possible to implement a lighweight parser using FlatBuffers for your data > scheme. > > Regards > Ben

Re: [OSM-dev] Simpler binary OSM formats

2016-02-16 Thread Andrew Byrd
Hi Colin, > On 08 Feb 2016, at 13:07, Colin Smale wrote: > There are discussions going on which may change the underlying data > metamodel. I am thinking of support for polygons/areas as primitive types and > multi-valued keys. Although the model has been stable since API0.6 it would > not be

Re: [OSM-dev] Spatial road network data sets and implementation using API

2017-03-27 Thread Andrew Byrd
Hi Debajyoti, I think you’ll need to provide more detailed information on your goals and the problems you are encountering for people to help you. Below I’ll list some of the questions that came to my mind in hopes that it will help clarify things, as well as some suggestions. > On 28 Mar 2017

Re: [OSM-dev] Spatial road network data sets and implementation using API

2017-03-28 Thread Andrew Byrd
> On 28 Mar 2017, at 14:46, Debajyoti Ghosh <4u.debajy...@gmail.com> wrote: > I need spatial data sets in this format > for each POIs need Both types of data are available in OSM. If the (start node, end node, distance) tuples you want are for individual edges in the graph (rather than dis

Re: [OSM-dev] Spatial road network data sets and implementation using API

2017-03-28 Thread Andrew Byrd
> On 28 Mar 2017, at 15:16, Debajyoti Ghosh <4u.debajy...@gmail.com> wrote: > 1] Can u specifically refer a site for data where from i can dwnld & > visualize in triplet format and n POIs is > ? In which editor i'll be able to see ? OpenStreetMap data is good for this purpose. Many people prov

Re: [OSM-dev] pyosmium: osmium extract

2018-11-10 Thread Andrew Byrd
> On 11 Nov 2018, at 08:52, koji higuchi wrote: > Thanks for your information. > Problem is that osmium is not available for windows, and it seems too > difficult to install in linux The osmium command line tool is available as a pre-compiled software package on some popular linux distrib

Re: [OSM-dev] public transport stop_area relations: have you implemented for routing or another purpose?

2019-08-01 Thread Andrew Byrd
Hello, I work on public transport journey planning software. We use OSM data heavily, but primarily for the parts of journeys outside public transportation: the first- or last-mile or transfer segments. In my experience we rarely use OSM data directly for the transit part of routing. We use dat