Re: [OSM-dev] pbf2osm development has started [code to test!]

Hartmut Holzgraefe wrote: > took a bit to figure out that things do not work the protobuf-c version > that comes with Ubuntu Maverik. Which Version is this? Hot some trouble on debian squeeze as well: src/ > ./pbf2osm ~/saarland.osm.pbf error parsing member id of DenseNodes error parsing

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Mon, Oct 25, 2010 at 4:15 AM, Stefan de Konink wrote: > On Mon, 25 Oct 2010, Chris Browet wrote: > > Sure. I don't think the C -> C++ will be problematic. >>> >> >> > I have been thinking about 'hooks' so the defines can become hooks you > replace (So somesort of ifndef way.) Or just inline f

Re: [OSM-dev] pbf2osm development has started [code to test!]

> > > If you'd accept some related patches, it would greatly improve >> re-usability and trackability of pbm2osm... >> > > Obviously I do ;) Just hadn't the time to integrate it yet. And busy day a > head with flying ;) > > That's ok, no rush needed, here... ;-) ___

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Mon, 25 Oct 2010, Chris Browet wrote: Sure. I don't think the C -> C++ will be problematic. It might be, because a completely different library/code generation program is used. Anyway, I still need to look into that. Things that I must do for Merkaartor: - Take the logic out of main()

Re: [OSM-dev] pbf2osm development has started [code to test!]

> > > The current plan is also to equip mapnik with a pbf reader. (Mapnik devs > here at the Mentor Summit...) but that does require a C++ approach, which > might be beneficial for Merkaartor as well. So we could also team up > rewriting the code using the C++ protobuf for integration in Merkaartor

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Mon, 25 Oct 2010, Chris Browet wrote: Would you mind if I re-use your code to implement a pbf reader in Merkaartor? Not at all, I just hope that everything that is coded now is correct, and you are able to track 'potential' further improvements. The current plan is also to equip mapnik w

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Fri, Sep 24, 2010 at 22:10, Stefan de Konink wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA512 > > Hi all, > > > As far as I can see with a quick look we are in a state of functional. > Tweaking will be done of course, especially because this version is only > a pipe. > > cat test.pbf

Re: [OSM-dev] pbf2osm development has started [code to test!]

Thanks for contributing :) really gives stimulance to work more on it. Currently sitting on a couch at the Mentor Summit at Google. Some other patches came in as well,so i'll try to integrate everything after the show here :) Totally offtopic: everyone check out the MetaWritter GSoC project of

Re: [OSM-dev] pbf2osm development has started [code to test!]

On 09/24/2010 10:10 PM, Stefan de Konink wrote: As far as I can see with a quick look we are in a state of functional. [...] (you need to have protobuf-c installed, to compile it, see the Makefile for that) took a bit to figure out that things do not work the protobuf-c version that comes wi

Re: [OSM-dev] pbf2osm development has started [code to test!]

Plan is first to release native support for osm2pgsql. Stefan Op 18 okt 2010 om 11:59 heeft Peter Körner het volgende geschreven:\ Am 26.09.2010 17:14, schrieb Stefan de Konink: When this tool is polished enough (must include some bbox'ing then) we can think about osm2pbf. Are there a

Re: [OSM-dev] pbf2osm development has started [code to test!]

Am 26.09.2010 17:14, schrieb Stefan de Konink: When this tool is polished enough (must include some bbox'ing then) we can think about osm2pbf. Are there any plans towards this? Will osm2pbf support the visible-flag? I tried to process the full-experimental dump into .osm.pbf using osmosis but

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Thu, Sep 30, 2010 at 00:47, Stefan de Konink wrote: > Roeland wanted to go for a library so other tools could call getNextNode() > etc. without fiddling with the internal structures. Is anyone working on such a thing? I could help. Zbynek ___ dev m

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Thu, 30 Sep 2010, Scott Crosby wrote: From what I observe now the bottleneck seems to be actually protocol buffers, while my output code can become slightly faster. This would imply that doing binary changesets isn't a critical necessity. You asked where the bottleneck is, for XML generat

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Thu, Sep 30, 2010 at 6:04 AM, Stefan de Konink wrote: > On Thu, 30 Sep 2010, Scott Crosby wrote: > > I proposed exactly this for the mkgmap splitter. If you're going to do >> this, >> can I propose a tweak where you can output thousands of files >> simultaneously? The differences are minor: I

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Thu, 30 Sep 2010, Scott Crosby wrote: I proposed exactly this for the mkgmap splitter. If you're going to do this, can I propose a tweak where you can output thousands of files simultaneously? The differences are minor: Instead of tracking if a node/way/relation was output or was missed with

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Thu, Sep 30, 2010 at 4:44 AM, Stefan de Konink wrote: > > > > - allow the use of the bbox > > > > > > What exactly do you mean by this: ? > > Filter output by applying a bbox (the expensive way). Extract nodes > within a bbox, sort them on ID, extract all ways, validate per item if > any o

Re: [OSM-dev] pbf2osm development has started [code to test!]

I implemented a PBF parser for MoNav. The speedup over XML and compressed XML is very good: It takes about 64 seconds to parse the German extract instead of 760 seconds. > But what I already discussed with Scott, we *need* a good 'has everything' > PBF file. Something that can test a parser and ha

Re: [OSM-dev] pbf2osm development has started [code to test!]

-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Op 30-09-10 06:30, Scott Crosby schreef: > Also Osmosis currently implements deflate, not bzip2/lzma. (At the > time of writing I implemented bzip2 as well, but couldn't test it.) > > > If I can find a lzma/bzip2 java code, its ready to plu

Re: [OSM-dev] pbf2osm development has started [code to test!]

> > > But what I already discussed with Scott, we *need* a good 'has everything' > PBF file. Something that can test a parser and has expected output. > > I know. I know. I need to hack a custom program that generates the entire protocol buffer, serializes it, hitting all of the interesting feature

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Wed, 29 Sep 2010, Anthony wrote: In addition to " and &, you need to escape <. planet.c also escapes . It uses character references for each (", &, <, and >). planet.c also escapes carriage return, line feed, and tab, as , , and . AFAICT it is legal to include these unescaped (though

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Wed, Sep 29, 2010 at 6:47 PM, Stefan de Konink wrote: > On Thu, 30 Sep 2010, Frederik Ramm wrote: >> Speaking of "polished": The program currently produces invalid XML because >> " and & are not escaped, leading to lines like > > Yes, Roeland pointed that out as well yesterday. We have discusse

Re: [OSM-dev] pbf2osm development has started [code to test!]

On Thu, 30 Sep 2010, Frederik Ramm wrote: Stefan de Konink wrote: When this tool is polished enough (must include some bbox'ing then) we can think about osm2pbf. Speaking of "polished": The program currently produces invalid XML because " and & are not escaped, leading to lines like Yes,

Re: [OSM-dev] pbf2osm development has started [code to test!]

Hi, Stefan de Konink wrote: When this tool is polished enough (must include some bbox'ing then) we can think about osm2pbf. Speaking of "polished": The program currently produces invalid XML because " and & are not escaped, leading to lines like Same for ampersand characters in user name

Re: [OSM-dev] pbf2osm development has started [code to test!]

-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi Peter, Op 26-09-10 17:00, Peter Körner schreef: > Am 24.09.2010 22:10, schrieb Stefan de Konink: >> Or the (static) 64bit version: >> http://mirror.openstreetmap.nl/pbf2osm64 > > p...@maps:~$ pv -cN 'pbf-input' /osm/data/planet-extract/hessen.os

Re: [OSM-dev] pbf2osm development has started [code to test!]

Am 24.09.2010 22:10, schrieb Stefan de Konink: Or the (static) 64bit version: http://mirror.openstreetmap.nl/pbf2osm64 p...@maps:~$ pv -cN 'pbf-input' /osm/data/planet-extract/hessen.osm.pbf | ./pbf2osm64 | pv -cN 'xml-output' > /osm/data/planet-extract/hessen.osm.xml pbf-input: 43.1MB 0:00:

Re: [OSM-dev] pbf2osm development has started [code to test!]

-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Op 24-09-10 23:20, Anthony schreef: > Some quick notes... > > Had to create ../OSM-binary/src > Had to download the .proto files and move them into ../OSM-binary/src > Had to make ../external and ../external/lib > > Make sure you have the newest ve

Re: [OSM-dev] pbf2osm development has started [code to test!]

Some quick notes... Had to create ../OSM-binary/src Had to download the .proto files and move them into ../OSM-binary/src Had to make ../external and ../external/lib Make sure you have the newest version of protobuf-c. After that, I was able to compile everything. The 64bit compiled version wor

Re: [OSM-dev] pbf2osm development has started [code to test!]

-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi all, As far as I can see with a quick look we are in a state of functional. Tweaking will be done of course, especially because this version is only a pipe. cat test.pbf | pbf2osm > output.osm You can check it out at: