On Sat, Sep 25, 2010 at 2:51 AM, Christian H. Bruhn <br...@arcor.de> wrote:
> Friday, September 24, 2010, 6:06:59 PM, Scott Crosby wrote: > > > Is there an error message thrown? If so, please paste it. > > No error message, everything seems to be OK. > > > If the error message is something about a possibly corrupt file..... I'm > > aware of the issue. To detect bad files and fail gracefully, I put in > some > > size limits that I later found might be too small. The patch with the > > increased limits is waiting for me to push out a new jar. > > I've just tested it with the planet-100922.osm.bz2 (MD5 checked) and > 20100922-20100923.osc.gz. All osmosis tasks run with the -v option > under Windows 7 Prof. 64 Bit with osmosis-snapshot 0.37 from > 23.09.2010. > I'm not replicating the problem. I'm running under my working tree. The only relevant differences are increased maximum blocksize limits. Here is what I ran (under ubuntu linux. pbzip2 is a parallel bzip2 decompressor, which is then buffered with mbuffer.): ## I converted the planet twice, once for faster writing, one not. pbzip2 -d < foo/planet-100922.osm.bz2.1 | ~/source/Map2/osmosis/package/bin/osmosis --read-xml file=- --lp --b bufferCapacity=12000 --write-bin file=planet.osm.pbf compress=none pbzip2 -d < foo/planet-100922.osm.bz2.1 | ~/source/Map2/osmosis/package/bin/osmosis --read-xml file=- --lp --b bufferCapacity=16000 --write-bin file=planet-default.osm.pbf ## Resulting sizes are as expected ## I then tried merging changesets. ~/source/Map2/osmosis/package/bin/osmosis --read-xml-change file=20100922-20100923.osc.gz --read-bin file=planet.osm.pbf --b bufferCapacity=12000 --apply-change --lp --b bufferCapacity=12000 --write-bin file=out.osm.pbf ~/source/Map2/osmosis/package/bin/osmosis --read-xml-change file=20100922-20100923.osc.gz --read-bin file=planet-default.osm.pbf --b bufferCapacity=12000 --apply-change --lp --b bufferCapacity=12000 --write-bin file=out-default.osm.pbf pbzip2 -d <foo/planet-100922.osm.bz2.1 | ~/source/Map2/osmosis/package/bin/osmosis --read-xml-change file=20100922-20100923.osc.gz --read-xml file=- --lp --b bufferCapacity=12000 --apply-change --lp --b bufferCapacity=12000 --write-bin file=planet-xmlin.osm.pbf ## All three run to completion, returning identical output files of about the expected sizes: 8835376 -rw-r--r-- 1 scrosby scrosby 9047420819 Sep 28 11:00 out-default.osm.pbf 8835376 -rw-r--r-- 1 scrosby scrosby 9047420819 Sep 28 10:14 out.osm.pbf 8835376 -rw-r--r-- 1 scrosby scrosby 9047420819 Sep 28 12:06 planet-xmlin.osm.pbf 8824112 -rw-r--r-- 1 scrosby scrosby 9035884739 Sep 28 10:37 planet-default.osm.pbf 18576056 -rw-r--r-- 1 scrosby scrosby 19021875191 Sep 28 10:12 planet.osm.pbf Is your problem repeatable? If it is repeatable, one explanation is that a 'too big' block was generated, and somehow the warning was missed. That can be tested by running a smaller batchlimit. Can you try a '--write-bin file=FOO batchlimit=2000' on the initial conversion from *.osm to *.osm.pbf and the later --read-xml-change tests and see if the problem goes away? If so, please report the result to help me fine-tune the block size and batch limit settings. Scott
_______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/listinfo/dev
