On Wed, Jul 30, 2008 at 1:24 PM, Brett Henderson <[EMAIL PROTECTED]> wrote:
> 2008/7/26 Karl Newman <[EMAIL PROTECTED]> > >> >> Well, I ran it for a while until it failed because of lack of heap space >> (I had -Xmx512M for the Java command but it wasn't enough for the >> ListIdTracker). But regardless, it already generated a 18.6 GB uncompressed >> OSM file (and that's only nodes), so I suspect there's something wrong with >> the creation of your segment_earth.osm. Make sure it's not failing silently. >> >> A few optimizations you could use: >> >> 1. Use planet.gz, not bz2 because Osmosis is much faster at gzip (it >> has native code). >> 2. Don't enable date parsing. >> 3. Add the parameter "idTrackerType=BitSet" to the bounding box task. >> It's more efficient for large bounding boxes. >> >> Karl >> > > Apologies for the slow reply, I've had a lot on. > > I had similar issues with out of memory errors and had to switch to the > BitSet id tracker. I ended up with a 3.25 GB gzip compressed file for the > segment file. > > Another optimisation is to stop using the log progress task. I haven't > tried running with and without logging recently but I suspect it adds a lot > of overhead. There's probably a much smarter way of doing this but the > current implementation isn't very efficient. > > I'll run the second step now. > > I've completed the New York extraction from the segment file and created a 71.3MB gzip compressed file, that will be approaching 1GB uncompressed. I ran this second invocation with idTrackerType=IdList due to the much smaller number of nodes in the result. It all seems to be working okay at my end. All I can think of is that the osmosis process failed unexpectedly due to an out of memory error. The files being created were too small.
_______________________________________________ dev mailing list dev@openstreetmap.org http://lists.openstreetmap.org/cgi-bin/mailman/listinfo/dev
