Hi, On Sat, May 7, 2011 at 5:46 AM, Scott Crosby <[email protected]> wrote: > On Fri, May 6, 2011 at 9:24 PM, Christian Vetter <[email protected]> > wrote: >> With regard to LZMA: I have some C++ code lying around to compress / >> decompress LZMA... I can test how much it would affect file size / >> decoding speed. > > Cool. You don't need a full-fledged PBF reader&writer to test it. Just > enough to parse out blobs and write blobs.
I quickly hacked it into MoNav's importer and tested it on the extract of Germany. I used maximum compression ( dictionary size == blob size ): size of zlib blobs: 849MB size of lzma blobs: 762MB time spent decoding zlib blobs: 6.986 s time spent decoding lzma blobs: 49.078 s We can reduce the size a bit by using lzma ( ~10% ) and adding it isn't much work ( about 10 lines of code for encoding / decoding ). However, it doesn't seem worth it, considering that it makes parsing slower. Increasing the block size would most likely not increase the compression: I tested compressing all uncompressed blobs at once using a 64MB dictionary and the size only decreased to 728MB Regards, Christian Vetter _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
