On Mon, Nov 21, 2011 at 12:07:42PM +0100, Peter Körner wrote: > I found a memory-leak in osmium-master. I compiled the > osmium_progress example and fed germany.osh.pbf > <http://ftp5.gwdg.de/pub/misc/openstreetmap/osm-full-history-extracts/110919/pbf/europe/germany.osh.pbf> > into it. > > During the nodes processing the program takes 70 MB of RAM. When it > starts to process Ways, the required memory raises constantly up to > 300 MB.
I don't think that there is a memory leak. Valgrind tells me that all is ok. The behaviour you see comes from the PBF reader and especially the Google Protobuf stuff. Protobuf does not free memory but keeps it around until the next time it is needed. So every time you hit a PBF block with "more" data in it than all the other blocks before (for instance more nodes per way), PBF will allocated more memory and not de-allocate it. So for a while you'll see the process slowly using more and more memory until it hits the peak. Only google::protobuf::ShutdownProtobufLibrary(); will release all the memory. Thats done at the end of the program. (Could be done at the end of the read, but for most programs thats the same as the end of the program anyway.) So there is no leak, all memory is accounted for. Jochen -- Jochen Topf [email protected] http://www.remote.org/jochen/ +49-721-388298 _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
