This post has been prompted by issue 4531 and r1643834 (http://subversion.tigris.org/issues/show_bug.cgi?id=4531 http://svn.apache.org/viewvc?view=revision&revision=r1643834). We had a somewhat similar issue with log -v & friends in summer this year. Now, I dug into the APR code and this what I found.
The _debug varieties of the APR pool functions don't use the allocator nor chunky allocation (8k+ at a time) but plain malloc / free instead and traces them individually. That means: * On 64 bits, each alloc has a 4x8=32 bytes extra overhead for typical CRT implementations. Memory usage can be expected to double for everything that's not a delta window or fulltext in SVN. * Our 4MB limit of unused memory on allocators is ineffective. On platforms that use MMAP in APR allocators, that can be a significant difference. Everywhere else, it may or may not be. * Since we interleave allocations from use scratch and result pools, we can expect high levels of fragmentation. This will most likely prevent memory from being freed after peak usage. It may also increase peak memory usage. IOW, pool debugging is nice for tracing allocations but if you want to measure memory consumption on the OS side, turn pool debugging off. APR pools also replace the usually desired malloc / free symmetry with a pool create / clear|destroy symmetry. The individual PALLOC / PCALLOC lines in the pool usage trace will give you context but the things one is interested in are CREATE, CLEAR and DESTROY lines. Finally, to minimize cache usage, make sure to disable fulltext caching as well (enabled by default in 1.9) and set the cache size to *1*, not 0. The latter would fall back to 1.6-style caches, keeping a fixed number of objects stored in the svn_fs_t, e.g. 8192 full directories. With ra_serf, a single large dir higher up in the tree is sufficient to blow get up to 100+ MB because it will be requested in 100s or 1000s of different revisions. Membuffer cache, OTOH, does not allocate memory after its creation. All cache-related allocations use the scratch and result pools provided by the caller to (de-)serialize cached data. The cache memory itself is fixed and objects that are too large to fit in will simply be ignored. -- Stefan^2.
