On Mon, Jun 23, 2014 at 11:55 PM, Stefan Sperling <s...@elego.de> wrote:
> I'm still seeing 180MB of memory used by svn log -v with ruby's r11708 > over ra_local with fsfs7-unpacked (120MB with fsfs6-unpacked). > The total output generated by svn log is only 5.5MB in size. > That seems somewhat disproportionate. > Yes, it is. The id_t struct is somewhat heavy but in total a change should be less than 200 bytes. So, everything that needs more than ~30MB for the ruby example is suspicious. > Of course, I'm happy that this invocation of svn log isn't running > out of memory anymore on my machine, as it did before r1604188! > So that's a huge step forward but I suppose we can still do better. > > It seems caching is partly responsible for the memory growth. > I'm seeing huge jumps during svn_cache__set() calls, though we might > still be missing some scratch pools or iterpools to keep memory > usage low. > The serializer itself uses a single, auto-growing buffer. It should show up as a temporary peak, though. The current 100 bytes estimation / change is way to low with the basic struct already taking 144 bytes on a 64bit system. r1604947 should help here. > Does anyone know what can be done about this? > Use a similar estimate as in r1604947 to skip svn_cache__set for uncachable items (call svn_cache__is_cachable). As soon as I arrived in UK, I'll look into this. It's probably at the point now where I need to counting bytes at every step to spot inappropriate memory usage. -- Stefan^2.
