Back in the bug #14231 thread [1], dealing with performance issues in reorderbuffer due to excessive number of expensive free() calls, I've proposed to resolve that by a custom slab-like memory allocator, suitable for fixed-size allocations. I'd like to put this into the next CF, and it's probably too invasive change to count as a bugfix anyway.

[1] https://www.postgresql.org/message-id/flat/20160706185502.1426.28143%40wrigleys.postgresql.org

This patch actually includes two new memory allocators (not one). Very brief summary (for more detailed explanation of the ideas, see comments at the beginning of slab.c and genslab.c):

* suitable for fixed-length allocations (other pallocs fail)
* much simpler than AllocSet (no global freelist management etc.)
* free space is tracked per block (using a simple bitmap)
* which allows freeing the block once all chunks are freed (AllocSet will hold the memory forever, in the hope of reusing it)

* suitable for non-fixed-length allocations, but with chunks of mostly the same size (initially unknown, the context will tune itself)
* a combination AllocSet and Slab (or a sequence of Slab allocators)
* the goal is to do most allocations in Slab context
* there's always a single 'current' Slab context, and every now and and then it's replaced with a new generation (with the chunk size computed from recent requests)
* the AllocSet context is used for chunks too large for current Slab

So none of this is meant as a universal replacement of AllocSet, but in the suitable cases the results seem really promising. For example for the simple test query in [1], the performance improvement is this:

        N |   master |  patched
    10000 |    100ms |    100ms
    50000 |  15000ms |    350ms
   100000 | 146000ms |    700ms
   200000 |        ? |   1400ms

That's a fairly significant improvement, and the submitted version of the patches should perform even better (~2x, IIRC).

There's a bunch of TODOs - e.g. handling of realloc() calls in the GenSlab, and probably things I haven't thought about.


Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment: 0001-simple-slab-allocator-fixed-size-allocations.patch
Description: binary/octet-stream

Attachment: 0002-generational-slab-auto-tuning-allocator.patch
Description: binary/octet-stream

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to