On 10/19/2016 02:51 PM, Tomas Vondra wrote:
Yeah. There are three contexts in reorder buffers:

- changes (fixed size)
- txns (fixed size)
- tuples (variable size)

The first two work perfectly fine with Slab.

The last one (tuples) is used to allocate variable-sized bits, so I've
tried to come up with something smart - a sequence of Slabs + overflow
AllocSet. I agree that in hindsight it's a bit strange, and that the
"generational" aspect is the key aspect here - i.e. it might be possible
to implement a memory context that allocates variable-length chunks but
still segregates them into generations. That is, don't build this on top
of Slab. That would also fix the issue with two allocators in GenSlab.
I'll think about this.

And here is a fairly complete prototype of this idea, adding "Gen" generational memory context based only on the "similar lifespan" assumption (and abandoning the fixed-size assumption). It's much simpler than GenSlab (which it's supposed to replace), and abandoning the idea of composing two memory contexts also fixed the warts with some of the API methods (e.g. repalloc).

I've also been thinking that perhaps "Gen" would be useful for all three contexts in ReorderBuffer - so I've done a quick test comparing the various combinations (using the test1() function used before).

         master       slabs   slabs+genslab   slabs+gen     gens
   50k      18700       210             220         190      190
  100k     160000       380             470         350      350
  200k        N/A       750             920         740      679
  500k        N/A      2250            2240        1790     1740
 1000k        N/A      4600            5000        3910     3700


* master - 23ed2ba812117
* slabs  - all three contexts use Slab (patch 0001)
* slabs+genslab - third context is GenSlab (patch 0002)
* slabs+gen - third context is the new Gen (patch 0003)
* gens - all three contexts use Gen

The results are a bit noisy, but I think it's clear the new Gen context performs well - it actually seems a bit faster than GenSlab, and using only Gen for all three contexts does not hurt peformance.

This is most likely due to the trivial (practically absent) freespace management in Gen context, compared to both Slab and GenSlab. So the speed is not the only criteria - I haven't measured memory consumption, but I'm pretty sure there are cases where Slab consumes much less memory than Gen, thanks to reusing free space.

I'd say throwing away GenSlab and keeping Slab+Gen is the way to go.

There's still a fair bit of work on this, particularly implementing the missing API methods in Gen - GenCheck() and GenStats(). As Robert pointed out, there's also quite a bit of duplicated code between the different memory contexts (randomization and valgrind-related), so this needs to be moved to a shared place.

I'm also thinking that we need better names, particularly for the Gen allocator. It's supposed to mean Generational, although there are no explicit generations anymore. Slab is probably OK - it does not match any of the existing kernel slab allocators exactly, but it follows the same principles, which is what matters.


Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment: slab-allocators-v4.tgz
Description: application/compressed-tar

Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to