On Wed, 2012-02-15 at 11:17 +0100, Marco Pizzoli wrote: > > > On Wed, Feb 15, 2012 at 9:47 AM, Jan Zelený <[email protected]> > wrote: > > Hi guys, > > while working of my thesis I've notice two features of > talloc that we > > might use to make sssd faster with only a little effort. > > > > The first one is about decreasing malloc() calls. We should > certainly > > do some measures here but I don't doubt that malloc() is a > big issue in > > sssd. We use it almost everywhere + we use it indirectly > with talloc > > which is about 10% slower (manpage). > > > > Samba team run to this issue as well and therefore they > created a new > > context type - talloc pool. The documentation says: > > > > "A talloc pool is a pure optimization for specific > situations. In the > > release process for Samba 3.2 we found out that we had > become > > considerably slower than Samba 3.0 was. Profiling showed > that malloc(3) > > was a large CPU consumer in benchmarks. For Samba 3.2 we > have > > internally converted many static buffers to dynamically > allocated ones, > > so malloc(3) being beaten more was no surprise. But it made > us slower." > > > > The great thing about this is that it is completely > transparent. The > > context can be created with talloc_pool(ctx, size) and it > works this > > way: > > - if we are allocating data on a pool context, it takes the > desirable > > amount of space from the pool, > > - if the context is a descendant of the pool context, it > takes the space > > from the pool as well, > > - if the pool is out of memory, it creates a new, normal > context, > > - if we change the parent of a child of talloc pool to a > parent that is > > outside of this pool, the whole pool memory will not be > freed until > > the child is freed. > > > > Example: > > /* allocate 1KiB in a pool */ > > TALLOC_CTX *pool_ctx = talloc_pool(NULL, 1024); > > > > /* take 512B from the pool, 512B left */ > > void *ptr = talloc_size(pool_ctx, 512); > > > > /* 1024B > 512B, this will create new talloc chunk */ > > void *ptr2 = talloc_size(ptr, 1024); > > > > /* the pool still contains 512 free bytes > > * this will take 200B from them */ > > void *ptr3 = talloc_size(ptr, 200); > > > > /* this will destroy context 'ptr3' but the memory > > * is not freed, the available space in the pool > > * will increase to 512B */ > > talloc_free(ptr3); > > > > /* this works as usual :) */ > > talloc_free(pool_ctx); > > > > Basically we can create large pool context for each > responder/dp request > > and create temporary context on the pool context or its > descendants. > > That's all that would need to change. > > > > I will send a description of the other feature in another > mail. > > > This is a very interesting feature. I agree that one possible > scenario would > be to allocate all tmp context on top of the pool but there > are also different > scenarios which would benefit from this feature. > > For example there are several places in the code where we do > realloc in the > cycle and each time we increase the size by one. These would > be among the first > parts of the code I would look into. If we decided to use > this, I would also > like to investigate the memberof plugin, IIRC there are quite > a few places > with cycles like this. > > Hi guys, > sorry for the possibly noise, but I'm interested to know if you > already considered the use of tcmalloc from Google. > If yes, I'm interested in your considerations in choosing to not use > it.
It's possible that at some point we may look into that to replace the standard malloc() that goes on under the hood of talloc. Talloc isn't just a memory allocator but is also a memory manager. It maintains memory in a directed graph structure such that if you free a node from the graph, all children of that node are also freed (with the added benefit of being able to run destructors immediately before the free()). I wrote up a more detailed explanation of why we chose talloc here: http://sgallagh.wordpress.com/2010/03/17/why-you-should-use-talloc-for-your-next-project/ or http://goo.gl/Jmsh6 if that gets ruined by mailman.
signature.asc
Description: This is a digitally signed message part
_______________________________________________ sssd-devel mailing list [email protected] https://fedorahosted.org/mailman/listinfo/sssd-devel
