> It is equivalent, as long as you can ensure that _all_ allocations go > through your wrappered malloc.
For the purposes of the live-block-walker, does it matter if we miss the occasional malloc()/free()? On Sat, Oct 6, 2012 at 11:51 AM, Benoit Jacob <[email protected]> wrote: > It is equivalent, as long as you can ensure that _all_ allocations go > through your wrappered malloc. The terror of anyone making such a > wrappered malloc is that some allocations might escape it, such as: > allocations made by libraries that your application uses, etc. Not > being familiar with the field, I just found it easier to be confident > that all allocations would be correctly wrappered by doing it in the > memory allocator itself, since if any allocation did not go through > it, we would have allocator mismatches anyway. > > Here is an example of what scared me away from trying to do it in the > application: this is a comment in jemalloc 3.0's src/jemalloc.c: > > #if ((is_malloc(je_malloc) == 1) && defined(__GLIBC__) && > !defined(__UCLIBC__)) > /* > * glibc provides the RTLD_DEEPBIND flag for dlopen which can make it possible > * to inconsistently reference libc's malloc(3)-compatible functions > * (https://bugzilla.mozilla.org/show_bug.cgi?id=493541). > * > * These definitions interpose hooks in glibc. The functions are actually > * passed an extra argument for the caller return address, which will be > * ignored. > */ > JEMALLOC_EXPORT void (* const __free_hook)(void *ptr) = je_free; > JEMALLOC_EXPORT void *(* const __malloc_hook)(size_t size) = je_malloc; > JEMALLOC_EXPORT void *(* const __realloc_hook)(void *ptr, size_t size) = > je_realloc; > JEMALLOC_EXPORT void *(* const __memalign_hook)(size_t alignment, size_t > size) = > je_memalign; > #endif > > Benoit > > 2012/10/6 Salvatore Sanfilippo <[email protected]>: >> Hello, >> >> what is the advantage of this approach compared to doing it entirely >> in the application code just wrapping malloc/realloc/free? >> Basically wrappered_malloc() allocates a bit more space accordingly to >> the metadata to store, store this metadata at the start of the block, >> and then returns the pointer advanced to the start of the empty space. >> >> Doing it in the context of the application makes it >> malloc-implementation agnostic that can be an advantage. >> >> Regards, >> Salvatore >> >> On Sat, Oct 6, 2012 at 7:28 AM, Benoit Jacob <[email protected]> >> wrote: >>> Hello, >>> >>> The attached patch instruments jemalloc 3.0, adding the ability to >>> enumerate all live blocks. >>> >>> Currently, the only information stored about blocks is their (payload) >>> address and size, but the plan is to also store their allocation call >>> stack. >>> >>> This is achieved by allocating larger blocks than requested, and using >>> the extra space to store doubly linked list elements. >>> >>> This is provided just in case it might be useful to anyone, not >>> considered ready for inclusion in jemalloc. It's been tested for 15 >>> minutes in a Firefox build. >>> >>> Details about the overhead, and how it could be reduced: >>> * Memory overhead is, per block: 32 bytes on 32-bit systems, 48 bytes >>> on 64-bit systems (assuming size_t == uintptr_t). Could easily be >>> reduced to 16 bytes in both cases (by using a XOR linked list and >>> assuming that no block exceeds 4G). >>> * Slowness overhead is essentially an additional mutex to lock on >>> every malloc/free call. Could be solved in various ways, either >>> copying what jemalloc does internally, or by using a lock-free list. >>> >>> If you want to test it out, currently the only built-in way to output >>> the list of blocks is to call je_dump_list_of_blocks(void) e.g. from >>> your debugger. See its code to see the relevant calls. Sample output >>> from Firefox: >>> >>> ...snip... >>> Block 193965: real block = 0x7fffba218580, payload = 0x7fffba2185b0, >>> payload size = 64 >>> Block 193966: real block = 0x7fffc02dd000, payload = 0x7fffc02dd030, >>> payload size = 1024 >>> Block 193967: real block = 0x7fffc2053ce0, payload = 0x7fffc2053d10, >>> payload size = 24 >>> Block 193968: real block = 0x7fffc5b4ed80, payload = 0x7fffc5b4edb0, >>> payload size = 80 >>> Block 193969: real block = 0x7fffd119e240, payload = 0x7fffd119e270, >>> payload size = 64 >>> Block 193970: real block = 0x7fffcfb4aa60, payload = 0x7fffcfb4aa90, >>> payload size = 24 >>> Block 193971: real block = 0x7fffbb85e1f0, payload = 0x7fffbb85e220, >>> payload size = 24 >>> Block 193972: real block = 0x7fffe20a62e0, payload = 0x7fffe20a6310, >>> payload size = 32 >>> >>> End enumeration of 193973 jemalloc blocks. >>> >>> Cheers, >>> Benoit >>> >>> _______________________________________________ >>> jemalloc-discuss mailing list >>> [email protected] >>> http://www.canonware.com/mailman/listinfo/jemalloc-discuss >>> >> >> >> >> -- >> Salvatore 'antirez' Sanfilippo >> open source developer - VMware >> http://invece.org >> >> Beauty is more important in computing than anywhere else in technology >> because software is so complicated. Beauty is the ultimate defence >> against complexity. >> — David Gelernter > _______________________________________________ > jemalloc-discuss mailing list > [email protected] > http://www.canonware.com/mailman/listinfo/jemalloc-discuss _______________________________________________ jemalloc-discuss mailing list [email protected] http://www.canonware.com/mailman/listinfo/jemalloc-discuss
