Hi Alexey, I have run into similar issues (exorbitantly long wait times for the malloc mutex). The only way I could reduce the pain was to either reduce the number of threads, or reduce the amount of malloc/free() done by the application.
If you're using a library (like this xml one) it may be hard to reduce the number of malloc/frees() actually done. Can you try getting a full set of stack traces on the next stall (for all threads?). I have a feeling that your particular stack trace may be held up behind an arena_flush() but it'd be good to know for sure. On Tue, Jan 21, 2014 at 5:46 AM, Alexey Kuts <[email protected]> wrote: > Hi, > > sometimes (maybe once per day) we have > an annoying hang in jemalloc(). > > jemalloc-version: 3.4.1-0-g0135fb806e4137dc9cdf152541926a2bc95e33f0 > Ubuntu 12.04.3 LTS, 3.2.0-33-generic x86_64 GNU/Linux > > > #0 0x00007f6bd681a89c in __lll_lock_wait () from /lib/x86_64-linux-gnu/ > libpthread.so.0 > #1 0x00007f6bd6816080 in _L_lock_903 () from /lib/x86_64-linux-gnu/ > libpthread.so.0 > #2 0x00007f6bd6815f19 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/ > libpthread.so.0 > #3 0x0000000000459127 in malloc_mutex_lock (mutex=0x7f6bd4400a08) at > include/jemalloc/internal/mutex.h:77 > #4 tcache_bin_flush_small (tbin=<optimized out>, binind=15, rem=3, > tcache=0x7f6bd4006000) at src/tcache.c:105 > #5 0x000000000045958d in tcache_event_hard (tcache=0x7f6bd4006000) at > src/tcache.c:39 > #6 0x0000000000437ebe in tcache_event (tcache=0x7f6bd4006000) at > include/jemalloc/internal/tcache.h:271 > #7 tcache_alloc_large (size=<optimized out>, tcache=<optimized out>, > zero=<optimized out>) at include/jemalloc/internal/tcache.h:383 > #8 arena_malloc (zero=false, size=<optimized out>, arena=0x0, > try_tcache=true) at include/jemalloc/internal/arena.h:944 > #9 imallocx (arena=0x0, try_tcache=true, size=<optimized out>) at > include/jemalloc/internal/jemalloc_internal.h:798 > #10 imalloc (size=<optimized out>) at include/jemalloc/internal/ > jemalloc_internal.h:807 > #11 jemalloc (size=<optimized out>) at src/jemalloc.c:887 > #12 0x00007f6bd6d3bad8 in xmlStrndup () from /usr/lib/x86_64-linux-gnu/ > libxml2.so.2 > #13 0x00007f6bd6ce3029 in xmlNodeListGetString () from > /usr/lib/x86_64-linux-gnu/libxml2.so.2 > #14 0x00007f6bd6d0038f in xmlValidateElement () from > /usr/lib/x86_64-linux-gnu/libxml2.so.2 > #15 0x00007f6bd6d0034e in xmlValidateElement () from > /usr/lib/x86_64-linux-gnu/libxml2.so.2 > #16 0x00007f6bd6d0034e in xmlValidateElement () from > /usr/lib/x86_64-linux-gnu/libxml2.so.2 > #17 0x00007f6bd6d00528 in xmlValidateDtd () from /usr/lib/x86_64-linux-gnu/ > libxml2.so.2 > #18 0x00007f6bd7005355 in xmlpp::DtdValidator::validate(xmlpp::Document > const*) () from /usr/lib/libxml++-2.6.so.2 > > Will appreciate any hints or suggestions > > -- > Kuts Alexey > _______________________________________________ > jemalloc-discuss mailing list > [email protected] > http://www.canonware.com/mailman/listinfo/jemalloc-discuss >
_______________________________________________ jemalloc-discuss mailing list [email protected] http://www.canonware.com/mailman/listinfo/jemalloc-discuss
