our case is always single threaded (it-is mapreduce-job).

22.01.2014 21:30, John Esmet пишет:
Hi Alexey,

I have run into similar issues (exorbitantly long wait times for the
malloc mutex). The only way I could reduce the pain was to either reduce
the number of threads, or reduce the amount of malloc/free() done by the
application.

If you're using a library (like this xml one) it may be hard to reduce
the number of malloc/frees() actually done.

Can you try getting a full set of stack traces on the next stall (for
all threads?). I have a feeling that your particular stack trace may be
held up behind an arena_flush() but it'd be good to know for sure.


On Tue, Jan 21, 2014 at 5:46 AM, Alexey Kuts <[email protected]
<mailto:[email protected]>> wrote:

    Hi,

    sometimes (maybe once per day) we have
    an annoying hang in jemalloc().

    jemalloc-version: 3.4.1-0-__g0135fb806e4137dc9cdf152541926__a2bc95e33f0
    Ubuntu 12.04.3 LTS, 3.2.0-33-generic x86_64 GNU/Linux


    #0  0x00007f6bd681a89c in __lll_lock_wait () from
    /lib/x86_64-linux-gnu/__libpthread.so.0
    #1  0x00007f6bd6816080 in _L_lock_903 () from
    /lib/x86_64-linux-gnu/__libpthread.so.0
    #2  0x00007f6bd6815f19 in pthread_mutex_lock () from
    /lib/x86_64-linux-gnu/__libpthread.so.0
    #3  0x0000000000459127 in malloc_mutex_lock (mutex=0x7f6bd4400a08)
    at include/jemalloc/internal/__mutex.h:77
    #4  tcache_bin_flush_small (tbin=<optimized out>, binind=15, rem=3,
    tcache=0x7f6bd4006000) at src/tcache.c:105
    #5  0x000000000045958d in tcache_event_hard (tcache=0x7f6bd4006000)
    at src/tcache.c:39
    #6  0x0000000000437ebe in tcache_event (tcache=0x7f6bd4006000) at
    include/jemalloc/internal/__tcache.h:271
    #7  tcache_alloc_large (size=<optimized out>, tcache=<optimized
    out>, zero=<optimized out>) at include/jemalloc/internal/__tcache.h:383
    #8  arena_malloc (zero=false, size=<optimized out>, arena=0x0,
    try_tcache=true) at include/jemalloc/internal/__arena.h:944
    #9  imallocx (arena=0x0, try_tcache=true, size=<optimized out>) at
    include/jemalloc/internal/__jemalloc_internal.h:798
    #10 imalloc (size=<optimized out>) at
    include/jemalloc/internal/__jemalloc_internal.h:807
    #11 jemalloc (size=<optimized out>) at src/jemalloc.c:887
    #12 0x00007f6bd6d3bad8 in xmlStrndup () from
    /usr/lib/x86_64-linux-gnu/__libxml2.so.2
    #13 0x00007f6bd6ce3029 in xmlNodeListGetString () from
    /usr/lib/x86_64-linux-gnu/__libxml2.so.2
    #14 0x00007f6bd6d0038f in xmlValidateElement () from
    /usr/lib/x86_64-linux-gnu/__libxml2.so.2
    #15 0x00007f6bd6d0034e in xmlValidateElement () from
    /usr/lib/x86_64-linux-gnu/__libxml2.so.2
    #16 0x00007f6bd6d0034e in xmlValidateElement () from
    /usr/lib/x86_64-linux-gnu/__libxml2.so.2
    #17 0x00007f6bd6d00528 in xmlValidateDtd () from
    /usr/lib/x86_64-linux-gnu/__libxml2.so.2
    #18 0x00007f6bd7005355 in
    xmlpp::DtdValidator::validate(__xmlpp::Document const*) () from
    /usr/lib/libxml++-2.6.so.2

    Will appreciate any hints or suggestions

    --
    Kuts Alexey
    _________________________________________________
    jemalloc-discuss mailing list
    [email protected] <mailto:[email protected]>
    http://www.canonware.com/__mailman/listinfo/jemalloc-__discuss
    <http://www.canonware.com/mailman/listinfo/jemalloc-discuss>



--
Куц Алексей
http://staff.yandex-team.ru/akuts
_______________________________________________
jemalloc-discuss mailing list
[email protected]
http://www.canonware.com/mailman/listinfo/jemalloc-discuss

Reply via email to