Re: [Nfs-ganesha-devel] crash in jemalloc leading to a deadlock.
Yes. we should handle postrotate signal. On Wed, Nov 15, 2017 at 11:53 PM, Frank Filz <ffilz...@mindspring.com> wrote: > If we keep the logging fd open, then we need a signal to tell it that > logrotate has occurred… > > > > Frank > > > > *From:* Malahal Naineni [mailto:mala...@gmail.com] > *Sent:* Tuesday, November 14, 2017 9:44 PM > *To:* Frank Filz <ffilz...@mindspring.com> > *Cc:* d...@redhat.com; nfs-ganesha-devel@lists.sourceforge.net > > *Subject:* Re: [Nfs-ganesha-devel] crash in jemalloc leading to a > deadlock. > > > > Silly glibc, they should have provided backtrace_symbols_func() to take a > callback function. Using backtrace_symbols_func() is a bit harder for us > here. Here is a thought: > > > > 1. Our logger should NOT open/re-open for every message. It should just > open once. > > 2. Our logger should have an interface to provide such fd using API or > global symbol. > > 3. There are some tricks to get "fd" with syslog tracing as well > > 4. Then pass such fd from step 2 to backtrace_symbols_fd() > > > > Step1 might be optional as open() should mostly succeed though. > > > > > > On Thu, Nov 9, 2017 at 7:53 PM, Frank Filz <ffilz...@mindspring.com> > wrote: > > That might be a good solution, though what fd would we use? Can we safely > open an fd during a sighandler? > > > > Frank > > > > *From:* Malahal Naineni [mailto:mala...@gmail.com] > *Sent:* Wednesday, November 8, 2017 11:51 PM > *To:* d...@redhat.com > *Cc:* nfs-ganesha-devel@lists.sourceforge.net > *Subject:* Re: [Nfs-ganesha-devel] crash in jemalloc leading to a > deadlock. > > > > backtrace_symbols_fd() takes the same buffer and size arguments as > backtrace_symbols(), but instead of returning an array of strings to the > caller, it writes the strings, one per line, to the file descriptor fd. > backtrace_symbols_fd() does not call malloc(3), and so can be employed in > situations where the latter function might fail. > > > > On Thu, Nov 9, 2017 at 12:24 AM, Daniel Gryniewicz <d...@redhat.com> > wrote: > > Allocating in a backtrace seems like a very bad idea. If there's ever a > crash during an allocation, it is guaranteed to deadlock. > > Daniel > > > > On 11/08/2017 01:43 PM, Pradeep wrote: > > I'm using Ganesha 2.6 dev.12 with jemalloc-3.6.0 and hitting a case > where jemalloc seem to be holding a lock and crashing. In Ganesha's > gsh_backtrace(), we try to allocate memory and that hangs (ended up in > deadlock). Have you seen this before? Perhaps it is a good idea not to > allocate memory in backtrace path? > > > #0 0x7f49b51ff1bd in __lll_lock_wait () from /lib64/libpthread.so.0 > #1 0x7f49b51fad02 in _L_lock_791 () from /lib64/libpthread.so.0 > #2 0x7f49b51fac08 in pthread_mutex_lock () from /lib64/libpthread.so.0 > #3 0x7f49b65d12dc in arena_bin_malloc_hard () from > /lib64/libjemalloc.so.1 > #4 0x7f49b65d1516 in je_arena_tcache_fill_small () from > /lib64/libjemalloc.so.1 > #5 0x7f49b65ea6ff in je_tcache_alloc_small_hard () from > /lib64/libjemalloc.so.1 > #6 0x7f49b65ca14f in malloc () from /lib64/libjemalloc.so.1 > #7 0x7f49b6c5a785 in _dl_scope_free () from /lib64/ld-linux-x86-64.so.2 > #8 0x7f49b6c55841 in _dl_map_object_deps () from > /lib64/ld-linux-x86-64.so.2 > #9 0x7f49b6c5ba4b in dl_open_worker () from /lib64/ld-linux-x86-64.so.2 > #10 0x7f49b6c57364 in _dl_catch_error () from > /lib64/ld-linux-x86-64.so.2 > #11 0x7f49b6c5b35b in _dl_open () from /lib64/ld-linux-x86-64.so.2 > #12 0x7f49b48f5ff2 in do_dlopen () from /lib64/libc.so.6 > #13 0x7f49b6c57364 in _dl_catch_error () from > /lib64/ld-linux-x86-64.so.2 > #14 0x7f49b48f60b2 in __libc_dlopen_mode () from /lib64/libc.so.6 > #15 0x7f49b48cf595 in init () from /lib64/libc.so.6 > #16 0x7f49b51fdbb0 in pthread_once () from /lib64/libpthread.so.0 > #17 0x7f49b48cf6ac in backtrace () from /lib64/libc.so.6 > #18 0x0045193d in gsh_backtrace () at > /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:228 > #19 0x004519fe in crash_handler (signo=11, > info=0x7f49b155db70, ctx=0x7f49b155da40) at > /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:244 > #20 > #21 0x7f49b65d0c61 in arena_purge () from /lib64/libjemalloc.so.1 > #22 0x7f49b65d218d in je_arena_dalloc_large () from > /lib64/libjemalloc.so.1 > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.
Re: [Nfs-ganesha-devel] crash in jemalloc leading to a deadlock.
If we keep the logging fd open, then we need a signal to tell it that logrotate has occurred… Frank From: Malahal Naineni [mailto:mala...@gmail.com] Sent: Tuesday, November 14, 2017 9:44 PM To: Frank Filz <ffilz...@mindspring.com> Cc: d...@redhat.com; nfs-ganesha-devel@lists.sourceforge.net Subject: Re: [Nfs-ganesha-devel] crash in jemalloc leading to a deadlock. Silly glibc, they should have provided backtrace_symbols_func() to take a callback function. Using backtrace_symbols_func() is a bit harder for us here. Here is a thought: 1. Our logger should NOT open/re-open for every message. It should just open once. 2. Our logger should have an interface to provide such fd using API or global symbol. 3. There are some tricks to get "fd" with syslog tracing as well 4. Then pass such fd from step 2 to backtrace_symbols_fd() Step1 might be optional as open() should mostly succeed though. On Thu, Nov 9, 2017 at 7:53 PM, Frank Filz <ffilz...@mindspring.com <mailto:ffilz...@mindspring.com> > wrote: That might be a good solution, though what fd would we use? Can we safely open an fd during a sighandler? Frank From: Malahal Naineni [mailto:mala...@gmail.com <mailto:mala...@gmail.com> ] Sent: Wednesday, November 8, 2017 11:51 PM To: d...@redhat.com <mailto:d...@redhat.com> Cc: nfs-ganesha-devel@lists.sourceforge.net <mailto:nfs-ganesha-devel@lists.sourceforge.net> Subject: Re: [Nfs-ganesha-devel] crash in jemalloc leading to a deadlock. backtrace_symbols_fd() takes the same buffer and size arguments as backtrace_symbols(), but instead of returning an array of strings to the caller, it writes the strings, one per line, to the file descriptor fd. backtrace_symbols_fd() does not call malloc(3), and so can be employed in situations where the latter function might fail. On Thu, Nov 9, 2017 at 12:24 AM, Daniel Gryniewicz <d...@redhat.com <mailto:d...@redhat.com> > wrote: Allocating in a backtrace seems like a very bad idea. If there's ever a crash during an allocation, it is guaranteed to deadlock. Daniel On 11/08/2017 01:43 PM, Pradeep wrote: I'm using Ganesha 2.6 dev.12 with jemalloc-3.6.0 and hitting a case where jemalloc seem to be holding a lock and crashing. In Ganesha's gsh_backtrace(), we try to allocate memory and that hangs (ended up in deadlock). Have you seen this before? Perhaps it is a good idea not to allocate memory in backtrace path? #0 0x7f49b51ff1bd in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x7f49b51fad02 in _L_lock_791 () from /lib64/libpthread.so.0 #2 0x7f49b51fac08 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x7f49b65d12dc in arena_bin_malloc_hard () from /lib64/libjemalloc.so.1 #4 0x7f49b65d1516 in je_arena_tcache_fill_small () from /lib64/libjemalloc.so.1 #5 0x7f49b65ea6ff in je_tcache_alloc_small_hard () from /lib64/libjemalloc.so.1 #6 0x7f49b65ca14f in malloc () from /lib64/libjemalloc.so.1 #7 0x7f49b6c5a785 in _dl_scope_free () from /lib64/ld-linux-x86-64.so.2 #8 0x7f49b6c55841 in _dl_map_object_deps () from /lib64/ld-linux-x86-64.so.2 #9 0x7f49b6c5ba4b in dl_open_worker () from /lib64/ld-linux-x86-64.so.2 #10 0x7f49b6c57364 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 #11 0x7f49b6c5b35b in _dl_open () from /lib64/ld-linux-x86-64.so.2 #12 0x7f49b48f5ff2 in do_dlopen () from /lib64/libc.so.6 #13 0x7f49b6c57364 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 #14 0x7f49b48f60b2 in __libc_dlopen_mode () from /lib64/libc.so.6 #15 0x7f49b48cf595 in init () from /lib64/libc.so.6 #16 0x7f49b51fdbb0 in pthread_once () from /lib64/libpthread.so.0 #17 0x7f49b48cf6ac in backtrace () from /lib64/libc.so.6 #18 0x0045193d in gsh_backtrace () at /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:228 #19 0x004519fe in crash_handler (signo=11, info=0x7f49b155db70, ctx=0x7f49b155da40) at /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:244 #20 #21 0x7f49b65d0c61 in arena_purge () from /lib64/libjemalloc.so.1 #22 0x7f49b65d218d in je_arena_dalloc_large () from /lib64/libjemalloc.so.1 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net <mailto:Nfs-ganesha-devel@lists.sourceforge.net> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net <mailto:Nfs
Re: [Nfs-ganesha-devel] crash in jemalloc leading to a deadlock.
Silly glibc, they should have provided backtrace_symbols_func() to take a callback function. Using backtrace_symbols_func() is a bit harder for us here. Here is a thought: 1. Our logger should NOT open/re-open for every message. It should just open once. 2. Our logger should have an interface to provide such fd using API or global symbol. 3. There are some tricks to get "fd" with syslog tracing as well 4. Then pass such fd from step 2 to backtrace_symbols_fd() Step1 might be optional as open() should mostly succeed though. On Thu, Nov 9, 2017 at 7:53 PM, Frank Filz <ffilz...@mindspring.com> wrote: > That might be a good solution, though what fd would we use? Can we safely > open an fd during a sighandler? > > > > Frank > > > > *From:* Malahal Naineni [mailto:mala...@gmail.com] > *Sent:* Wednesday, November 8, 2017 11:51 PM > *To:* d...@redhat.com > *Cc:* nfs-ganesha-devel@lists.sourceforge.net > *Subject:* Re: [Nfs-ganesha-devel] crash in jemalloc leading to a > deadlock. > > > > backtrace_symbols_fd() takes the same buffer and size arguments as > backtrace_symbols(), but instead of returning an array of strings to the > caller, it writes the strings, one per line, to the file descriptor fd. > backtrace_symbols_fd() does not call malloc(3), and so can be employed in > situations where the latter function might fail. > > > > On Thu, Nov 9, 2017 at 12:24 AM, Daniel Gryniewicz <d...@redhat.com> > wrote: > > Allocating in a backtrace seems like a very bad idea. If there's ever a > crash during an allocation, it is guaranteed to deadlock. > > Daniel > > > > On 11/08/2017 01:43 PM, Pradeep wrote: > > I'm using Ganesha 2.6 dev.12 with jemalloc-3.6.0 and hitting a case > where jemalloc seem to be holding a lock and crashing. In Ganesha's > gsh_backtrace(), we try to allocate memory and that hangs (ended up in > deadlock). Have you seen this before? Perhaps it is a good idea not to > allocate memory in backtrace path? > > > #0 0x7f49b51ff1bd in __lll_lock_wait () from /lib64/libpthread.so.0 > #1 0x7f49b51fad02 in _L_lock_791 () from /lib64/libpthread.so.0 > #2 0x7f49b51fac08 in pthread_mutex_lock () from /lib64/libpthread.so.0 > #3 0x7f49b65d12dc in arena_bin_malloc_hard () from > /lib64/libjemalloc.so.1 > #4 0x7f49b65d1516 in je_arena_tcache_fill_small () from > /lib64/libjemalloc.so.1 > #5 0x7f49b65ea6ff in je_tcache_alloc_small_hard () from > /lib64/libjemalloc.so.1 > #6 0x7f49b65ca14f in malloc () from /lib64/libjemalloc.so.1 > #7 0x7f49b6c5a785 in _dl_scope_free () from /lib64/ld-linux-x86-64.so.2 > #8 0x7f49b6c55841 in _dl_map_object_deps () from > /lib64/ld-linux-x86-64.so.2 > #9 0x7f49b6c5ba4b in dl_open_worker () from /lib64/ld-linux-x86-64.so.2 > #10 0x7f49b6c57364 in _dl_catch_error () from > /lib64/ld-linux-x86-64.so.2 > #11 0x7f49b6c5b35b in _dl_open () from /lib64/ld-linux-x86-64.so.2 > #12 0x7f49b48f5ff2 in do_dlopen () from /lib64/libc.so.6 > #13 0x7f49b6c57364 in _dl_catch_error () from > /lib64/ld-linux-x86-64.so.2 > #14 0x7f49b48f60b2 in __libc_dlopen_mode () from /lib64/libc.so.6 > #15 0x7f49b48cf595 in init () from /lib64/libc.so.6 > #16 0x7f49b51fdbb0 in pthread_once () from /lib64/libpthread.so.0 > #17 0x7f49b48cf6ac in backtrace () from /lib64/libc.so.6 > #18 0x0045193d in gsh_backtrace () at > /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:228 > #19 0x004519fe in crash_handler (signo=11, > info=0x7f49b155db70, ctx=0x7f49b155da40) at > /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:244 > #20 > #21 0x7f49b65d0c61 in arena_purge () from /lib64/libjemalloc.so.1 > #22 0x7f49b65d218d in je_arena_dalloc_large () from > /lib64/libjemalloc.so.1 > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Nfs-ganesha-devel mailing list > Nfs-ganesha-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > > > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Nfs-ganesha-devel mailing list > Nfs-ganesha-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > > > > > <https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=email
Re: [Nfs-ganesha-devel] crash in jemalloc leading to a deadlock.
backtrace_symbols_fd() takes the same buffer and size arguments as backtrace_symbols(), but instead of returning an array of strings to the caller, it writes the strings, one per line, to the file descriptor fd. backtrace_symbols_fd() does not call malloc(3), and so can be employed in situations where the latter function might fail. On Thu, Nov 9, 2017 at 12:24 AM, Daniel Gryniewiczwrote: > Allocating in a backtrace seems like a very bad idea. If there's ever a > crash during an allocation, it is guaranteed to deadlock. > > Daniel > > > On 11/08/2017 01:43 PM, Pradeep wrote: > >> I'm using Ganesha 2.6 dev.12 with jemalloc-3.6.0 and hitting a case >> where jemalloc seem to be holding a lock and crashing. In Ganesha's >> gsh_backtrace(), we try to allocate memory and that hangs (ended up in >> deadlock). Have you seen this before? Perhaps it is a good idea not to >> allocate memory in backtrace path? >> >> >> #0 0x7f49b51ff1bd in __lll_lock_wait () from /lib64/libpthread.so.0 >> #1 0x7f49b51fad02 in _L_lock_791 () from /lib64/libpthread.so.0 >> #2 0x7f49b51fac08 in pthread_mutex_lock () from /lib64/libpthread.so.0 >> #3 0x7f49b65d12dc in arena_bin_malloc_hard () from >> /lib64/libjemalloc.so.1 >> #4 0x7f49b65d1516 in je_arena_tcache_fill_small () from >> /lib64/libjemalloc.so.1 >> #5 0x7f49b65ea6ff in je_tcache_alloc_small_hard () from >> /lib64/libjemalloc.so.1 >> #6 0x7f49b65ca14f in malloc () from /lib64/libjemalloc.so.1 >> #7 0x7f49b6c5a785 in _dl_scope_free () from >> /lib64/ld-linux-x86-64.so.2 >> #8 0x7f49b6c55841 in _dl_map_object_deps () from >> /lib64/ld-linux-x86-64.so.2 >> #9 0x7f49b6c5ba4b in dl_open_worker () from >> /lib64/ld-linux-x86-64.so.2 >> #10 0x7f49b6c57364 in _dl_catch_error () from >> /lib64/ld-linux-x86-64.so.2 >> #11 0x7f49b6c5b35b in _dl_open () from /lib64/ld-linux-x86-64.so.2 >> #12 0x7f49b48f5ff2 in do_dlopen () from /lib64/libc.so.6 >> #13 0x7f49b6c57364 in _dl_catch_error () from >> /lib64/ld-linux-x86-64.so.2 >> #14 0x7f49b48f60b2 in __libc_dlopen_mode () from /lib64/libc.so.6 >> #15 0x7f49b48cf595 in init () from /lib64/libc.so.6 >> #16 0x7f49b51fdbb0 in pthread_once () from /lib64/libpthread.so.0 >> #17 0x7f49b48cf6ac in backtrace () from /lib64/libc.so.6 >> #18 0x0045193d in gsh_backtrace () at >> /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:228 >> #19 0x004519fe in crash_handler (signo=11, >> info=0x7f49b155db70, ctx=0x7f49b155da40) at >> /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:244 >> #20 >> #21 0x7f49b65d0c61 in arena_purge () from /lib64/libjemalloc.so.1 >> #22 0x7f49b65d218d in je_arena_dalloc_large () from >> /lib64/libjemalloc.so.1 >> >> >> -- >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot >> ___ >> Nfs-ganesha-devel mailing list >> Nfs-ganesha-devel@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel >> >> > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Nfs-ganesha-devel mailing list > Nfs-ganesha-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
Re: [Nfs-ganesha-devel] crash in jemalloc leading to a deadlock.
Allocating in a backtrace seems like a very bad idea. If there's ever a crash during an allocation, it is guaranteed to deadlock. Daniel On 11/08/2017 01:43 PM, Pradeep wrote: I'm using Ganesha 2.6 dev.12 with jemalloc-3.6.0 and hitting a case where jemalloc seem to be holding a lock and crashing. In Ganesha's gsh_backtrace(), we try to allocate memory and that hangs (ended up in deadlock). Have you seen this before? Perhaps it is a good idea not to allocate memory in backtrace path? #0 0x7f49b51ff1bd in __lll_lock_wait () from /lib64/libpthread.so.0 #1 0x7f49b51fad02 in _L_lock_791 () from /lib64/libpthread.so.0 #2 0x7f49b51fac08 in pthread_mutex_lock () from /lib64/libpthread.so.0 #3 0x7f49b65d12dc in arena_bin_malloc_hard () from /lib64/libjemalloc.so.1 #4 0x7f49b65d1516 in je_arena_tcache_fill_small () from /lib64/libjemalloc.so.1 #5 0x7f49b65ea6ff in je_tcache_alloc_small_hard () from /lib64/libjemalloc.so.1 #6 0x7f49b65ca14f in malloc () from /lib64/libjemalloc.so.1 #7 0x7f49b6c5a785 in _dl_scope_free () from /lib64/ld-linux-x86-64.so.2 #8 0x7f49b6c55841 in _dl_map_object_deps () from /lib64/ld-linux-x86-64.so.2 #9 0x7f49b6c5ba4b in dl_open_worker () from /lib64/ld-linux-x86-64.so.2 #10 0x7f49b6c57364 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 #11 0x7f49b6c5b35b in _dl_open () from /lib64/ld-linux-x86-64.so.2 #12 0x7f49b48f5ff2 in do_dlopen () from /lib64/libc.so.6 #13 0x7f49b6c57364 in _dl_catch_error () from /lib64/ld-linux-x86-64.so.2 #14 0x7f49b48f60b2 in __libc_dlopen_mode () from /lib64/libc.so.6 #15 0x7f49b48cf595 in init () from /lib64/libc.so.6 #16 0x7f49b51fdbb0 in pthread_once () from /lib64/libpthread.so.0 #17 0x7f49b48cf6ac in backtrace () from /lib64/libc.so.6 #18 0x0045193d in gsh_backtrace () at /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:228 #19 0x004519fe in crash_handler (signo=11, info=0x7f49b155db70, ctx=0x7f49b155da40) at /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:244 #20 #21 0x7f49b65d0c61 in arena_purge () from /lib64/libjemalloc.so.1 #22 0x7f49b65d218d in je_arena_dalloc_large () from /lib64/libjemalloc.so.1 -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel
Re: [Nfs-ganesha-devel] crash in jemalloc leading to a deadlock.
> I'm using Ganesha 2.6 dev.12 with jemalloc-3.6.0 and hitting a case where > jemalloc seem to be holding a lock and crashing. In Ganesha's > gsh_backtrace(), we try to allocate memory and that hangs (ended up in > deadlock). Have you seen this before? Perhaps it is a good idea not to > allocate memory in backtrace path? Hmm, sadly backtrace_symbols is a library function, and yea, it allocates memory... Not sure there's much we can do about that. It would be interesting to understand what is causing jemalloc to throw a signal. We probably managed to corrupt memory... You might run under ASAN and see if it highlights anything. When we detect out of memory, we abort() which is not catchable and thus no backtrace, so at least that path is safe... Frank > #0 0x7f49b51ff1bd in __lll_lock_wait () from /lib64/libpthread.so.0 > #1 0x7f49b51fad02 in _L_lock_791 () from /lib64/libpthread.so.0 > #2 0x7f49b51fac08 in pthread_mutex_lock () from /lib64/libpthread.so.0 > #3 0x7f49b65d12dc in arena_bin_malloc_hard () from > /lib64/libjemalloc.so.1 > #4 0x7f49b65d1516 in je_arena_tcache_fill_small () from > /lib64/libjemalloc.so.1 > #5 0x7f49b65ea6ff in je_tcache_alloc_small_hard () from > /lib64/libjemalloc.so.1 > #6 0x7f49b65ca14f in malloc () from /lib64/libjemalloc.so.1 > #7 0x7f49b6c5a785 in _dl_scope_free () from /lib64/ld-linux-x86-64.so.2 > #8 0x7f49b6c55841 in _dl_map_object_deps () from /lib64/ld-linux-x86- > 64.so.2 > #9 0x7f49b6c5ba4b in dl_open_worker () from /lib64/ld-linux-x86- > 64.so.2 > #10 0x7f49b6c57364 in _dl_catch_error () from /lib64/ld-linux-x86- > 64.so.2 > #11 0x7f49b6c5b35b in _dl_open () from /lib64/ld-linux-x86-64.so.2 > #12 0x7f49b48f5ff2 in do_dlopen () from /lib64/libc.so.6 > #13 0x7f49b6c57364 in _dl_catch_error () from /lib64/ld-linux-x86- > 64.so.2 > #14 0x7f49b48f60b2 in __libc_dlopen_mode () from /lib64/libc.so.6 > #15 0x7f49b48cf595 in init () from /lib64/libc.so.6 > #16 0x7f49b51fdbb0 in pthread_once () from /lib64/libpthread.so.0 > #17 0x7f49b48cf6ac in backtrace () from /lib64/libc.so.6 > #18 0x0045193d in gsh_backtrace () at > /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:228 > #19 0x004519fe in crash_handler (signo=11, info=0x7f49b155db70, > ctx=0x7f49b155da40) at > /usr/src/debug/nfs-ganesha-2.6-dev.12/MainNFSD/nfs_init.c:244 > #20 > #21 0x7f49b65d0c61 in arena_purge () from /lib64/libjemalloc.so.1 > #22 0x7f49b65d218d in je_arena_dalloc_large () from > /lib64/libjemalloc.so.1 > > -- > Check out the vibrant tech community on one of the world's most engaging > tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Nfs-ganesha-devel mailing list > Nfs-ganesha-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel --- This email has been checked for viruses by Avast antivirus software. https://www.avast.com/antivirus -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Nfs-ganesha-devel mailing list Nfs-ganesha-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/nfs-ganesha-devel