Hi Zhenyu, Added lttng-dev mailing list to CC.
----- Original Message ----- > From: "zhenyu.ren" <[email protected]> > To: "Mathieu Desnoyers" <[email protected]> > Sent: Monday, October 21, 2013 9:41:39 AM > Subject: 答复: lttng list -u coredump > Hi,Mathieu > I have known why lttng-sessiond coredumps. Suppose lfhash table to be empty : > int ust_app_list_events(struct lttng_event **events) > ... > cds_lfht_for_each_entry(ust_app_ht->ht, &iter.iter, app, pid_n.node) { > ... > #define cds_lfht_for_each_entry(ht, iter, pos, member)\ > for (cds_lfht_first(ht, iter),\ > pos = caa_container_of(cds_lfht_iter_get_node(iter), \ > __typeof__(*(pos)), member);\ > &(pos)->member != NULL;\ > ... > If cds_lfht_iter_get_node(iter) returns NULL,then pos points to an unvalid > value ,this is where core happens. The NULL check will ensure that this "invalid" value is never actually dereferenced: whenever cds_lfht_first() populates the iterator "iter" with a NULL node, the "pos" returned by caa_container_of() will be a very large pointer value which corresponds to a structure where the lfht node fields is the NULL pointer. Therefore, the &(pos)->member != NULL check will break out of the loop immediately, For the backtrace I see below, it appears that there might be mixup with respect to struct ust_app hash table node usage somewhere. I don't see where for the moment. David, any idea ? could be a mixup between: struct lttng_ht_node_ulong pid_n; struct lttng_ht_node_ulong sock_n; struct lttng_ht_node_ulong notify_sock_n; causing the start of loop not to succeed its NULL pointer check. Please provide details about the lttng-tools, lttng-ust, lttng-modules, userspace RCU versions you are using. Thanks, Mathieu > thanks > zhenyu.ren > ------------------------------------------------------------------ > 发件人：Mathieu Desnoyers <[email protected]> > 发送时间：2013年10月17日(星期四) 23:27 > 收件人：zhenyu.ren <[email protected]> > 主 题：Re: lttng list -u coredump > Please ask this on [email protected], or open a bug report report on > bugs.lttng.org, detailing versions used and detailing the error you > encounter. > Thanks, > Mathieu > ----- Original Message ----- > > From: "zhenyu.ren" <[email protected]> > > > To: "Mathieu Desnoyers" <[email protected]> > > > Sent: Thursday, October 17, 2013 4:56:52 AM > > > Subject: lttng list -u coredump > > > Hi,Mathieu > > > regardless or not running as root,lttng list -u coredumps > > > #lttng list -u > > > Spawning a session daemon > > > FATAL: Module lttng_tracer not found. > > > Error: Unable to load module lttng-tracer > > > Error: Unable to remove module lttng-tracer > > > Warning: No kernel tracer available > > > Warning: No tracing group detected > > > Error: Session daemon died (exit status 0) > > > UST events: > > > ------------- > > > None > > > *** glibc detected *** lttng: free(): invalid pointer: 0x000000000061ad00 > > *** > > > ======= Backtrace: ========= > > > /lib64/libc.so.6[0x34b64722ef] > > > /lib64/libc.so.6(cfree+0x4b)[0x34b647273b] > > > lttng[0x404ef9] > > > lttng[0x40cf6a] > > > /lib64/libc.so.6(__libc_start_main+0xf4)[0x34b641d994] > > > lttng[0x403109] > > > ======= Memory map: ======== > > > 00400000-0041a000 r-xp 00000000 08:02 2494453 /usr/local/bin/lttng > > > 00619000-0061b000 rw-p 00019000 08:02 2494453 /usr/local/bin/lttng > > > 0061b000-0063d000 rw-p 00000000 00:00 0 [heap] > > > 34b6000000-34b601c000 r-xp 00000000 08:02 1884196 /lib64/ld-2.5.so > > > 34b621b000-34b621c000 r--p 0001b000 08:02 1884196 /lib64/ld-2.5.so > > > 34b621c000-34b621d000 rw-p 0001c000 08:02 1884196 /lib64/ld-2.5.so > > > 34b6400000-34b654d000 r-xp 00000000 08:02 1884473 /lib64/libc-2.5.so > > > 34b654d000-34b674d000 ---p 0014d000 08:02 1884473 /lib64/libc-2.5.so > > > 34b674d000-34b6751000 r--p 0014d000 08:02 1884473 /lib64/libc-2.5.so > > > 34b6751000-34b6752000 rw-p 00151000 08:02 1884473 /lib64/libc-2.5.so > > > 34b6752000-34b6757000 rw-p 00000000 00:00 0 > > > 34b6800000-34b6802000 r-xp 00000000 08:02 1884474 /lib64/libdl-2.5.so > > > 34b6802000-34b6a02000 ---p 00002000 08:02 1884474 /lib64/libdl-2.5.so > > > 34b6a02000-34b6a03000 r--p 00002000 08:02 1884474 /lib64/libdl-2.5.so > > > 34b6a03000-34b6a04000 rw-p 00003000 08:02 1884474 /lib64/libdl-2.5.so > > > 34b6c00000-34b6c16000 r-xp 00000000 08:02 1884478 /lib64/libpthread-2.5.so > > > 34b6c16000-34b6e15000 ---p 00016000 08:02 1884478 /lib64/libpthread-2.5.so > > > 34b6e15000-34b6e16000 r--p 00015000 08:02 1884478 /lib64/libpthread-2.5.so > > > 34b6e16000-34b6e17000 rw-p 00016000 08:02 1884478 /lib64/libpthread-2.5.so > > > 34b6e17000-34b6e1b000 rw-p 00000000 00:00 0 > > > 34b7400000-34b7407000 r-xp 00000000 08:02 1884480 /lib64/librt-2.5.so > > > 34b7407000-34b7607000 ---p 00007000 08:02 1884480 /lib64/librt-2.5.so > > > 34b7607000-34b7608000 r--p 00007000 08:02 1884480 /lib64/librt-2.5.so > > > 34b7608000-34b7609000 rw-p 00008000 08:02 1884480 /lib64/librt-2.5.so > > > 34b8400000-34b840d000 r-xp 00000000 08:02 1884371 > > /lib64/libgcc_s-4.1.2-20080825.so.1 > > > 34b840d000-34b860d000 ---p 0000d000 08:02 1884371 > > /lib64/libgcc_s-4.1.2-20080825.so.1 > > > 34b860d000-34b860e000 rw-p 0000d000 08:02 1884371 > > /lib64/libgcc_s-4.1.2-20080825.so.1 > > > 34b9400000-34b9407000 r-xp 00000000 08:02 2493847 > > /usr/lib64/libpopt.so.0.0.0 > > > 34b9407000-34b9607000 ---p 00007000 08:02 2493847 > > /usr/lib64/libpopt.so.0.0.0 > > > 34b9607000-34b9608000 rw-p 00007000 08:02 2493847 > > /usr/lib64/libpopt.so.0.0.0 > > > 37a5e00000-37a5e04000 r-xp 00000000 08:02 1884499 /lib64/libuuid.so.1.2 > > > 37a5e04000-37a6003000 ---p 00004000 08:02 1884499 /lib64/libuuid.so.1.2 > > > 37a6003000-37a6004000 rw-p 00003000 08:02 1884499 /lib64/libuuid.so.1.2 > > > 7f596db29000-7f596db2d000 rw-p 00000000 00:00 0 > > > 7f596db2d000-7f596db32000 r-xp 00000000 08:02 2296098 > > /usr/local/lib/liburcu.so.2.0.0 > > > 7f596db32000-7f596dd31000 ---p 00005000 08:02 2296098 > > /usr/local/lib/liburcu.so.2.0.0 > > > 7f596dd31000-7f596dd32000 rw-p 00004000 08:02 2296098 > > /usr/local/lib/liburcu.so.2.0.0 > > > 7f596dd32000-7f596dd33000 rw-p 00000000 00:00 0 > > > 7f596dd33000-7f596dd36000 r-xp 00000000 08:02 2296094 > > /usr/local/lib/liburcu-common.so.2.0.0 > > > 7f596dd36000-7f596df35000 ---p 00003000 08:02 2296094 > > /usr/local/lib/liburcu-common.so.2.0.0 > > > 7f596df35000-7f596df36000 rw-p 00002000 08:02 2296094 > > /usr/local/lib/liburcu-common.so.2.0.0 > > > 7f596df49000-7f596df4a000 rw-p 00000000 00:00 0 > > > 7f596df4a000-7f596df6d000 r-xp 00000000 08:02 2294055 > > /usr/local/lib/liblttng-ctl.so.0.0.0 > > > 7f596df6d000-7f596e16c000 ---p 00023000 08:02 2294055 > > /usr/local/lib/liblttng-ctl.so.0.0.0 > > > 7f596e16c000-7f596e16e000 rw-p 00022000 08:02 2294055 > > /usr/local/lib/liblttng-ctl.so.0.0.0 > > > 7f596e16e000-7f596e170000 rw-p 00000000 00:00 0 > > > 7f596e170000-7f596e171000 r-xp 00000000 08:02 1427686 > > /home/tops/lib/snoopy.so > > > 7f596e171000-7f596e371000 ---p 00001000 08:02 1427686 > > /home/tops/lib/snoopy.so > > > 7f596e371000-7f596e372000 rw-p 00001000 08:02 1427686 > > /home/tops/lib/snoopy.so > > > 7f596e372000-7f596e375000 rw-p 00000000 00:00 0 > > > 7fffe92bf000-7fffe92d4000 rw-p 00000000 00:00 0 [stack] > > > 7fffe93ff000-7fffe9400000 r-xp 00000000 00:00 0 [vdso] > > > ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] > > > Aborted (core dumped) > > > $ls -l core.12535* > > > -rw------- 1 root root 471040 Oct 17 16:46 core.125350 > > > -rw------- 1 root root 84475904 Oct 17 16:46 core.125351 > > > #file core.12535* > > > core.125350: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV), > > SVR4-style, from 'lttng' > > > core.125351: ELF 64-bit LSB core file AMD x86-64, version 1 (SYSV), > > SVR4-style, from 'lttng-sessiond' > > > #gdb --core=./core.125351 /usr/local/bin/lttng-sessiond > > > warning: Can't read pathname for load map: Input/output error. > > > Reading symbols from /home/tops/lib/snoopy.so...done. > > > ... > > > Core was generated by `lttng-sessiond --sig-parent --quiet'. > > > Program terminated with signal 11, Segmentation fault. > > > [New process 125356] > > > [New process 125358] > > > [New process 125359] > > > [New process 125361] > > > [New process 125351] > > > [New process 125354] > > > [New process 125357] > > > [New process 125360] > > > [New process 125355] > > > #0 ust_app_list_events (events=0x40a62e10) at ust-app.c:3026 > > > 3026 if (!app->compatible) { > > > (gdb) p ust_app_ht > > > $1 = (struct lttng_ht *) 0x65d030 > > > (gdb) p ust_app_ht->ht > > > $2 = (struct cds_lfht *) 0x65d0a0 > > > (gdb) p *(ust_app_ht->ht) > > > $7 = {max_nr_buckets = 9223372036854775808, mm = 0x7f297bd6ac20, flavor = > > 0x7f297b9445e0, count = 0, resize_mutex = {__data = {__lock = 0, __count = > > 0, __owner = 0, > > > __nusers = 0, __kind = 0, __spins = 0, __list = {__prev = 0x0, __next = > > 0x0}}, __size = '\0' <repeats 39 times>, __align = 0}, resize_attr = 0x0, > > > in_progress_resize = 0, in_progress_destroy = 0, resize_target = 4, > > resize_initiated = 0, flags = 3, min_alloc_buckets_order = 0, > > min_nr_alloc_buckets = 1, > > > split_count = 0x65d340, size = 4, bucket_at = 0x7f297bb5aa50 <bucket_at>, > > {tbl_order = {0x65d050, 0x65e350, 0x65e370, 0x0 <repeats 61 times>}, > > tbl_chunk = 0x65d130, > > > tbl_mmap = 0x65d050}} > > > (gdb) p app > > > $8 = (struct ust_app *) 0xffffffffffffff88 > > > Since rcu hash table is unkown to me now,Could you please give me any > > advices > > on how to look at this issue? > > > thanks in advance > > > zhenyu.ren > > -- > Mathieu Desnoyers > EfficiOS Inc. > http://www.efficios.com -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
_______________________________________________ lttng-dev mailing list [email protected] http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
