I've attached the full bt.
Michael
-----Original Message-----
From: Mathieu Desnoyers [mailto:[email protected]]
Sent: Tuesday, June 12, 2012 12:25 PM
To: Burton, Michael
Cc: [email protected]; [email protected]
Subject: Re: [lttng-dev] UST Hanging at Tracepoint
Please build you app and lttng-ust with:
make CFLAGS=-g (so with -O0 optimisation, and debug info)
then, under gdb, please provide the full backtrace of each thread when
stucked with:
thread apply all bt full
Thanks,
Mathieu
* Burton, Michael ([email protected]) wrote:
> Also, I stepped through the code with gdb and things hang within the UST
> constructors and the main function of my application is never reached.
>
> -----Original Message-----
> From: Burton, Michael [mailto:[email protected]]
> Sent: Tuesday, June 12, 2012 11:45 AM
> To: Mathieu Desnoyers; [email protected]
> Cc: [email protected]
> Subject: Re: [lttng-dev] UST Hanging at Tracepoint
>
> I put a fprintf at the beginning of main() and it never prints.
>
> -----Original Message-----
> From: Mathieu Desnoyers [mailto:[email protected]]
> Sent: Tuesday, June 12, 2012 10:14 AM
> To: Burton, Michael; [email protected]
> Cc: [email protected]
> Subject: Re: [lttng-dev] UST Hanging at Tracepoint
>
> * Mathieu Desnoyers ([email protected]) wrote:
> > Hi Michael,
> >
> > Please note that you should be able to run UST with either (or both) of:
> >
> > - root lttng-sessiond, for system-wide tracing
> > - per-user lttng-sessiond, for per-user tracing.
> >
> > It is entirely normal that one of the two UST threads within the
> > application can't find an active sessiond if, for instance, you have
> > just a root (and no per-user) sessiond running.
> >
> > So although I'm glad that you got things working, I fail to understand
> > your explanation: it should have worked fine with the root sessiond,
> > even if no ~/.lttng exists.
> >
> > David, can you look into this ?
>
> hrm, further insight: is your program _really_ hang ? Try to make sure
> you put a fprintf(stderr, ".....") at the beginning of your main().
> Looking once more at your detailed UST output indicates that the
> per-user UST thread tells you that it cannot see any running sessiond.
> So yes, this thread blocks, but the thread talking to the root sessiond
> can very well be working fine, and your application might also be
> running fine. This should be double-checked.
>
> Thanks,
>
> Mathieu
>
> >
> > Thanks,
> >
> > Mathieu
> >
> > * Burton, Michael ([email protected]) wrote:
> > > Mathieu,
> > >
> > > I figured out the problem. When we start sessiond on our system as root,
> > > the /.lttng/ folder is not created, along with its contents. UST timed
> > > out since there was no sock_path. I was able to get UST working once I
> > > got sessiond running.
> > >
> > > Thanks.
> > > Michael
> > >
> > > ________________________________________
> > > From: Mathieu Desnoyers [[email protected]]
> > > Sent: 08 June 2012 21:38
> > > To: Burton, Michael
> > > Cc: [email protected]
> > > Subject: Re: [lttng-dev] UST Hanging at Tracepoint
> > >
> > > * Burton, Michael ([email protected]) wrote:
> > > > For further insight, I've attached the strace from the hanging
> > > > program. I also upgraded LTTng-UST to 2.0.3 and userspace-rcu to
> > > > 0.7.3.
> > >
> > > Haven't had time to look at it yet, but could you also provide the
> > > output of:
> > >
> > > LTTNG_UST_DEBUG=1 youprogname
> > >
> > > (starting your UST instrumented program with LTTNG_UST_DEBUG=1 env. var.
> > > set)
> > >
> > > Thanks!
> > >
> > > Mathieu
> > >
> > > >
> > > > Michael
> > > >
> > > > From: Burton, Michael [mailto:[email protected]]
> > > > Sent: Thursday, June 07, 2012 2:16 PM
> > > > To: [email protected]
> > > > Subject: [lttng-dev] UST Hanging at Tracepoint
> > > >
> > > > I am working on getting LTTng 2.0 working in our 2.6.35 kernel. I have
> > > > kernel tracing working but I'm having problems with UST.
> > > >
> > > > I have a program with a dynamic UST tracepoint. When I run the program
> > > > with all UST events enabled (lttng enable-event -a -u) the program
> > > > hangs on the tracepoint. The last line from the UST debug is:
> > > >
> > > > libust[6184/6185]: Info: sessiond not accepting connections to local
> > > > apps socket (in ust_listener_thread() at lttng-ust-comm.c:699)
> > > >
> > > > Any insight into why this is happening? I've attached the UST and
> > > > lttng-sessiond debug generated by the program.
> > > >
> > > > I am running the following:
> > > > lttng-modules-2.6.32 (found through this mailing list)
> > > > lttng-tools-2.0.1
> > > > lttng-ust-2.0.2
> > > > userspace-rcu-0.7.0
> > > >
> > > > Thanks,
> > > > Michael
> > >
> > > Content-Description: strace_debug.txt
> > > > execve("/ciena/bin/idp", ["idp", "help"], [/* 11 vars */]) = 0
> > > > brk(0) = 0x804e000
> > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> > > > 0) = 0xb783c000
> > > > access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or
> > > > directory)
> > > > open("/etc/ld.so.cache", O_RDONLY) = 3
> > > > fstat64(3, {st_mode=S_IFREG|0644, st_size=17420, ...}) = 0
> > > > mmap2(NULL, 17420, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb7837000
> > > > close(3) = 0
> > > > open("/usr/lib/liblttng-ust.so.0", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220A\0\0004\0\0\0$"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=209876, ...}) = 0
> > > > mmap2(NULL, 212940, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb7803000
> > > > mmap2(0xb7832000, 20480, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2e) = 0xb7832000
> > > > close(3) = 0
> > > > open("/usr/lib/liblttng-ust-ctl.so.0", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200-\0\0004\0\0\0\274"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=140020, ...}) = 0
> > > > mmap2(NULL, 142828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77e0000
> > > > mmap2(0xb7802000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x21) = 0xb7802000
> > > > close(3) = 0
> > > > open("/usr/lib/liblttng-ust-fork.so.0", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\0\5\0\0004\0\0\0X"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=3864, ...}) = 0
> > > > mmap2(NULL, 6820, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
> > > > = 0xb77de000
> > > > mmap2(0xb77df000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77df000
> > > > close(3) = 0
> > > > open("/usr/lib/liblttng-ust-tracepoint.so.0", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\v\0\0004\0\0\0008"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=16632, ...}) = 0
> > > > mmap2(NULL, 19944, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77d9000
> > > > mmap2(0xb77dd000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3) = 0xb77dd000
> > > > close(3) = 0
> > > > open("/usr/lib/liblttng-ust-libc-wrapper.so.0", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\t\0\0004\0\0\0\364"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9300, ...}) = 0
> > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> > > > 0) = 0xb77d8000
> > > > mmap2(NULL, 12104, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77d5000
> > > > mmap2(0xb77d7000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77d7000
> > > > close(3) = 0
> > > > open("/usr/lib/liburcu.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0
> > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77d0000
> > > > mmap2(0xb77d4000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77d4000
> > > > close(3) = 0
> > > > open("/usr/lib/liburcu-common.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\5\0\0004\0\0\0\34"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=5084, ...}) = 0
> > > > mmap2(NULL, 8032, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0)
> > > > = 0xb77ce000
> > > > mmap2(0xb77cf000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0) = 0xb77cf000
> > > > close(3) = 0
> > > > open("/usr/lib/liburcu-qsbr.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360\22\0\0004\0\0\0\374"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0
> > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77c9000
> > > > mmap2(0xb77cd000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77cd000
> > > > close(3) = 0
> > > > open("/usr/lib/liburcu-mb.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\22\0\0004\0\0\0\374"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18188, ...}) = 0
> > > > mmap2(NULL, 17080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77c4000
> > > > mmap2(0xb77c8000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c8000
> > > > close(3) = 0
> > > > open("/usr/lib/liburcu-signal.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0`\23\0\0004\0\0\0\10"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=18712, ...}) = 0
> > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> > > > 0) = 0xb77c3000
> > > > mmap2(NULL, 21704, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77bd000
> > > > mmap2(0xb77c2000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77c2000
> > > > close(3) = 0
> > > > open("/usr/lib/liburcu-cds.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\f\0\0004\0\0\0t"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=26204, ...}) = 0
> > > > mmap2(NULL, 25004, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77b6000
> > > > mmap2(0xb77bc000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb77bc000
> > > > close(3) = 0
> > > > open("/usr/lib/liburcu-bp.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\24\0\0004\0\0\0\360"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=19968, ...}) = 0
> > > > mmap2(NULL, 23128, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77b0000
> > > > mmap2(0xb77b5000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x4) = 0xb77b5000
> > > > close(3) = 0
> > > > open("/lib/libdl.so.2", O_RDONLY) = 3
> > > > read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0
> > > > \n\0\0004\0\0\0<"..., 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=9668, ...}) = 0
> > > > mmap2(NULL, 12408, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77ac000
> > > > mmap2(0xb77ae000, 8192, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1) = 0xb77ae000
> > > > close(3) = 0
> > > > open("/lib/libuuid.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0@\r\0\0004\0\0\0\350"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=12872, ...}) = 0
> > > > mmap2(NULL, 15632, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb77a8000
> > > > mmap2(0xb77ab000, 4096, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2) = 0xb77ab000
> > > > close(3) = 0
> > > > open("/lib/libc.so.6", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220n\1\0004\0\0\0L"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=1347780, ...}) = 0
> > > > mmap2(NULL, 1358120, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb765c000
> > > > mprotect(0xb77a1000, 4096, PROT_NONE) = 0
> > > > mmap2(0xb77a2000, 12288, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x145) = 0xb77a2000
> > > > mmap2(0xb77a5000, 10536, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb77a5000
> > > > close(3) = 0
> > > > open("/lib/librt.so.1", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p\30\0\0004\0\0\0\230"...,
> > > > 512) = 512
> > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> > > > 0) = 0xb765b000
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=30616, ...}) = 0
> > > > mmap2(NULL, 33364, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb7652000
> > > > mmap2(0xb7659000, 8192, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6) = 0xb7659000
> > > > close(3) = 0
> > > > open("/lib/libpthread.so.0", O_RDONLY) = 3
> > > > read(3,
> > > > "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\360J\0\0004\0\0\0\354"...,
> > > > 512) = 512
> > > > fstat64(3, {st_mode=S_IFREG|0755, st_size=88420, ...}) = 0
> > > > mmap2(NULL, 98828, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3,
> > > > 0) = 0xb7639000
> > > > mmap2(0xb764e000, 8192, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x14) = 0xb764e000
> > > > mmap2(0xb7650000, 4620, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7650000
> > > > close(3) = 0
> > > > mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> > > > 0) = 0xb7638000
> > > > mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> > > > 0) = 0xb7636000
> > > > set_thread_area({entry_number:-1 -> 6, base_addr:0xb7636d80,
> > > > limit:1048575, seg_32bit:1, contents:0, read_exec_only:0,
> > > > limit_in_pages:1, seg_not_present:0, useable:1}) = 0
> > > > mprotect(0xb764e000, 4096, PROT_READ) = 0
> > > > mprotect(0xb7659000, 4096, PROT_READ) = 0
> > > > mprotect(0xb77a2000, 8192, PROT_READ) = 0
> > > > mprotect(0xb77ae000, 4096, PROT_READ) = 0
> > > > mprotect(0xb785a000, 4096, PROT_READ) = 0
> > > > munmap(0xb7837000, 17420) = 0
> > > > set_tid_address(0xb7636de8) = 3050
> > > > set_robust_list(0xb7636df0, 0xc) = 0
> > > > futex(0xbfaaf560, FUTEX_WAKE_PRIVATE, 1) = 0
> > > > futex(0xbfaaf560, 0x189 /* FUTEX_??? */, 1, NULL, bfaaf570) = -1 EAGAIN
> > > > (Resource temporarily unavailable)
> > > > rt_sigaction(SIGRTMIN, {0xb763d4f0, [], SA_SIGINFO}, NULL, 8) = 0
> > > > rt_sigaction(SIGRT_1, {0xb763d9c0, [], SA_RESTART|SA_SIGINFO}, NULL, 8)
> > > > = 0
> > > > rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> > > > getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = > > > > 0
> > > > uname({sys="Linux", node="5410", ...}) = 0
> > > > rt_sigaction(SIGUSR1, {0xb77bec0a, [], SA_RESTART|SA_SIGINFO}, NULL, 8)
> > > > = 0
> > > > futex(0xb77af06c, FUTEX_WAKE_PRIVATE, 2147483647) = 0
> > > > brk(0) = 0x804e000
> > > > brk(0x806f000) = 0x806f000
> > > > gettimeofday({1339187216, 880428}, NULL) = 0
> > > > open("/dev/urandom", O_RDONLY|O_LARGEFILE) = 3
> > > > fcntl64(3, F_GETFD) = 0
> > > > fcntl64(3, F_SETFD, FD_CLOEXEC) = 0
> > > > getuid32() = 0
> > > > gettimeofday({1339187216, 889603}, NULL) = 0
> > > > gettimeofday({1339187216, 894471}, NULL) = 0
> > > > read(3, "\275\0051F\215\373\371\202\377kfb/\362\303N"..., 16) = 16
> > > > clock_gettime(CLOCK_REALTIME, {1339187216, 902329405}) = 0
> > > > getuid32() = 0
> > > > geteuid32() = 0
> > > > rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
> > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6e36000
> > > > mprotect(0xb6e36000, 4096, PROT_NONE) = 0
> > > > clone(child_stack=0xb7634d64,
> > > > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED,
> > > > parent_tidptr=0xb7635bd8, {entry_number:6, base_addr:0xb7635b70,
> > > > limit:1048575, seg_32bit:1, contents:0, read_exec_only:0,
> > > > limit_in_pages:1, seg_not_present:0, useable:1},
> > > > child_tidptr=0xb7635bd8) = 3051
> > > > mmap2(NULL, 8388608, PROT_READ|PROT_WRITE,
> > > > MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0xb6636000
> > > > mprotect(0xb6636000, 4096, PROT_NONE) = 0
> > > > clone(child_stack=0xb6e34d64,
> > > > flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS|CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID|CLONE_DETACHED,
> > > > parent_tidptr=0xb6e35bd8, {entry_number:6, base_addr:0xb6e35b70,
> > > > limit:1048575, seg_32bit:1, contents:0, read_exec_only:0,
> > > > limit_in_pages:1, seg_not_present:0, useable:1},
> > > > child_tidptr=0xb6e35bd8) = 3052
> > > > rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
> > > > gettimeofday({1339187216, 974869}, NULL) = 0
> > > > futex(0xb7836e30, FUTEX_WAIT_PRIVATE, 0, {2, 927460405}) = -1 ETIMEDOUT
> > > > (Connection timed out)
> > > > gettid() = 3050
> > > > write(2, "libust[3050/3050]: Error: Timed o"..., 107libust[3050/3050]:
> > > > Error: Timed out waiting for ltt-sessiond (in lttng_ust_init() at
> > > > lttng-ust-comm.c:912)
> > > > ) = 107
> > > > futex(0xb7836e80, FUTEX_WAIT_PRIVATE, 2, NULL
> > > > _______________________________________________
> > > > lttng-dev mailing list
> > > > [email protected]
> > > > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
> > >
> > >
> > > --
> > > Mathieu Desnoyers
> > > Operating System Efficiency R&D Consultant
> > > EfficiOS Inc.
> > > http://www.efficios.com
> >
> > --
> > Mathieu Desnoyers
> > Operating System Efficiency R&D Consultant
> > EfficiOS Inc.
> > http://www.efficios.com
> >
> > _______________________________________________
> > lttng-dev mailing list
> > [email protected]
> > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
>
> --
> Mathieu Desnoyers
> Operating System Efficiency R&D Consultant
> EfficiOS Inc.
> http://www.efficios.com
>
> _______________________________________________
> lttng-dev mailing list
> [email protected]
> http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
--
Mathieu Desnoyers
Operating System Efficiency R&D Consultant
EfficiOS Inc.
http://www.efficios.com
(gdb) thread apply all bt full
[New Thread 8029]
[New Thread 8050]
Thread 3 (Thread 8050):
#0 __lll_lock_wait () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:144
No locals.
#1 0xb762d2e8 in _L_lock_850 () from
/localdata/mburton-cn5410-main/build/saos-sim5410/fs/releasefs/debug/ctm10/i486-sim/rootfs/lib/libpthread.so.0
No symbol table info available.
#2 0xb762d12b in __pthread_mutex_lock (mutex=0xb7822e80) at
pthread_mutex_lock.c:61
ignore3 = 0
ignore1 = -512
ignore2 = 128
type = <value optimized out>
__PRETTY_FUNCTION__ = "__pthread_mutex_lock"
#3 0xb77283b6 in pthread_mutex_lock (mutex=0xb7822e80) at forward.c:182
__p = 0xfffffe00
#4 0xb77ffb38 in ust_lock () at ltt-events.c:70
No locals.
#5 0xb77fa05d in ust_listener_thread (arg=0xb781e9a0) at lttng-ust-comm.c:681
sock_info = 0xb781e9a0
sock = 0
ret = 0
prev_connect_failed = 0
has_waited = 0
__func__ = "ust_listener_thread"
__PRETTY_FUNCTION__ = "ust_listener_thread"
#6 0xb762b723 in start_thread (arg=0xb6e21b70) at pthread_create.c:304
__res = <value optimized out>
__ignore1 = <value optimized out>
__ignore2 = <value optimized out>
pd = 0xb6e21b70
unwind_buf = {
cancel_jmp_buf = {{
jmp_buf = {-1218203660,
0,
8195840,
-1226699416,
119572838,
106195301},
mask_was_saved = 0
}},
priv = {
pad = {0x0,
0x0,
0x0,
0x0},
data = {
prev = 0x0,
cleanup = 0x0,
canceltype = 0
}
}
}
not_first_call = 0
robust = <value optimized out>
---Type <return> to continue, or q <return> to quit---
freesize = <value optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#7 0xb771aade in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:133
No locals.
Thread 2 (Thread 8029):
#0 __lll_lock_wait () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:144
No locals.
#1 0xb762d2e8 in _L_lock_850 () from
/localdata/mburton-cn5410-main/build/saos-sim5410/fs/releasefs/debug/ctm10/i486-sim/rootfs/lib/libpthread.so.0
No symbol table info available.
#2 0xb762d12b in __pthread_mutex_lock (mutex=0xb7822e80) at
pthread_mutex_lock.c:61
ignore3 = 5
ignore1 = -512
ignore2 = 128
type = <value optimized out>
__PRETTY_FUNCTION__ = "__pthread_mutex_lock"
#3 0xb77283b6 in pthread_mutex_lock (mutex=0xb7822e80) at forward.c:182
__p = 0xfffffe00
#4 0xb77ffb38 in ust_lock () at ltt-events.c:70
No locals.
#5 0xb77fb5aa in ust_before_fork (save_sigset=0xb761fa60) at
lttng-ust-comm.c:1021
all_sigs = {
__val = {2147483647,
4294967294,
4294967295 <repeats 30 times>}
}
ret = 0
__func__ = "ust_before_fork"
#6 0xb77ca647 in fork () at ustfork.c:47
sigset = {
__val = {2147221231,
4294967294,
3076651693,
3076651736,
3078871104,
3078398428,
3078785376,
4294967295,
448139,
3078909940,
3078783968,
3078784408,
3076651728,
3078847288,
3078784408,
3076815328,
1,
5,
0,
5,
3078556861,
189,
3078549504,
---Type <return> to continue, or q <return> to quit---
194532,
3078398428,
3078743412,
8029,
3078715683,
3076653128,
3078871104,
2,
3078399436}
}
retval = 2
plibc_func = 0xb76e32f0 <__libc_fork>
#7 0xb77f91cd in get_wait_shm (sock_info=0xb78209c0, mmap_size=4096) at
lttng-ust-comm.c:472
wait_shm_fd = -1
ret = -1216417792
pid = 193988
__func__ = "get_wait_shm"
#8 0xb77f99a1 in get_map_shm (sock_info=0xb78209c0) at lttng-ust-comm.c:581
mmap_size = 4096
wait_shm_fd = -1218313384
ret = -1217231946
wait_shm_mmap = 0xb7822e80 "\002"
__func__ = "get_map_shm"
#9 0xb77f9c87 in wait_for_sessiond (sock_info=0xb78209c0) at
lttng-ust-comm.c:615
ret = -1217231866
__func__ = "wait_for_sessiond"
#10 0xb77fa02d in ust_listener_thread (arg=0xb78209c0) at lttng-ust-comm.c:668
sock_info = 0xb78209c0
sock = 1
ret = 0
prev_connect_failed = 1
has_waited = 0
__func__ = "ust_listener_thread"
__PRETTY_FUNCTION__ = "ust_listener_thread"
#11 0xb762b723 in start_thread (arg=0xb7621b70) at pthread_create.c:304
__res = <value optimized out>
__ignore1 = <value optimized out>
__ignore2 = <value optimized out>
pd = 0xb7621b70
unwind_buf = {
cancel_jmp_buf = {{
jmp_buf = {-1218203660,
0,
8195840,
-1218310808,
119572837,
106195301},
mask_was_saved = 0
}},
priv = {
pad = {0x0,
0x0,
0x0,
---Type <return> to continue, or q <return> to quit---
0x0},
data = {
prev = 0x0,
cleanup = 0x0,
canceltype = 0
}
}
}
not_first_call = 0
robust = <value optimized out>
freesize = <value optimized out>
__PRETTY_FUNCTION__ = "start_thread"
#12 0xb771aade in clone () at ../sysdeps/unix/sysv/linux/i386/clone.S:133
No locals.
Thread 1 (Thread 4238):
#0 __lll_lock_wait () at
../nptl/sysdeps/unix/sysv/linux/i386/i486/lowlevellock.S:144
No locals.
#1 0xb762d2e8 in _L_lock_850 () from
/localdata/mburton-cn5410-main/build/saos-sim5410/fs/releasefs/debug/ctm10/i486-sim/rootfs/lib/libpthread.so.0
No symbol table info available.
#2 0xb762d12b in __pthread_mutex_lock (mutex=0xb7822e80) at
pthread_mutex_lock.c:61
ignore3 = -1218151504
ignore1 = -512
ignore2 = 128
type = <value optimized out>
__PRETTY_FUNCTION__ = "__pthread_mutex_lock"
#3 0xb77283b6 in pthread_mutex_lock (mutex=0xb7822e80) at forward.c:182
__p = 0xfffffe00
#4 0xb77ffb38 in ust_lock () at ltt-events.c:70
No locals.
#5 0xb77fd8cc in ltt_probe_register (desc=0xb77c3e40) at ltt-probes.c:77
iter = 0xb77f001c
ret = 0
i = -1216595880
__func__ = "ltt_probe_register"
__PRETTY_FUNCTION__ = "ltt_probe_register"
#6 0xb77c210f in __lttng_events_init__ust_libc () at
../include/lttng/ust-tracepoint-event.h:550
ret = -1216225080
__PRETTY_FUNCTION__ = "__lttng_events_init__ust_libc"
#7 0xb77c2412 in __do_global_ctors_aux () from
/localdata/mburton-cn5410-main/build/saos-sim5410/fs/releasefs/debug/ctm10/i486-sim/rootfs/usr/lib/liblttng-ust-libc-wrapper.so.0
No locals.
#8 0xb77c1930 in _init () from
/localdata/mburton-cn5410-main/build/saos-sim5410/fs/releasefs/debug/ctm10/i486-sim/rootfs/usr/lib/liblttng-ust-libc-wrapper.so.0
No locals.
#9 0xb7838128 in call_init (l=0xb77c4000, argc=1, argv=0xbfa31814,
env=0xbfa3181c) at dl-init.c:70
init_array = <value optimized out>
#10 0xb783823f in _dl_init (main_map=0xb7847908, argc=1, argv=0xbfa31814,
env=0xbfa3181c) at dl-init.c:134
preinit_array = <value optimized out>
preinit_array_size = <value optimized out>
i = 3
#11 0xb782984f in _dl_start_user () from
/localdata/mburton-cn5410-main/build/saos-sim5410/fs/releasefs/debug/ctm10/i486-sim/rootfs/lib/ld-linux.so.2
No symbol table info available._______________________________________________
lttng-dev mailing list
[email protected]
http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
