Re: Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
(2015/03/03 22:31), Namhyung Kim wrote: > Hi Masami, > > On Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu wrote: >> Fix perf probe to handle aliased symbols correctly in glibc. >> In the glibc, several symbols are defined as an alias of >> __libc_XXX, e.g. malloc is an alias of __libc_malloc. >> In such cases, dwarf has no subroutine instances of the >> alias functions (e.g. no "malloc" instance), but the map >> has that symbol and its address. >> Thus, if we search the alieased symbol in debuginfo, we >> always fail to find it, but it is in the map. >> >> To solve this problem, this fails back to address-based >> alternative search, which searches the symbol in the map, >> translates its address to alternative (correct) function >> name by using debuginfo, and retry to find the alternative >> function point from debuginfo. >> >> This adds fail-back process to --vars, --lines and --add >> options. So, now you can use those on malloc@libc :) > > So this is only for binaries that have debuginfo, right? > > I have a similar issue with no debuginfo. > > $ perf probe -x /usr/lib/libc.so.6 -V calloc > The /usr/lib/libc-2.21.so file has no debug information. > Rebuild with -g, or install an appropriate debuginfo package. > Error: Failed to show vars. > > > But it also failed to add a probe to calloc: > > $ perf probe -x /usr/lib/libc.so.6 -a calloc > Failed to find symbol calloc in /usr/lib/libc-2.21.so > Error: Failed to add events. > > > Of course there's calloc in the libc binary. > > $ nm /usr/lib/libc.so.6 | grep calloc > 0007b1f0 t __calloc > 0007b1f0 T __libc_calloc > 0007b1f0 W calloc > > > I think the problem is that calloc is a weak symbol so it'll be > discarded during the symbol loading. Right, I also hit same problem on waitpid, which is also an weak symbol. > It's because to avoid multiple > symbols (or aliases) at a same address so we choose a better symbol > using heuristics. But for this case I think we can allow aliases > since it's used only for finding probe points. I'm just using struct map and dso, so those should be improved. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Hi Masami, On Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu wrote: > Fix perf probe to handle aliased symbols correctly in glibc. > In the glibc, several symbols are defined as an alias of > __libc_XXX, e.g. malloc is an alias of __libc_malloc. > In such cases, dwarf has no subroutine instances of the > alias functions (e.g. no "malloc" instance), but the map > has that symbol and its address. > Thus, if we search the alieased symbol in debuginfo, we > always fail to find it, but it is in the map. > > To solve this problem, this fails back to address-based > alternative search, which searches the symbol in the map, > translates its address to alternative (correct) function > name by using debuginfo, and retry to find the alternative > function point from debuginfo. > > This adds fail-back process to --vars, --lines and --add > options. So, now you can use those on malloc@libc :) So this is only for binaries that have debuginfo, right? I have a similar issue with no debuginfo. $ perf probe -x /usr/lib/libc.so.6 -V calloc The /usr/lib/libc-2.21.so file has no debug information. Rebuild with -g, or install an appropriate debuginfo package. Error: Failed to show vars. But it also failed to add a probe to calloc: $ perf probe -x /usr/lib/libc.so.6 -a calloc Failed to find symbol calloc in /usr/lib/libc-2.21.so Error: Failed to add events. Of course there's calloc in the libc binary. $ nm /usr/lib/libc.so.6 | grep calloc 0007b1f0 t __calloc 0007b1f0 T __libc_calloc 0007b1f0 W calloc I think the problem is that calloc is a weak symbol so it'll be discarded during the symbol loading. It's because to avoid multiple symbols (or aliases) at a same address so we choose a better symbol using heuristics. But for this case I think we can allow aliases since it's used only for finding probe points. Thanks, Namhyung -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Hi Masami, On Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu wrote: Fix perf probe to handle aliased symbols correctly in glibc. In the glibc, several symbols are defined as an alias of __libc_XXX, e.g. malloc is an alias of __libc_malloc. In such cases, dwarf has no subroutine instances of the alias functions (e.g. no malloc instance), but the map has that symbol and its address. Thus, if we search the alieased symbol in debuginfo, we always fail to find it, but it is in the map. To solve this problem, this fails back to address-based alternative search, which searches the symbol in the map, translates its address to alternative (correct) function name by using debuginfo, and retry to find the alternative function point from debuginfo. This adds fail-back process to --vars, --lines and --add options. So, now you can use those on malloc@libc :) So this is only for binaries that have debuginfo, right? I have a similar issue with no debuginfo. $ perf probe -x /usr/lib/libc.so.6 -V calloc The /usr/lib/libc-2.21.so file has no debug information. Rebuild with -g, or install an appropriate debuginfo package. Error: Failed to show vars. But it also failed to add a probe to calloc: $ perf probe -x /usr/lib/libc.so.6 -a calloc Failed to find symbol calloc in /usr/lib/libc-2.21.so Error: Failed to add events. Of course there's calloc in the libc binary. $ nm /usr/lib/libc.so.6 | grep calloc 0007b1f0 t __calloc 0007b1f0 T __libc_calloc 0007b1f0 W calloc I think the problem is that calloc is a weak symbol so it'll be discarded during the symbol loading. It's because to avoid multiple symbols (or aliases) at a same address so we choose a better symbol using heuristics. But for this case I think we can allow aliases since it's used only for finding probe points. Thanks, Namhyung -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
(2015/03/03 22:31), Namhyung Kim wrote: Hi Masami, On Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu wrote: Fix perf probe to handle aliased symbols correctly in glibc. In the glibc, several symbols are defined as an alias of __libc_XXX, e.g. malloc is an alias of __libc_malloc. In such cases, dwarf has no subroutine instances of the alias functions (e.g. no malloc instance), but the map has that symbol and its address. Thus, if we search the alieased symbol in debuginfo, we always fail to find it, but it is in the map. To solve this problem, this fails back to address-based alternative search, which searches the symbol in the map, translates its address to alternative (correct) function name by using debuginfo, and retry to find the alternative function point from debuginfo. This adds fail-back process to --vars, --lines and --add options. So, now you can use those on malloc@libc :) So this is only for binaries that have debuginfo, right? I have a similar issue with no debuginfo. $ perf probe -x /usr/lib/libc.so.6 -V calloc The /usr/lib/libc-2.21.so file has no debug information. Rebuild with -g, or install an appropriate debuginfo package. Error: Failed to show vars. But it also failed to add a probe to calloc: $ perf probe -x /usr/lib/libc.so.6 -a calloc Failed to find symbol calloc in /usr/lib/libc-2.21.so Error: Failed to add events. Of course there's calloc in the libc binary. $ nm /usr/lib/libc.so.6 | grep calloc 0007b1f0 t __calloc 0007b1f0 T __libc_calloc 0007b1f0 W calloc I think the problem is that calloc is a weak symbol so it'll be discarded during the symbol loading. Right, I also hit same problem on waitpid, which is also an weak symbol. It's because to avoid multiple symbols (or aliases) at a same address so we choose a better symbol using heuristics. But for this case I think we can allow aliases since it's used only for finding probe points. I'm just using struct map and dso, so those should be improved. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Em Tue, Mar 03, 2015 at 01:11:17PM +0900, Masami Hiramatsu escreveu: > (2015/03/03 12:05), Arnaldo Carvalho de Melo wrote: > > Em Mon, Mar 02, 2015 at 11:45:12PM -0300, Arnaldo Carvalho de Melo escreveu: > >> Em Tue, Mar 03, 2015 at 11:39:02AM +0900, Masami Hiramatsu escreveu: > >>> (2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: > Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: > > With this patch; > > - > > # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc > > Available variables at malloc > > @<__libc_malloc+0> > > size_t bytes > > # ./perf probe -x /usr/lib64/libc-2.17.so -a "malloc bytes" > > Added new event: > > probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with > > bytes) > > > > You can now use it in all perf tools, such as: > > > > perf record -e probe_libc:malloc -aR sleep 1 > > > Reported-by: Arnaldo Carvalho de Melo > > Humm, not working for me, after the patch: > > [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc > Available variables at malloc > @<__malloc_check_init+96> > (No matched variables) > >>> > >> > >> Will try after a 'make build-test' finishes for the current batch > > > > Thank you for checking this, > > > [root@ssdandy ~]# perf probe -vvv -x /usr/lib64/libc-2.17.so -V malloc > > probe-definition(0): malloc > > symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) > > 0 arguments > > Open Debuginfo file: /usr/lib/debug/usr/lib64/libc-2.17.so.debug > > Searching variables at malloc > > Symbol malloc address found : 800c0 > > Get 2611 lines from this CU > > Hmm, something wrong at here. > > > Probe point found: __malloc_check_init+96 > > This seems that the debuginfo is a bit odd. > Could you also run eu-addr2line for 0x800c0 and 0x800b0? > > I'm using CentOS7 with a bit newer glibc rpms. > > # rpm -q glibc glibc-debuginfo > glibc-2.17-55.el7_0.5.x86_64 > glibc-debuginfo-2.17-55.el7_0.5.x86_64 > > And I got following results. > > [mhiramat@localhost perf]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800c0 > __libc_malloc > /usr/src/debug/glibc-2.17-c758a686/malloc/malloc.c:2855 > [mhiramat@localhost perf]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800b0 > __malloc_check_init > /usr/src/debug/glibc-2.17-c758a686/malloc/hooks.c:75 > > So, 0x800b0 correctly points __malloc_check_init+96 but its address is not > 0x800c0. [acme@ssdandy linux]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800c0 malloc ??:0 [acme@ssdandy linux]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800b0 __malloc_check_init ??:0 [acme@ssdandy linux]$ If I do it over the debuginfo files: [acme@ssdandy linux]$ addr2line -fi -e /usr/lib/debug/usr/lib64/libc-2.17.so.debug 0x800c0 __malloc_check_init /usr/src/debug/glibc-2.17-c758a686/malloc/hooks.c:75 [acme@ssdandy linux]$ addr2line -fi -e /usr/lib/debug/usr/lib64/libc-2.17.so.debug 0x800b0 __malloc_check_init /usr/src/debug/glibc-2.17-c758a686/malloc/hooks.c:82 [acme@ssdandy linux]$ Tomorrow I'll check for newer packages or any reports about problems with debuginfo files, please let me know if you need any further info, Off to bed, zzZzz - Arnaldo > Thank you, > > > > Available variables at malloc > > @<__malloc_check_init+96> > > (No matched variables) > > [root@ssdandy ~]# > > > > If I add one more 'v' I get the symtabs as read by symbol-elf.c and this is > > what is there for the malloc routines: > > Yeah, it seems symbol maps works fine. Debuginfo analysis failed. > > > > > [root@ssdandy ~]# grep malloc /tmp/4 > > probe-definition(0): malloc > > symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) > > Searching variables at malloc > > symbol__new: ptmalloc_lock_all 0x7b060-0x7b173 > > symbol__new: malloc_atfork 0x801c0-0x802dd > > symbol__new: ptmalloc_unlock_all2 0x7b180-0x7b21e > > symbol__new: ptmalloc_unlock_all 0x7ba90-0x7bb3e > > symbol__new: malloc_printerr 0x7bb40-0x7bc28 > > symbol__new: malloc_consolidate 0x7c3d0-0x7c9be > > symbol__new: _int_malloc 0x7dee0-0x7f32b > > symbol__new: malloc_check 0x7f330-0x7f44a > > symbol__new: ptmalloc_init.part.8 0x80fe0-0x813d7 > > symbol__new: ptmalloc_init 0x813e0-0x813f5 > > symbol__new: malloc_hook_ini 0x81400-0x8143c > > symbol__new: mallochook 0x82fc0-0x831af > > symbol__new: tr_mallochook 0x83fc0-0x8407d > > symbol__new: __malloc_set_state 0x814f0-0x819f6 > > symbol__new: __malloc_usable_size 0x80d30-0x80e0e > > symbol__new: __malloc_trim 0x81d50-0x81fac > > symbol__new: __GI___libc_malloc 0x800c0-0x801b5 > > symbol__new: __malloc_get_state 0x802e0-0x804db > > symbol__new: __malloc_stats 0x820c0-0x822aa > > symbol__new: __malloc_check_init 0x80050-0x800bb > > symbol__new: __malloc 0x800c0-0x801b5 > > symbol__new: malloc_set_state 0x814f0-0x819f6 > > symbol__new: malloc 0x800c0-0x801b5 > >
Re: Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
(2015/03/03 12:05), Arnaldo Carvalho de Melo wrote: > Em Mon, Mar 02, 2015 at 11:45:12PM -0300, Arnaldo Carvalho de Melo escreveu: >> Em Tue, Mar 03, 2015 at 11:39:02AM +0900, Masami Hiramatsu escreveu: >>> (2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: > With this patch; > - > # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc > Available variables at malloc > @<__libc_malloc+0> > size_t bytes > # ./perf probe -x /usr/lib64/libc-2.17.so -a "malloc bytes" > Added new event: > probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) > > You can now use it in all perf tools, such as: > > perf record -e probe_libc:malloc -aR sleep 1 > Reported-by: Arnaldo Carvalho de Melo Humm, not working for me, after the patch: [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @<__malloc_check_init+96> (No matched variables) >>> >> >> Will try after a 'make build-test' finishes for the current batch > Thank you for checking this, > [root@ssdandy ~]# perf probe -vvv -x /usr/lib64/libc-2.17.so -V malloc > probe-definition(0): malloc > symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) > 0 arguments > Open Debuginfo file: /usr/lib/debug/usr/lib64/libc-2.17.so.debug > Searching variables at malloc > Symbol malloc address found : 800c0 > Get 2611 lines from this CU Hmm, something wrong at here. > Probe point found: __malloc_check_init+96 This seems that the debuginfo is a bit odd. Could you also run eu-addr2line for 0x800c0 and 0x800b0? I'm using CentOS7 with a bit newer glibc rpms. # rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.5.x86_64 glibc-debuginfo-2.17-55.el7_0.5.x86_64 And I got following results. [mhiramat@localhost perf]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800c0 __libc_malloc /usr/src/debug/glibc-2.17-c758a686/malloc/malloc.c:2855 [mhiramat@localhost perf]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800b0 __malloc_check_init /usr/src/debug/glibc-2.17-c758a686/malloc/hooks.c:75 So, 0x800b0 correctly points __malloc_check_init+96 but its address is not 0x800c0. Thank you, > Available variables at malloc > @<__malloc_check_init+96> > (No matched variables) > [root@ssdandy ~]# > > If I add one more 'v' I get the symtabs as read by symbol-elf.c and this is > what is there for the malloc routines: Yeah, it seems symbol maps works fine. Debuginfo analysis failed. > > [root@ssdandy ~]# grep malloc /tmp/4 > probe-definition(0): malloc > symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) > Searching variables at malloc > symbol__new: ptmalloc_lock_all 0x7b060-0x7b173 > symbol__new: malloc_atfork 0x801c0-0x802dd > symbol__new: ptmalloc_unlock_all2 0x7b180-0x7b21e > symbol__new: ptmalloc_unlock_all 0x7ba90-0x7bb3e > symbol__new: malloc_printerr 0x7bb40-0x7bc28 > symbol__new: malloc_consolidate 0x7c3d0-0x7c9be > symbol__new: _int_malloc 0x7dee0-0x7f32b > symbol__new: malloc_check 0x7f330-0x7f44a > symbol__new: ptmalloc_init.part.8 0x80fe0-0x813d7 > symbol__new: ptmalloc_init 0x813e0-0x813f5 > symbol__new: malloc_hook_ini 0x81400-0x8143c > symbol__new: mallochook 0x82fc0-0x831af > symbol__new: tr_mallochook 0x83fc0-0x8407d > symbol__new: __malloc_set_state 0x814f0-0x819f6 > symbol__new: __malloc_usable_size 0x80d30-0x80e0e > symbol__new: __malloc_trim 0x81d50-0x81fac > symbol__new: __GI___libc_malloc 0x800c0-0x801b5 > symbol__new: __malloc_get_state 0x802e0-0x804db > symbol__new: __malloc_stats 0x820c0-0x822aa > symbol__new: __malloc_check_init 0x80050-0x800bb > symbol__new: __malloc 0x800c0-0x801b5 > symbol__new: malloc_set_state 0x814f0-0x819f6 > symbol__new: malloc 0x800c0-0x801b5 > symbol__new: malloc_info 0x82320-0x8240f > symbol__new: malloc_trim 0x81d50-0x81fac > symbol__new: malloc_usable_size 0x80d30-0x80e0e > symbol__new: malloc_get_state 0x802e0-0x804db > symbol__new: __libc_malloc 0x800c0-0x801b5 > symbol__new: malloc_stats 0x820c0-0x822aa > symbol__new: malloc@plt 0x1f300-0x1f310 > Symbol malloc address found : 800c0 > Probe point found: __malloc_check_init+96 > [root@ssdandy ~]# > [root@ssdandy ~]# And then the one asking for 'bytes' to be collectd fails. After processing the other patches I'll try to debug this... [root@ssdandy ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.0 (Maipo) [root@ssdandy ~]# rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.3.x86_64 glibc-debuginfo-2.17-55.el7_0.1.x86_64 >>> ^^^ why is this different from the glibc >>> version?? >>> [root@ssdandy ~]# [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc >>> [...] 4849: 00080050 107 FUNC
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Em Mon, Mar 02, 2015 at 11:45:12PM -0300, Arnaldo Carvalho de Melo escreveu: > Em Tue, Mar 03, 2015 at 11:39:02AM +0900, Masami Hiramatsu escreveu: > > (2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: > > > Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: > > >> With this patch; > > >> - > > >> # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc > > >> Available variables at malloc > > >> @<__libc_malloc+0> > > >> size_t bytes > > >> # ./perf probe -x /usr/lib64/libc-2.17.so -a "malloc bytes" > > >> Added new event: > > >> probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with > > >> bytes) > > >> > > >> You can now use it in all perf tools, such as: > > >> > > >> perf record -e probe_libc:malloc -aR sleep 1 > > > > > >> Reported-by: Arnaldo Carvalho de Melo > > > > > > Humm, not working for me, after the patch: > > > > > > [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc > > > Available variables at malloc > > > @<__malloc_check_init+96> > > > (No matched variables) > > > > Will try after a 'make build-test' finishes for the current batch [root@ssdandy ~]# perf probe -vvv -x /usr/lib64/libc-2.17.so -V malloc probe-definition(0): malloc symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) 0 arguments Open Debuginfo file: /usr/lib/debug/usr/lib64/libc-2.17.so.debug Searching variables at malloc Symbol malloc address found : 800c0 Get 2611 lines from this CU Probe point found: __malloc_check_init+96 Available variables at malloc @<__malloc_check_init+96> (No matched variables) [root@ssdandy ~]# If I add one more 'v' I get the symtabs as read by symbol-elf.c and this is what is there for the malloc routines: [root@ssdandy ~]# grep malloc /tmp/4 probe-definition(0): malloc symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) Searching variables at malloc symbol__new: ptmalloc_lock_all 0x7b060-0x7b173 symbol__new: malloc_atfork 0x801c0-0x802dd symbol__new: ptmalloc_unlock_all2 0x7b180-0x7b21e symbol__new: ptmalloc_unlock_all 0x7ba90-0x7bb3e symbol__new: malloc_printerr 0x7bb40-0x7bc28 symbol__new: malloc_consolidate 0x7c3d0-0x7c9be symbol__new: _int_malloc 0x7dee0-0x7f32b symbol__new: malloc_check 0x7f330-0x7f44a symbol__new: ptmalloc_init.part.8 0x80fe0-0x813d7 symbol__new: ptmalloc_init 0x813e0-0x813f5 symbol__new: malloc_hook_ini 0x81400-0x8143c symbol__new: mallochook 0x82fc0-0x831af symbol__new: tr_mallochook 0x83fc0-0x8407d symbol__new: __malloc_set_state 0x814f0-0x819f6 symbol__new: __malloc_usable_size 0x80d30-0x80e0e symbol__new: __malloc_trim 0x81d50-0x81fac symbol__new: __GI___libc_malloc 0x800c0-0x801b5 symbol__new: __malloc_get_state 0x802e0-0x804db symbol__new: __malloc_stats 0x820c0-0x822aa symbol__new: __malloc_check_init 0x80050-0x800bb symbol__new: __malloc 0x800c0-0x801b5 symbol__new: malloc_set_state 0x814f0-0x819f6 symbol__new: malloc 0x800c0-0x801b5 symbol__new: malloc_info 0x82320-0x8240f symbol__new: malloc_trim 0x81d50-0x81fac symbol__new: malloc_usable_size 0x80d30-0x80e0e symbol__new: malloc_get_state 0x802e0-0x804db symbol__new: __libc_malloc 0x800c0-0x801b5 symbol__new: malloc_stats 0x820c0-0x822aa symbol__new: malloc@plt 0x1f300-0x1f310 Symbol malloc address found : 800c0 Probe point found: __malloc_check_init+96 [root@ssdandy ~]# > > > [root@ssdandy ~]# > > > > > > And then the one asking for 'bytes' to be collectd fails. > > > > > > After processing the other patches I'll try to debug this... > > > > > > [root@ssdandy ~]# cat /etc/redhat-release > > > Red Hat Enterprise Linux Server release 7.0 (Maipo) > > > [root@ssdandy ~]# rpm -q glibc glibc-debuginfo > > > glibc-2.17-55.el7_0.3.x86_64 > > > glibc-debuginfo-2.17-55.el7_0.1.x86_64 > > ^^^ why is this different from the glibc > > version?? > > > > > [root@ssdandy ~]# > > > [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc > > [...] > > > 4849: 00080050 107 FUNCLOCAL DEFAULT 12 > > > __malloc_check_init > > > > __malloc_check_init+96(0x60) becomes 0x80050 + 0x60 = 0x800b0 > > > > > 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc > > > 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12 > > > malloc@@GLIBC_2.2.5 > > > > that is not 0x800c0, so something goes wrong when translating the address > > to probe point. Could you check the patch 1/4 was applied? > > Actually there is a bug in the routine which gets the probe point from > > address. 1/4 fixes it. > > This one? > > commit 0104fe69e0287cf3635657b4c6b26a18e0091697 > Author: Masami Hiramatsu > Date: Mon Mar 2 21:49:46 2015 +0900 > > perf probe: Remove bias offset to find probe point by address > > Remove bias offset to find probe point by address. > > > > Yes, it is applied. > > - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Em Tue, Mar 03, 2015 at 11:39:02AM +0900, Masami Hiramatsu escreveu: > (2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: > > Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: > >> With this patch; > >> - > >> # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc > >> Available variables at malloc > >> @<__libc_malloc+0> > >> size_t bytes > >> # ./perf probe -x /usr/lib64/libc-2.17.so -a "malloc bytes" > >> Added new event: > >> probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) > >> > >> You can now use it in all perf tools, such as: > >> > >> perf record -e probe_libc:malloc -aR sleep 1 > > > >> Reported-by: Arnaldo Carvalho de Melo > > > > Humm, not working for me, after the patch: > > > > [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc > > Available variables at malloc > > @<__malloc_check_init+96> > > (No matched variables) > Will try after a 'make build-test' finishes for the current batch > > [root@ssdandy ~]# > > > > And then the one asking for 'bytes' to be collectd fails. > > > > After processing the other patches I'll try to debug this... > > > > [root@ssdandy ~]# cat /etc/redhat-release > > Red Hat Enterprise Linux Server release 7.0 (Maipo) > > [root@ssdandy ~]# rpm -q glibc glibc-debuginfo > > glibc-2.17-55.el7_0.3.x86_64 > > glibc-debuginfo-2.17-55.el7_0.1.x86_64 > ^^^ why is this different from the glibc > version?? > > > [root@ssdandy ~]# > > [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc > [...] > > 4849: 00080050 107 FUNCLOCAL DEFAULT 12 > > __malloc_check_init > > __malloc_check_init+96(0x60) becomes 0x80050 + 0x60 = 0x800b0 > > > 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc > > 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12 > > malloc@@GLIBC_2.2.5 > > that is not 0x800c0, so something goes wrong when translating the address > to probe point. Could you check the patch 1/4 was applied? > Actually there is a bug in the routine which gets the probe point from > address. 1/4 fixes it. This one? commit 0104fe69e0287cf3635657b4c6b26a18e0091697 Author: Masami Hiramatsu Date: Mon Mar 2 21:49:46 2015 +0900 perf probe: Remove bias offset to find probe point by address Remove bias offset to find probe point by address. Yes, it is applied. - Arnaldo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
(2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: > Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: >> With this patch; >> - >> # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc >> Available variables at malloc >> @<__libc_malloc+0> >> size_t bytes >> # ./perf probe -x /usr/lib64/libc-2.17.so -a "malloc bytes" >> Added new event: >> probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) >> >> You can now use it in all perf tools, such as: >> >> perf record -e probe_libc:malloc -aR sleep 1 > >> Reported-by: Arnaldo Carvalho de Melo > > Humm, not working for me, after the patch: > > [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc > Available variables at malloc > @<__malloc_check_init+96> > (No matched variables) Could you run it with -v (verbose) option? > [root@ssdandy ~]# > > And then the one asking for 'bytes' to be collectd fails. > > After processing the other patches I'll try to debug this... > > [root@ssdandy ~]# cat /etc/redhat-release > Red Hat Enterprise Linux Server release 7.0 (Maipo) > [root@ssdandy ~]# rpm -q glibc glibc-debuginfo > glibc-2.17-55.el7_0.3.x86_64 > glibc-debuginfo-2.17-55.el7_0.1.x86_64 ^^^ why is this different from the glibc version?? > [root@ssdandy ~]# > [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc [...] > 4849: 00080050 107 FUNCLOCAL DEFAULT 12 __malloc_check_init __malloc_check_init+96(0x60) becomes 0x80050 + 0x60 = 0x800b0 > 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc > 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12 malloc@@GLIBC_2.2.5 that is not 0x800c0, so something goes wrong when translating the address to probe point. Could you check the patch 1/4 was applied? Actually there is a bug in the routine which gets the probe point from address. 1/4 fixes it. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: > With this patch; > - > # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc > Available variables at malloc > @<__libc_malloc+0> > size_t bytes > # ./perf probe -x /usr/lib64/libc-2.17.so -a "malloc bytes" > Added new event: > probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) > > You can now use it in all perf tools, such as: > > perf record -e probe_libc:malloc -aR sleep 1 > Reported-by: Arnaldo Carvalho de Melo Humm, not working for me, after the patch: [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @<__malloc_check_init+96> (No matched variables) [root@ssdandy ~]# And then the one asking for 'bytes' to be collectd fails. After processing the other patches I'll try to debug this... [root@ssdandy ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.0 (Maipo) [root@ssdandy ~]# rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.3.x86_64 glibc-debuginfo-2.17-55.el7_0.1.x86_64 [root@ssdandy ~]# [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc 438: 000800c0 245 FUNCGLOBAL DEFAULT 12 __libc_malloc@@GLIBC_2.2.5 545: 00082320 239 FUNCGLOBAL DEFAULT 12 malloc_info@@GLIBC_2.10 810: 000820c0 490 FUNCWEAK DEFAULT 12 malloc_stats@@GLIBC_2.2.5 981: 000802e0 507 FUNCWEAK DEFAULT 12 malloc_get_state@@GLIBC_2.2.5 1077: 003ba740 8 OBJECT WEAK DEFAULT 32 __malloc_hook@@GLIBC_2.2.5 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12 malloc@@GLIBC_2.2.5 1204: 00080d30 222 FUNCWEAK DEFAULT 12 malloc_usable_size@@GLIBC_2.2.5 1450: 00081d50 604 FUNCWEAK DEFAULT 12 malloc_trim@@GLIBC_2.2.5 1767: 003bca60 8 OBJECT WEAK DEFAULT 33 __malloc_initialize_hook@@GLIBC_2.2.5 2061: 000814f0 1286 FUNCWEAK DEFAULT 12 malloc_set_state@@GLIBC_2.2.5 95: 003bbaa0 4 OBJECT LOCAL DEFAULT 33 cache_malloced 1004: 0 FILELOCAL DEFAULT ABS malloc.c 1005: 0007b060 275 FUNCLOCAL DEFAULT 12 ptmalloc_lock_all 1011: 003bcb30 8 OBJECT LOCAL DEFAULT 33 save_malloc_hook 1013: 000801c0 285 FUNCLOCAL DEFAULT 12 malloc_atfork 1017: 0007b180 158 FUNCLOCAL DEFAULT 12 ptmalloc_unlock_all2 1030: 0007ba90 174 FUNCLOCAL DEFAULT 12 ptmalloc_unlock_all 1033: 0007bb40 232 FUNCLOCAL DEFAULT 12 malloc_printerr 1057: 0007c3d0 1518 FUNCLOCAL DEFAULT 12 malloc_consolidate 1089: 0007dee0 5195 FUNCLOCAL DEFAULT 12 _int_malloc 1100: 0007f330 282 FUNCLOCAL DEFAULT 12 malloc_check 1117: 003bca90 4 OBJECT LOCAL DEFAULT 33 disallow_malloc_check 1118: 003bcaa0 4 OBJECT LOCAL DEFAULT 33 using_malloc_checking 1136: 00080fe0 1015 FUNCLOCAL DEFAULT 12 ptmalloc_init.part.8 1138: 000813e021 FUNCLOCAL DEFAULT 12 ptmalloc_init 1139: 0008140060 FUNCLOCAL DEFAULT 12 malloc_hook_ini 1160: 00082fc0 495 FUNCLOCAL DEFAULT 12 mallochook 1162: 003bcbf0 8 OBJECT LOCAL DEFAULT 33 old_malloc_hook 1181: 003bcc40 8 OBJECT LOCAL DEFAULT 33 tr_old_malloc_hook 1182: 00083fc0 189 FUNCLOCAL DEFAULT 12 tr_mallochook 1194: 003bcc60 8 OBJECT LOCAL DEFAULT 33 malloc_trace_buffer 3673: 003ba170 4 OBJECT LOCAL DEFAULT 32 __libc_malloc_initialized 3734: 000814f0 1286 FUNCLOCAL DEFAULT 12 __malloc_set_state 4047: 00080d30 222 FUNCLOCAL DEFAULT 12 __malloc_usable_size 4101: 00081d50 604 FUNCLOCAL DEFAULT 12 __malloc_trim 4338: 000800c0 245 FUNCLOCAL DEFAULT 12 __GI___libc_malloc 4531: 000802e0 507 FUNCLOCAL DEFAULT 12 __malloc_get_state 4569: 000820c0 490 FUNCLOCAL DEFAULT 12 __malloc_stats 4849: 00080050 107 FUNCLOCAL DEFAULT 12 __malloc_check_init 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc 5490: 003bca60 8 OBJECT WEAK DEFAULT 33 __malloc_initialize_hook 5571: 000814f0 1286 FUNCWEAK DEFAULT 12 malloc_set_state 5868: 000800c0 245 FUNCGLOBAL DEFAULT 12 malloc 5878: 00082320 239 FUNCGLOBAL DEFAULT 12 malloc_info 5988: 00081d50 604 FUNCWEAK DEFAULT 12 malloc_trim 6526: 003ba740 8 OBJECT WEAK DEFAULT 32 __malloc_hook 6615: 00080d30 222 FUNCWEAK DEFAULT 12 malloc_usable_size 7087: 000802e0 507 FUNCWEAK DEFAULT 12 malloc_get_state 7104:
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
(2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: With this patch; - # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__libc_malloc+0 size_t bytes # ./perf probe -x /usr/lib64/libc-2.17.so -a malloc bytes Added new event: probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) You can now use it in all perf tools, such as: perf record -e probe_libc:malloc -aR sleep 1 Reported-by: Arnaldo Carvalho de Melo a...@kernel.org Humm, not working for me, after the patch: [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__malloc_check_init+96 (No matched variables) Could you run it with -v (verbose) option? [root@ssdandy ~]# And then the one asking for 'bytes' to be collectd fails. After processing the other patches I'll try to debug this... [root@ssdandy ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.0 (Maipo) [root@ssdandy ~]# rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.3.x86_64 glibc-debuginfo-2.17-55.el7_0.1.x86_64 ^^^ why is this different from the glibc version?? [root@ssdandy ~]# [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc [...] 4849: 00080050 107 FUNCLOCAL DEFAULT 12 __malloc_check_init __malloc_check_init+96(0x60) becomes 0x80050 + 0x60 = 0x800b0 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12 malloc@@GLIBC_2.2.5 that is not 0x800c0, so something goes wrong when translating the address to probe point. Could you check the patch 1/4 was applied? Actually there is a bug in the routine which gets the probe point from address. 1/4 fixes it. Thank you, -- Masami HIRAMATSU Software Platform Research Dept. Linux Technology Research Center Hitachi, Ltd., Yokohama Research Laboratory E-mail: masami.hiramatsu...@hitachi.com -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Em Tue, Mar 03, 2015 at 11:39:02AM +0900, Masami Hiramatsu escreveu: (2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: With this patch; - # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__libc_malloc+0 size_t bytes # ./perf probe -x /usr/lib64/libc-2.17.so -a malloc bytes Added new event: probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) You can now use it in all perf tools, such as: perf record -e probe_libc:malloc -aR sleep 1 Reported-by: Arnaldo Carvalho de Melo a...@kernel.org Humm, not working for me, after the patch: [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__malloc_check_init+96 (No matched variables) Will try after a 'make build-test' finishes for the current batch [root@ssdandy ~]# And then the one asking for 'bytes' to be collectd fails. After processing the other patches I'll try to debug this... [root@ssdandy ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.0 (Maipo) [root@ssdandy ~]# rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.3.x86_64 glibc-debuginfo-2.17-55.el7_0.1.x86_64 ^^^ why is this different from the glibc version?? [root@ssdandy ~]# [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc [...] 4849: 00080050 107 FUNCLOCAL DEFAULT 12 __malloc_check_init __malloc_check_init+96(0x60) becomes 0x80050 + 0x60 = 0x800b0 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12 malloc@@GLIBC_2.2.5 that is not 0x800c0, so something goes wrong when translating the address to probe point. Could you check the patch 1/4 was applied? Actually there is a bug in the routine which gets the probe point from address. 1/4 fixes it. This one? commit 0104fe69e0287cf3635657b4c6b26a18e0091697 Author: Masami Hiramatsu masami.hiramatsu...@hitachi.com Date: Mon Mar 2 21:49:46 2015 +0900 perf probe: Remove bias offset to find probe point by address Remove bias offset to find probe point by address. Yes, it is applied. - Arnaldo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Em Mon, Mar 02, 2015 at 11:45:12PM -0300, Arnaldo Carvalho de Melo escreveu: Em Tue, Mar 03, 2015 at 11:39:02AM +0900, Masami Hiramatsu escreveu: (2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: With this patch; - # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__libc_malloc+0 size_t bytes # ./perf probe -x /usr/lib64/libc-2.17.so -a malloc bytes Added new event: probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) You can now use it in all perf tools, such as: perf record -e probe_libc:malloc -aR sleep 1 Reported-by: Arnaldo Carvalho de Melo a...@kernel.org Humm, not working for me, after the patch: [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__malloc_check_init+96 (No matched variables) Will try after a 'make build-test' finishes for the current batch [root@ssdandy ~]# perf probe -vvv -x /usr/lib64/libc-2.17.so -V malloc probe-definition(0): malloc symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) 0 arguments Open Debuginfo file: /usr/lib/debug/usr/lib64/libc-2.17.so.debug Searching variables at malloc Symbol malloc address found : 800c0 Get 2611 lines from this CU Probe point found: __malloc_check_init+96 Available variables at malloc @__malloc_check_init+96 (No matched variables) [root@ssdandy ~]# If I add one more 'v' I get the symtabs as read by symbol-elf.c and this is what is there for the malloc routines: [root@ssdandy ~]# grep malloc /tmp/4 probe-definition(0): malloc symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) Searching variables at malloc symbol__new: ptmalloc_lock_all 0x7b060-0x7b173 symbol__new: malloc_atfork 0x801c0-0x802dd symbol__new: ptmalloc_unlock_all2 0x7b180-0x7b21e symbol__new: ptmalloc_unlock_all 0x7ba90-0x7bb3e symbol__new: malloc_printerr 0x7bb40-0x7bc28 symbol__new: malloc_consolidate 0x7c3d0-0x7c9be symbol__new: _int_malloc 0x7dee0-0x7f32b symbol__new: malloc_check 0x7f330-0x7f44a symbol__new: ptmalloc_init.part.8 0x80fe0-0x813d7 symbol__new: ptmalloc_init 0x813e0-0x813f5 symbol__new: malloc_hook_ini 0x81400-0x8143c symbol__new: mallochook 0x82fc0-0x831af symbol__new: tr_mallochook 0x83fc0-0x8407d symbol__new: __malloc_set_state 0x814f0-0x819f6 symbol__new: __malloc_usable_size 0x80d30-0x80e0e symbol__new: __malloc_trim 0x81d50-0x81fac symbol__new: __GI___libc_malloc 0x800c0-0x801b5 symbol__new: __malloc_get_state 0x802e0-0x804db symbol__new: __malloc_stats 0x820c0-0x822aa symbol__new: __malloc_check_init 0x80050-0x800bb symbol__new: __malloc 0x800c0-0x801b5 symbol__new: malloc_set_state 0x814f0-0x819f6 symbol__new: malloc 0x800c0-0x801b5 symbol__new: malloc_info 0x82320-0x8240f symbol__new: malloc_trim 0x81d50-0x81fac symbol__new: malloc_usable_size 0x80d30-0x80e0e symbol__new: malloc_get_state 0x802e0-0x804db symbol__new: __libc_malloc 0x800c0-0x801b5 symbol__new: malloc_stats 0x820c0-0x822aa symbol__new: malloc@plt 0x1f300-0x1f310 Symbol malloc address found : 800c0 Probe point found: __malloc_check_init+96 [root@ssdandy ~]# [root@ssdandy ~]# And then the one asking for 'bytes' to be collectd fails. After processing the other patches I'll try to debug this... [root@ssdandy ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.0 (Maipo) [root@ssdandy ~]# rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.3.x86_64 glibc-debuginfo-2.17-55.el7_0.1.x86_64 ^^^ why is this different from the glibc version?? [root@ssdandy ~]# [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc [...] 4849: 00080050 107 FUNCLOCAL DEFAULT 12 __malloc_check_init __malloc_check_init+96(0x60) becomes 0x80050 + 0x60 = 0x800b0 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12 malloc@@GLIBC_2.2.5 that is not 0x800c0, so something goes wrong when translating the address to probe point. Could you check the patch 1/4 was applied? Actually there is a bug in the routine which gets the probe point from address. 1/4 fixes it. This one? commit 0104fe69e0287cf3635657b4c6b26a18e0091697 Author: Masami Hiramatsu masami.hiramatsu...@hitachi.com Date: Mon Mar 2 21:49:46 2015 +0900 perf probe: Remove bias offset to find probe point by address Remove bias offset to find probe point by address. Yes, it is applied. - Arnaldo -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the
Re: Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Em Tue, Mar 03, 2015 at 01:11:17PM +0900, Masami Hiramatsu escreveu: (2015/03/03 12:05), Arnaldo Carvalho de Melo wrote: Em Mon, Mar 02, 2015 at 11:45:12PM -0300, Arnaldo Carvalho de Melo escreveu: Em Tue, Mar 03, 2015 at 11:39:02AM +0900, Masami Hiramatsu escreveu: (2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: With this patch; - # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__libc_malloc+0 size_t bytes # ./perf probe -x /usr/lib64/libc-2.17.so -a malloc bytes Added new event: probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) You can now use it in all perf tools, such as: perf record -e probe_libc:malloc -aR sleep 1 Reported-by: Arnaldo Carvalho de Melo a...@kernel.org Humm, not working for me, after the patch: [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__malloc_check_init+96 (No matched variables) Will try after a 'make build-test' finishes for the current batch Thank you for checking this, [root@ssdandy ~]# perf probe -vvv -x /usr/lib64/libc-2.17.so -V malloc probe-definition(0): malloc symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) 0 arguments Open Debuginfo file: /usr/lib/debug/usr/lib64/libc-2.17.so.debug Searching variables at malloc Symbol malloc address found : 800c0 Get 2611 lines from this CU Hmm, something wrong at here. Probe point found: __malloc_check_init+96 This seems that the debuginfo is a bit odd. Could you also run eu-addr2line for 0x800c0 and 0x800b0? I'm using CentOS7 with a bit newer glibc rpms. # rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.5.x86_64 glibc-debuginfo-2.17-55.el7_0.5.x86_64 And I got following results. [mhiramat@localhost perf]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800c0 __libc_malloc /usr/src/debug/glibc-2.17-c758a686/malloc/malloc.c:2855 [mhiramat@localhost perf]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800b0 __malloc_check_init /usr/src/debug/glibc-2.17-c758a686/malloc/hooks.c:75 So, 0x800b0 correctly points __malloc_check_init+96 but its address is not 0x800c0. [acme@ssdandy linux]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800c0 malloc ??:0 [acme@ssdandy linux]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800b0 __malloc_check_init ??:0 [acme@ssdandy linux]$ If I do it over the debuginfo files: [acme@ssdandy linux]$ addr2line -fi -e /usr/lib/debug/usr/lib64/libc-2.17.so.debug 0x800c0 __malloc_check_init /usr/src/debug/glibc-2.17-c758a686/malloc/hooks.c:75 [acme@ssdandy linux]$ addr2line -fi -e /usr/lib/debug/usr/lib64/libc-2.17.so.debug 0x800b0 __malloc_check_init /usr/src/debug/glibc-2.17-c758a686/malloc/hooks.c:82 [acme@ssdandy linux]$ Tomorrow I'll check for newer packages or any reports about problems with debuginfo files, please let me know if you need any further info, Off to bed, zzZzz - Arnaldo Thank you, Available variables at malloc @__malloc_check_init+96 (No matched variables) [root@ssdandy ~]# If I add one more 'v' I get the symtabs as read by symbol-elf.c and this is what is there for the malloc routines: Yeah, it seems symbol maps works fine. Debuginfo analysis failed. [root@ssdandy ~]# grep malloc /tmp/4 probe-definition(0): malloc symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) Searching variables at malloc symbol__new: ptmalloc_lock_all 0x7b060-0x7b173 symbol__new: malloc_atfork 0x801c0-0x802dd symbol__new: ptmalloc_unlock_all2 0x7b180-0x7b21e symbol__new: ptmalloc_unlock_all 0x7ba90-0x7bb3e symbol__new: malloc_printerr 0x7bb40-0x7bc28 symbol__new: malloc_consolidate 0x7c3d0-0x7c9be symbol__new: _int_malloc 0x7dee0-0x7f32b symbol__new: malloc_check 0x7f330-0x7f44a symbol__new: ptmalloc_init.part.8 0x80fe0-0x813d7 symbol__new: ptmalloc_init 0x813e0-0x813f5 symbol__new: malloc_hook_ini 0x81400-0x8143c symbol__new: mallochook 0x82fc0-0x831af symbol__new: tr_mallochook 0x83fc0-0x8407d symbol__new: __malloc_set_state 0x814f0-0x819f6 symbol__new: __malloc_usable_size 0x80d30-0x80e0e symbol__new: __malloc_trim 0x81d50-0x81fac symbol__new: __GI___libc_malloc 0x800c0-0x801b5 symbol__new: __malloc_get_state 0x802e0-0x804db symbol__new: __malloc_stats 0x820c0-0x822aa symbol__new: __malloc_check_init 0x80050-0x800bb symbol__new: __malloc 0x800c0-0x801b5 symbol__new: malloc_set_state 0x814f0-0x819f6 symbol__new: malloc 0x800c0-0x801b5 symbol__new: malloc_info 0x82320-0x8240f symbol__new: malloc_trim 0x81d50-0x81fac symbol__new: malloc_usable_size 0x80d30-0x80e0e symbol__new: malloc_get_state 0x802e0-0x804db symbol__new: __libc_malloc 0x800c0-0x801b5 symbol__new: malloc_stats 0x820c0-0x822aa
Re: Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
(2015/03/03 12:05), Arnaldo Carvalho de Melo wrote: Em Mon, Mar 02, 2015 at 11:45:12PM -0300, Arnaldo Carvalho de Melo escreveu: Em Tue, Mar 03, 2015 at 11:39:02AM +0900, Masami Hiramatsu escreveu: (2015/03/03 0:46), Arnaldo Carvalho de Melo wrote: Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: With this patch; - # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__libc_malloc+0 size_t bytes # ./perf probe -x /usr/lib64/libc-2.17.so -a malloc bytes Added new event: probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) You can now use it in all perf tools, such as: perf record -e probe_libc:malloc -aR sleep 1 Reported-by: Arnaldo Carvalho de Melo a...@kernel.org Humm, not working for me, after the patch: [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__malloc_check_init+96 (No matched variables) Will try after a 'make build-test' finishes for the current batch Thank you for checking this, [root@ssdandy ~]# perf probe -vvv -x /usr/lib64/libc-2.17.so -V malloc probe-definition(0): malloc symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) 0 arguments Open Debuginfo file: /usr/lib/debug/usr/lib64/libc-2.17.so.debug Searching variables at malloc Symbol malloc address found : 800c0 Get 2611 lines from this CU Hmm, something wrong at here. Probe point found: __malloc_check_init+96 This seems that the debuginfo is a bit odd. Could you also run eu-addr2line for 0x800c0 and 0x800b0? I'm using CentOS7 with a bit newer glibc rpms. # rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.5.x86_64 glibc-debuginfo-2.17-55.el7_0.5.x86_64 And I got following results. [mhiramat@localhost perf]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800c0 __libc_malloc /usr/src/debug/glibc-2.17-c758a686/malloc/malloc.c:2855 [mhiramat@localhost perf]$ eu-addr2line -fi -e /usr/lib64/libc-2.17.so 0x800b0 __malloc_check_init /usr/src/debug/glibc-2.17-c758a686/malloc/hooks.c:75 So, 0x800b0 correctly points __malloc_check_init+96 but its address is not 0x800c0. Thank you, Available variables at malloc @__malloc_check_init+96 (No matched variables) [root@ssdandy ~]# If I add one more 'v' I get the symtabs as read by symbol-elf.c and this is what is there for the malloc routines: Yeah, it seems symbol maps works fine. Debuginfo analysis failed. [root@ssdandy ~]# grep malloc /tmp/4 probe-definition(0): malloc symbol:malloc file:(null) line:0 offset:0 return:0 lazy:(null) Searching variables at malloc symbol__new: ptmalloc_lock_all 0x7b060-0x7b173 symbol__new: malloc_atfork 0x801c0-0x802dd symbol__new: ptmalloc_unlock_all2 0x7b180-0x7b21e symbol__new: ptmalloc_unlock_all 0x7ba90-0x7bb3e symbol__new: malloc_printerr 0x7bb40-0x7bc28 symbol__new: malloc_consolidate 0x7c3d0-0x7c9be symbol__new: _int_malloc 0x7dee0-0x7f32b symbol__new: malloc_check 0x7f330-0x7f44a symbol__new: ptmalloc_init.part.8 0x80fe0-0x813d7 symbol__new: ptmalloc_init 0x813e0-0x813f5 symbol__new: malloc_hook_ini 0x81400-0x8143c symbol__new: mallochook 0x82fc0-0x831af symbol__new: tr_mallochook 0x83fc0-0x8407d symbol__new: __malloc_set_state 0x814f0-0x819f6 symbol__new: __malloc_usable_size 0x80d30-0x80e0e symbol__new: __malloc_trim 0x81d50-0x81fac symbol__new: __GI___libc_malloc 0x800c0-0x801b5 symbol__new: __malloc_get_state 0x802e0-0x804db symbol__new: __malloc_stats 0x820c0-0x822aa symbol__new: __malloc_check_init 0x80050-0x800bb symbol__new: __malloc 0x800c0-0x801b5 symbol__new: malloc_set_state 0x814f0-0x819f6 symbol__new: malloc 0x800c0-0x801b5 symbol__new: malloc_info 0x82320-0x8240f symbol__new: malloc_trim 0x81d50-0x81fac symbol__new: malloc_usable_size 0x80d30-0x80e0e symbol__new: malloc_get_state 0x802e0-0x804db symbol__new: __libc_malloc 0x800c0-0x801b5 symbol__new: malloc_stats 0x820c0-0x822aa symbol__new: malloc@plt 0x1f300-0x1f310 Symbol malloc address found : 800c0 Probe point found: __malloc_check_init+96 [root@ssdandy ~]# [root@ssdandy ~]# And then the one asking for 'bytes' to be collectd fails. After processing the other patches I'll try to debug this... [root@ssdandy ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.0 (Maipo) [root@ssdandy ~]# rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.3.x86_64 glibc-debuginfo-2.17-55.el7_0.1.x86_64 ^^^ why is this different from the glibc version?? [root@ssdandy ~]# [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc [...] 4849: 00080050 107 FUNCLOCAL DEFAULT 12 __malloc_check_init __malloc_check_init+96(0x60) becomes 0x80050 + 0x60 = 0x800b0 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12
Re: [PATCH perf/core 2/4] perf-probe: Fix to handle aliased symbols in glibc
Em Mon, Mar 02, 2015 at 09:49:53PM +0900, Masami Hiramatsu escreveu: With this patch; - # ./perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__libc_malloc+0 size_t bytes # ./perf probe -x /usr/lib64/libc-2.17.so -a malloc bytes Added new event: probe_libc:malloc(on malloc in /usr/lib64/libc-2.17.so with bytes) You can now use it in all perf tools, such as: perf record -e probe_libc:malloc -aR sleep 1 Reported-by: Arnaldo Carvalho de Melo a...@kernel.org Humm, not working for me, after the patch: [root@ssdandy ~]# perf probe -x /usr/lib64/libc-2.17.so -V malloc Available variables at malloc @__malloc_check_init+96 (No matched variables) [root@ssdandy ~]# And then the one asking for 'bytes' to be collectd fails. After processing the other patches I'll try to debug this... [root@ssdandy ~]# cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.0 (Maipo) [root@ssdandy ~]# rpm -q glibc glibc-debuginfo glibc-2.17-55.el7_0.3.x86_64 glibc-debuginfo-2.17-55.el7_0.1.x86_64 [root@ssdandy ~]# [acme@ssdandy linux]$ readelf -Ws /usr/lib64/libc-2.17.so| grep malloc 438: 000800c0 245 FUNCGLOBAL DEFAULT 12 __libc_malloc@@GLIBC_2.2.5 545: 00082320 239 FUNCGLOBAL DEFAULT 12 malloc_info@@GLIBC_2.10 810: 000820c0 490 FUNCWEAK DEFAULT 12 malloc_stats@@GLIBC_2.2.5 981: 000802e0 507 FUNCWEAK DEFAULT 12 malloc_get_state@@GLIBC_2.2.5 1077: 003ba740 8 OBJECT WEAK DEFAULT 32 __malloc_hook@@GLIBC_2.2.5 1170: 000800c0 245 FUNCGLOBAL DEFAULT 12 malloc@@GLIBC_2.2.5 1204: 00080d30 222 FUNCWEAK DEFAULT 12 malloc_usable_size@@GLIBC_2.2.5 1450: 00081d50 604 FUNCWEAK DEFAULT 12 malloc_trim@@GLIBC_2.2.5 1767: 003bca60 8 OBJECT WEAK DEFAULT 33 __malloc_initialize_hook@@GLIBC_2.2.5 2061: 000814f0 1286 FUNCWEAK DEFAULT 12 malloc_set_state@@GLIBC_2.2.5 95: 003bbaa0 4 OBJECT LOCAL DEFAULT 33 cache_malloced 1004: 0 FILELOCAL DEFAULT ABS malloc.c 1005: 0007b060 275 FUNCLOCAL DEFAULT 12 ptmalloc_lock_all 1011: 003bcb30 8 OBJECT LOCAL DEFAULT 33 save_malloc_hook 1013: 000801c0 285 FUNCLOCAL DEFAULT 12 malloc_atfork 1017: 0007b180 158 FUNCLOCAL DEFAULT 12 ptmalloc_unlock_all2 1030: 0007ba90 174 FUNCLOCAL DEFAULT 12 ptmalloc_unlock_all 1033: 0007bb40 232 FUNCLOCAL DEFAULT 12 malloc_printerr 1057: 0007c3d0 1518 FUNCLOCAL DEFAULT 12 malloc_consolidate 1089: 0007dee0 5195 FUNCLOCAL DEFAULT 12 _int_malloc 1100: 0007f330 282 FUNCLOCAL DEFAULT 12 malloc_check 1117: 003bca90 4 OBJECT LOCAL DEFAULT 33 disallow_malloc_check 1118: 003bcaa0 4 OBJECT LOCAL DEFAULT 33 using_malloc_checking 1136: 00080fe0 1015 FUNCLOCAL DEFAULT 12 ptmalloc_init.part.8 1138: 000813e021 FUNCLOCAL DEFAULT 12 ptmalloc_init 1139: 0008140060 FUNCLOCAL DEFAULT 12 malloc_hook_ini 1160: 00082fc0 495 FUNCLOCAL DEFAULT 12 mallochook 1162: 003bcbf0 8 OBJECT LOCAL DEFAULT 33 old_malloc_hook 1181: 003bcc40 8 OBJECT LOCAL DEFAULT 33 tr_old_malloc_hook 1182: 00083fc0 189 FUNCLOCAL DEFAULT 12 tr_mallochook 1194: 003bcc60 8 OBJECT LOCAL DEFAULT 33 malloc_trace_buffer 3673: 003ba170 4 OBJECT LOCAL DEFAULT 32 __libc_malloc_initialized 3734: 000814f0 1286 FUNCLOCAL DEFAULT 12 __malloc_set_state 4047: 00080d30 222 FUNCLOCAL DEFAULT 12 __malloc_usable_size 4101: 00081d50 604 FUNCLOCAL DEFAULT 12 __malloc_trim 4338: 000800c0 245 FUNCLOCAL DEFAULT 12 __GI___libc_malloc 4531: 000802e0 507 FUNCLOCAL DEFAULT 12 __malloc_get_state 4569: 000820c0 490 FUNCLOCAL DEFAULT 12 __malloc_stats 4849: 00080050 107 FUNCLOCAL DEFAULT 12 __malloc_check_init 5351: 000800c0 245 FUNCLOCAL DEFAULT 12 __malloc 5490: 003bca60 8 OBJECT WEAK DEFAULT 33 __malloc_initialize_hook 5571: 000814f0 1286 FUNCWEAK DEFAULT 12 malloc_set_state 5868: 000800c0 245 FUNCGLOBAL DEFAULT 12 malloc 5878: 00082320 239 FUNCGLOBAL DEFAULT 12 malloc_info 5988: 00081d50 604 FUNCWEAK DEFAULT 12 malloc_trim 6526: 003ba740 8 OBJECT WEAK DEFAULT 32 __malloc_hook 6615: 00080d30 222 FUNCWEAK DEFAULT 12 malloc_usable_size 7087: 000802e0 507 FUNCWEAK DEFAULT 12 malloc_get_state 7104:
