----- On Sep 22, 2016, at 8:47 AM, Milian Wolff [email protected] wrote:
> On Wednesday, September 21, 2016 6:09:15 PM CEST Mathieu Desnoyers wrote: >> ----- On Sep 20, 2016, at 4:34 PM, Milian Wolff [email protected] wrote: >> > On Montag, 12. September 2016 16:24:04 CEST Mathieu Desnoyers wrote: >> >> ----- On Sep 12, 2016, at 11:40 AM, Milian Wolff [email protected] >> > >> > wrote: >> >> > On Monday, September 12, 2016 3:03:04 PM CEST Mathieu Desnoyers wrote: >> >> >> ----- On Sep 6, 2016, at 3:00 PM, Milian Wolff [email protected] >> > >> > wrote: >> >> >> > Hey all, >> >> >> > >> >> >> > where can I find more documentation on how to use sdt.h to add >> >> >> > static >> >> >> > tracepoints to user-land applications? If this is not the right >> >> >> > place >> >> >> > to >> >> >> > ask, please refer to to elsewhere. >> >> >> >> >> >> Hi Milian, >> >> > >> >> > Hey Mathieu, >> >> > >> >> >> LTTng-UST offers a "tracepoint" instrumentation facility, which can >> >> >> optionally emit "sdt.h" probe points too for compatibility with >> >> >> SystemTAP. >> >> >> >> >> >> > I plan to upstream a collection of tracepoints to Qt, and possibly >> >> >> > elsewhere. One problem I'm having right now is figuring out how to >> >> >> > "design" the tracepoints such that they have minimal overhead. >> >> >> >> >> >> The main question here would be: do you want your instrumentation to >> >> >> be >> >> >> usable with LTTng-UST ? If yes, then you want the tracepoint >> >> >> instrumentation facility. Else, if you only plan on using SystemTAP, >> >> >> you >> >> >> can use sdt.h instrumentation. >> >> >> >> >> >> > So my questions: >> >> >> My answers will be about lttng-ust tracepoints. >> >> > >> >> > Thanks for the clarifications! Are the LTTNG-UST tracepoints also >> >> > "zero- >> >> > overhead" like the SystemTap once, i.e. put into a separate section and >> >> > "only" one nop is added when tracing is disabled? I could not find such >> >> > an information anywhere yet. >> >> >> >> The lttng-ust tracepoint mechanism comes from the design of the >> >> Linux kernel tracepoints, which have proven to be unnoticeable >> >> when disabled. There is one slight technicality though: >> >> >> >> The linux kernel tracepoints use the asm goto mechanism and code patching >> >> to flip between a no-op and a jump to dynamically enable each tracepoint. >> >> This is all very fine for the kernel. >> >> >> >> Now for user-space, the lttng-ust tracepoints rather use a conditional >> >> branch to skip over the entire stack setup and the function call. There >> >> are a few reasons for using a old-fashioned conditional branch: first, >> >> there are no widely adopted multi-core, live code patching library >> >> available in user-space that am I aware of. Second, even if we would >> >> have one, we may not want to send SIGSTOP/SIGCONT to the traced process, >> >> due to debugger interactions. Thirdly, doing code patching may not >> >> interact will security features such as read-only code sections and >> >> code checksumming. Finally, doing code patching of library and >> >> executable code will trigger copy-on-write of the touched pages, which >> >> will prevent sharing those pages between processes running the same >> >> apps/libraries. All in all, the conditional branch does not seem like >> >> a too bad alternative after all. >> >> >> >> If we look at the code generated by SystemTAP sdt.h probes (as well >> >> as DTrace), they are actually more heavyweight than what has been >> >> claimed by Sun's marketing department back in the days: it does >> >> cost a stack setup of all the variables passed to the the probe, >> >> and indeed the function call is no-op'd. >> >> >> >> The downside here is that all side-effects, and layout of the arguments >> >> for the no-op'd call, are taken by the instrumented application, even >> >> though tracing is not dynamically enabled. >> >> >> >> Therefore, the SDT mechanism is not a lightweight as is generally >> >> claimed. It is not "just a single no-op per site", but rather a >> >> function call stack setup and a no-op. >> > >> > Thanks a lot for the in-depth explanation Mathieu, much appreciated! From >> > what I gather, sdt.h does allow to check whether a given trace point is >> > enabled or not, no? I.e. via the semaphores. Is there any performance >> > difference between checking the semaphore of a sdt.h tracepoint vs. doing >> > the analogous check for lttng-ust tracepoints? >> >> Not sure what you refer to about "semaphores". And it's unclear how you >> define "checking if active": is it to decide in the fast path whether to >> trace or not, or in a slow path to query what is currently active ? > > The fast path to check whether to trace or not: > > $ cat probes.d > provider milian { > probe test(); > }; > $ dtrace -C -h -s probes.d -o probes.h > $ cat probes.h > /* Generated by the Systemtap dtrace wrapper */ > > > > > > > #define _SDT_HAS_SEMAPHORES 1 > > > > > > > #define STAP_HAS_SEMAPHORES 1 /* deprecated */ > > > > > > > #include <sys/sdt.h> > > > > /* MILIAN_TEST ( ) */ > #if defined STAP_SDT_V1 > #define MILIAN_TEST_ENABLED() __builtin_expect (test_semaphore, 0) > #define milian_test_semaphore test_semaphore > #else > #define MILIAN_TEST_ENABLED() __builtin_expect (milian_test_semaphore, 0) > #endif > __extension__ extern unsigned short milian_test_semaphore __attribute__ > ((unused)) __attribute__ ((section (".probes"))); > #define MILIAN_TEST() \ > DTRACE_PROBE (milian, test) Got it. The performance difference between lttng-ust tracepoint_enabled() check and the sdt.h semaphore check will be none, because both perform an unlikely branch (using builtin expect 0). Thanks, Mathieu > > > -- > Milian Wolff | [email protected] | Software Engineer > KDAB (Deutschland) GmbH&Co KG, a KDAB Group company > Tel: +49-30-521325470 > KDAB - The Qt Experts > _______________________________________________ > lttng-dev mailing list > [email protected] > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com _______________________________________________ lttng-dev mailing list [email protected] https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
