Masami Hiramatsu writes:
> On Mon, 05 Sep 2016 16:37:01 +0200
> Lluís Vilanova <vilan...@ac.upc.edu> wrote:
>> Stefan Hajnoczi writes:
>>
>> > On Mon, Aug 29, 2016 at 08:46:02PM +0200, Lluís Vilanova wrote:
>> >> >> Also, I'm still not sure how to interact with QEMU's monitor interface
>> >> >> from
>> >> >> within the probe code (probes execute in kernel mode, including "guru
>> >> >> mode"
>> >> >> code).
>> >>
>> >> > When SystemTap is used the QEMU monitor interface does nothing.
>> >>
>> >> That's not what I've experienced. I was able to use a stap script to
>> >> change the
>> >> tracing state of events:
>> >>
>> >> #!/usr/bin/env stap
>> >>
>> >> %{
>> >> #include </home/lluis/Projects/qemu-dbi-test/test.h>
>> >> %}
>> >>
>> >> function event:long(cpu:long, addr:long, info:long)
>> >> %{
>> >> char *argv[4] = {"/bin/sh", "-c", "echo 'trace-event * off' | telnet
>> >> localhost 1234", NULL};
>> >> call_usermodehelper(argv[0], argv, NULL, UMH_WAIT_EXEC);
>> >> STAP_RETURN(0);
>> >> %}
>> >>
>> >> probe begin {
>> >> printf("hello\n")
>> >> }
>> >> probe
>> >> process("./install/vanilla/bin/qemu-system-i386").mark("guest_mem_before_exec")
>> >> {
>> >> printf("%x %d %d\n", $arg1, $arg2, $arg3)
>> >> event($arg1, $arg2, $arg3)
>> >> exit()
>> >> }
>> >>
>> >> The only caveat is that you must pass the "-g" argument to stap.
>> >>
>> >> Also, for some reason the printf in the probe always prints zeros, no
>> >> matter
>> >> what the actual event receives (I've debugged QEMU down to the call to the
>> >> auto-generated stap functions). Could this be an error in systemtap?
>>
>> > It's strange that arguments do not have valid values. Debugging the
>> > stap functions is the next step if you want to figure out what happened.
>> > I've never had this issue before so maybe something with Debian
>> > SystemTap userspace probes is broken.
>>
>> I already debugged it, to the point where QEMU executes the trap injected by
>> systemtap, and the register values that were supposed to hold the arguments
>> are
>> correct.
>>
>> I suppose that if you execute the stap script I pasted it will show the
>> proper
>> values. Then it's definitely a problem with Debian's userspace probes.
> Would you have tried to update your kernel to mainline and tested it ?
I've compiled the tarball for 4.8-rc5 (.config from "make localmodconfig") and a
printf of the probe arguments still shows zeroes. Also, I've had to add a small
patch to [1] to properly lock/unlock inodes in this kernel version (using
inode_lock/unlock instead of mutex_lock/unlock on inode->i_mutex).
[1] /usr/share/systemtap/runtime/transport/transport.c
> If it occurs, you also should try to use a raw uprobe via
> ftrace(uprobe_events)
> and perftools.
> If you have the latest perf (maybe you'll need checkout the latest tip tree),
> you can use SDT as below (currently it doesn't support args, so you'll need
> debuginfo.)
> # perf buildid-cache --add ./install/vanilla/bin/qemu-system-i386
> # perf probe -x ./install/vanilla/bin/qemu-system-i386 -a
> 'guest_mem_before_exec $vars'
> And you'll see new event is registered which can be traced by ftrace or perf.
It does show something (I'm interested in stap probe "guest_hypertrace", raised
on function "trace_guest_hypertrace"), but is incorrect:
50.00% (55e36d5ee32b) __cpu=0x55e370446fd0 arg1=0xcafe
50.00% (55e36d5ee32b) __cpu=0x7f2ee0a10b20 arg1=0x198
My test app calls "trace_guest_hypertrace" twice, always with the same "__cpu"
and "arg1" argument values. Just in case, this is runnign a QEMU compiled with
"-O0".
Running "record" multiple times shows different "random" values on the
arguments, and keeps changing which of the two trace elements shows incorrect
values.
Thanks,
Lluis
