Is PTRACE_SINGLEBLOCK buggy?
Hi Roland, hi everybody,
I have finished teaching my spring term so I am back working on utrace.
I am porting my stuff about virtualquare kmview on the new version of
kernels.
I ran into something that seems to be a bug on PTRACE_SINGLEBLOCK.
The source code here enclosed says OKAY on a standard 2.6.25.4,
while it generates a kernel panic on a 2.6.25.4 +
http://people.redhat.com/roland/utrace/2.6-current/linux-2.6-utrace.patch.
Is this a bug? (I think so, no combination of syscall parms should
ever generate kernel panics ;)
Is this a known bug? (e.g. because PTRACE_SINGLEBLOCK is already a WIP
with utrace and you are already working on it...)
ciao
renzo
---
#include stdio.h
#include signal.h
#include sys/wait.h
#include sys/ptrace.h
#include errno.h
static int child(void *arg)
{
if(ptrace(PTRACE_TRACEME, 0, 0, 0) 0){
perror(ptrace traceme);
}
kill(getpid(), SIGSTOP);
return 0;
}
int main()
{
int pid, status, rv;
static char stack[1024];
if((pid = clone(child, stack[1020], SIGCHLD, NULL)) 0){
perror(clone);
return 0;
}
if((pid = waitpid(pid, status, WUNTRACED)) 0){
perror(Waiting for stop);
return 0;
}
ptrace(33, pid, 0, 0); /* PTRACE_SINGLEBLOCK */
printf(OKAY\n);
return 0;
}
Re: Tracing Syscalls under Fedora 9
On Fri, Jun 06, 2008 at 04:38:34PM +0200, Martin Süßkraut wrote: has the tracing of system calls changed in utrace between Fedora 8 and 9? For some reason the only way to trace the syscall is to trace also UTRACE_EVENT(SIGNAL_TERM) or CORE. I added an empty report_signal function and now it works. Martin told me by an E-mail message that the change proposed above solved his problem. This is for the people on the ML concerned with the same trouble. renzo
Re: UTRACE_STOP race condition?
On Wed, Feb 11, 2009 at 09:45:15AM -0500, Frank Ch. Eigler wrote: This may not answer your question, but I believe it is not proper to to make this call at any time t1, only once you receive the quiesce callback. Maybe I am wrong but the quiesce callback gets called *before* the other report_* (say syscall_entry). So when I capture UTRACE_QUIESCE, I got the report call before t=1. Some communication from utrace to the module should happen *after* utrace-stopped is set to 1 (something similar to the code Roland added for ptrace). Even if it worked this way (i.e. return STOP and wait for report_quiesce, I think the race condition there is in any case) the interface to the module would be horrible. When the module receives a report callback, it returns UTRACE_STOP and then it needs to use some data structure to wait for a report_quiesce to restart the traced process. With the idea of patch included in my previous mail there is no need of such a complexity. Thank you for taking part to this discussion renzo
Re: [PATCH] #2 UTRACE_STOP race condition nesting
Dear Roland, dear utrace developers,
I have update also the second patch (which includes the first).
This patch fixes the utrace_stop race condition and
implements a consistent model of tracing engine nesting.
renzo
On Sat, Feb 14, 2009 at 10:11:55AM +0100, Renzo Davoli wrote:
This is an updated patch. It solves the race condition + it gives a quick (a
bit dirty)
solution to issues 34.
3- Nesting, is it really useful to run all the reports in a row and
(eventually) stop and the end waiting for all the engines?
The patch waits for each engine to resume before notifying the next
registered engine.
4- report_syscall_entry engines evaluation order should be reversed
REPORT macros have an extra reverse argument. The macros append this string
to the
list_for_each_entry_safe function name. All the macro calls skip this
argument except
the one in report_syscall_entry where it is set to _reverse.
With this patch it is possible to run nested kmview machines and ptrace works
inside
the virtual machines.
This patch is a bit dirty because variables and sections of code needed to
count and test
the stopped engines are useless here: a task can be kept stopped for at most
one engine at
a time.
This patch is a proof-of concept to show what I meant in my previous message.
For what concerns 12 (not included in this patch):
1- Virtual Machines may need to change the system call
THis is just to simplify the implementation of arch. independent virtual
machine.
I have kept the definition of missing functions in the kmview module code.
2- UTRACE_SYSCALL_ABORT: is it really useful as a return value for
report_syscall_entry?
It is useless for kmview as the decision of aborting the system call is taken
while
the process is stopped, I am currently setting the syscall number to -1 to
skip the syscall.
For the sake of completeness there is another way to implement the partial
virtual machine
stuff by introducing another quiescence state inside the report upcalls.
I mean: when utrace calls a report function (say for example
report_syscall_entry), the function
in the module puts the process in a stopped state (maybe its TASK_TRACED and
calls the schedule).
From utrace's point of view the report function does not return until all
the changes in
the task state have been completed and the decision
UTRACE_RESUME/UTRACE_SYSCALL_ABORT has been taken.
In this way UTRACE_STOP is never used because the module has to implement
another feature
similar to UTRACE_STOP on its own. So what is UTRACE_STOP for?
ciao
renzo
---
--- kernel/utrace.c.mcgrath 2009-03-05 15:09:57.0 +0100
+++ kernel/utrace.c 2009-03-06 11:49:15.0 +0100
@@ -369,6 +369,13 @@
return killed;
}
+static void mark_engine_wants_stop(struct utrace_engine *engine);
+static void clear_engine_wants_stop(struct utrace_engine *engine);
+static bool engine_wants_stop(struct utrace_engine *engine);
+static void mark_engine_wants_resume(struct utrace_engine *engine);
+static void clear_engine_wants_resume(struct utrace_engine *engine);
+static bool engine_wants_resume(struct utrace_engine *engine);
+
/*
* Perform %UTRACE_STOP, i.e. block in TASK_TRACED until woken up.
* @task == current, @utrace == current-utrace, which is not locked.
@@ -378,6 +385,7 @@
static bool utrace_stop(struct task_struct *task, struct utrace *utrace)
{
bool killed;
+ struct utrace_engine *engine, *next;
/*
* @utrace-stopped is the flag that says we are safely
@@ -399,7 +407,23 @@
return true;
}
- utrace-stopped = 1;
+ /* final check: is really needed to stop? */
+ list_for_each_entry_safe(engine, next, utrace-attached, entry) {
+ if ((engine-ops != utrace_detached_ops)
engine_wants_stop(engine)) {
+ if (engine_wants_resume(engine)) {
+ clear_engine_wants_stop(engine);
+ clear_engine_wants_resume(engine);
+ }
+ else
+ utrace-stopped = 1;
+ }
+ }
+ if (unlikely(!utrace-stopped)) {
+ spin_unlock_irq(task-sighand-siglock);
+ spin_unlock(utrace-lock);
+ return false;
+ }
+
__set_current_state(TASK_TRACED);
/*
@@ -625,6 +649,7 @@
* to record whether the engine is keeping the target thread stopped.
*/
#define ENGINE_STOP(1UL _UTRACE_NEVENTS)
+#define ENGINE_RESUME (1UL (_UTRACE_NEVENTS+1))
static void mark_engine_wants_stop(struct utrace_engine *engine)
{
@@ -641,6 +666,21 @@
return (engine-flags ENGINE_STOP) != 0;
}
+static void mark_engine_wants_resume(struct utrace_engine *engine)
+{
+ engine-flags |= ENGINE_RESUME;
+}
+
+static void clear_engine_wants_resume(struct
[PATCH 1/2] UTRACE_STOP race condition (updated)
Dear Roland, dear utrace developers,
I have updated my patch #1 (it solves the race condition on utrace_stop but
not the nesting issue) for the latest version of utrace.
I am trying to get the patches updated downloading, compiling and testing
the fixes every week or so...
Things would be easier if these patch could be merged in the mainstream ;-)
renzo
diff -Naur linux-2.6.29-rc7-git5-utrace/kernel/utrace.c
linux-2.6.29-rc7-git5-utrace-p1/kernel/utrace.c
--- linux-2.6.29-rc7-git5-utrace/kernel/utrace.c2009-03-12
11:00:09.0 +0100
+++ linux-2.6.29-rc7-git5-utrace-p1/kernel/utrace.c 2009-03-12
11:05:50.0 +0100
@@ -376,6 +376,13 @@
return killed;
}
+static void mark_engine_wants_stop(struct utrace_engine *engine);
+static void clear_engine_wants_stop(struct utrace_engine *engine);
+static bool engine_wants_stop(struct utrace_engine *engine);
+static void mark_engine_wants_resume(struct utrace_engine *engine);
+static void clear_engine_wants_resume(struct utrace_engine *engine);
+static bool engine_wants_resume(struct utrace_engine *engine);
+
/*
* Perform %UTRACE_STOP, i.e. block in TASK_TRACED until woken up.
* @task == current, @utrace == current-utrace, which is not locked.
@@ -385,6 +392,7 @@
static bool utrace_stop(struct task_struct *task, struct utrace *utrace)
{
bool killed;
+ struct utrace_engine *engine, *next;
/*
* @utrace-stopped is the flag that says we are safely
@@ -406,7 +414,23 @@
return true;
}
- utrace-stopped = 1;
+ /* final check: it is really needed to stop? */
+ list_for_each_entry_safe(engine, next, utrace-attached, entry) {
+ if ((engine-ops != utrace_detached_ops)
engine_wants_stop(engine)) {
+ if (engine_wants_resume(engine)) {
+ clear_engine_wants_stop(engine);
+ clear_engine_wants_resume(engine);
+ }
+ else
+ utrace-stopped = 1;
+ }
+ }
+ if (unlikely(!utrace-stopped)) {
+ spin_unlock_irq(task-sighand-siglock);
+ spin_unlock(utrace-lock);
+ return false;
+ }
+
__set_current_state(TASK_TRACED);
/*
@@ -632,6 +656,7 @@
* to record whether the engine is keeping the target thread stopped.
*/
#define ENGINE_STOP(1UL _UTRACE_NEVENTS)
+#define ENGINE_RESUME (1UL (_UTRACE_NEVENTS+1))
static void mark_engine_wants_stop(struct utrace_engine *engine)
{
@@ -648,6 +673,21 @@
return (engine-flags ENGINE_STOP) != 0;
}
+static void mark_engine_wants_resume(struct utrace_engine *engine)
+{
+ engine-flags |= ENGINE_RESUME;
+}
+
+static void clear_engine_wants_resume(struct utrace_engine *engine)
+{
+ engine-flags = ~ENGINE_RESUME;
+}
+
+static bool engine_wants_resume(struct utrace_engine *engine)
+{
+ return (engine-flags ENGINE_RESUME) != 0;
+}
+
/**
* utrace_set_events - choose which event reports a tracing engine gets
* @target:thread to affect
@@ -906,6 +946,10 @@
list_move(engine-entry, detached);
} else {
flags |= engine-flags | UTRACE_EVENT(REAP);
+ if (engine_wants_resume(engine)) {
+ clear_engine_wants_stop(engine);
+ clear_engine_wants_resume(engine);
+ }
wake = wake !engine_wants_stop(engine);
}
}
@@ -1133,6 +1177,7 @@
* There might not be another report before it just
* resumes, so make sure single-step is not left set.
*/
+ mark_engine_wants_resume(engine);
if (likely(resume))
user_disable_single_step(target);
break;
Re: [PATCH 2/3] utrace core
On Sat, Mar 21, 2009 at 03:34:57PM +0100, Ingo Molnar wrote: * Renzo Davoli re...@cs.unibo.it wrote: Tracing does not mean only debug. Some tracing facilities can be used for virtualization. For example User-Mode Linux is based on ptrace. I have a prototype of kernel module for virtualization (kmview) based on utrace. [...] Hm, i cannot find the source code. Can it be downloaded from somewhere? Sure! kmview is not included in our Debian packages yet as it relies on (still) non mainstream features (utrace), but the code is available on our view-os svn repository. Check out: svn co https://view-os.svn.sourceforge.net/svnroot/view-os view-os More specifically to browse the code/specifications: The kmview device protocol is here: http://wiki.virtualsquare.org/index.php/KMview_module_interface_specifications The kernel module itself is here: http://view-os.svn.sourceforge.net/viewvc/view-os/trunk/kmview-kernel-module/ The VMM userland application share most of the code with umview, the source code for both is here: http://view-os.svn.sourceforge.net/viewvc/view-os/trunk/xmview-os/xmview/ kmview kernel module (current version) needs the following patches: utrace http://www.mail-archive.com/utrace-devel@redhat.com/msg00654.html http://www.mail-archive.com/utrace-devel@redhat.com/msg00655.html I am trying to keep everything up to date, but the whole stuff is evolving in a quite fast way. Everything has been released under GPLv2. renzo
Bug: report_reap is never called
Hi Ronald utrace developers
I am back...
I am upgrading my kmview support and I have stepped into a clear bug.
in utrace_reap:
--
list_for_each_entry_safe(engine, next, utrace-attached, entry) {
ops = engine-ops;
engine-ops = NULL;
engine-flags = 0;
list_move(engine-entry, detached);
/*
* If it didn't need a callback, we don't need to drop
* the lock. Now nothing else refers to this engine.
*/
if (!(engine-flags UTRACE_EVENT(REAP)))
continue;
The code following this 'if' is never executed (i.e. the reap callback never
called).
In fact it is impossible for (engine-flags UTRACE_EVENT(REAP)) to be
true given that a few statement above engine-flags has been set to 0!
To fix the bug:
clean all the events but reap:
engine-flags = UTRACE_EVENT(REAP);
or save the flag in a temporary var before cleaning it, as you do for
engine-ops.
ciao
renzo
Re: linux-next: add utrace tree
Let me add my two euro-cents to this discussion. Mark Wielaard m...@redhat.com: Unfortunately ptrace does all that magic already (badly). People don't just use it for (s)tracing syscalls, but also for tracing signals, for single step debugging and poking at memory, register state, for process jailing and virtualization (uml) through syscall emulation. So when they are talking about these fancy things that is because that is what ptrace gives them currently. And they hate it, because the ptrace interface is such a pain to work with. And all these things don't really work together. You cannot trace, emulate, debug, jail at the same time. I support Mark's words. I don't use ptrace for debugging/tracing and I have experienced severe limitations of ptrace interface. (I have tried to post some extensions for ptrace to overcome some constraints see my posts on ptrace_vm or ptrace_multi on LKML). Oleg Nesterov, writing to Andrew Morton said: First of all, utrace makes other things possible. gdbstub, nondestructive core dump, uprobes, kmview, hopefully more. I didn't look at these projects closely, perhaps other people can tell more. As for their merge status, until utrace itself is merged it is very hard to develop them out of tree. In the list above there is also kmview, which is a creature of mines. umview and kmview are partial virtual machines, processes running in a [uk]mview machine can have their own view for the file system, networking support, user-id, system-name, etc. A [uk]mview machine virtualizes just what the user need: the filesystem or just a subtree/some subtrees or networking or define one/some virtual devices, etc. The view provided by a [uk]mview machine can be a composition of real resources (provided by the Linux kernel) and virtual resources. Each system call request gets hijacked to a module of [uk]mview when it refers to a virtual resource. The request is forwarded to the kernel otherwise. umview is based on ptrace, kmview uses a kernel module based on utrace. (umview is included in debian lenny (to sid), tutorial and manuals in wiki.virtualsquare.org) IMHO utrace is better than ptrace (or an optimized version of it): 1 - Frank Ch. Eigler wrote: At least one reason is that ptrace is single-usage-only, so for example you cannot concurrently debug strace the same program. - exactly. utrace allows multiple tracing engines, this means that kmview machines can be nested (in a natural way, no extra code is needed for this feature). In the same way strace/gdb can run on virtualized processes, too. 2 - kmview kernel module implements several optimizations to minimize the number of requests forwarded to the kmview process (the virtual machine monitor). kmview is just a module using the utrace interface, prior attempts of optimized umview required kernel patches. Like kmview any other service requiring process tracing can include specific optimizations in its own kernel module. On the other hand, all these services could use the standardized utrace interface for their optimizations, instead asking for messy patches to change code all around the kernel source. 3 - ptrace takes SIGSTOP/SIGCONT for its own management. Strace/gdb and umview cannot be transparent for programs using these signals. Oleg Nesterov talking about Ptrace said: Of course they can't use other interfaces, we don't have them. And without the new abstraction layer we will never have, I think. I agree. THe following list includes the execution times I got in a recent test (make vde-2, see http://www.cs.unibo.it/~renzo/view-os-lk2009.pdf) plain kernel 22.7s, kmview (no modules) 23.9s (+5.5%), full kmview (modules loaded, all syscall virtualized) 38.5s (+70%) optimized umview 51.0 (+124%), umview on vanilla kernel 75.7s (+233%). utrace can be used to speedup virtualization (at least in my case it worked in this way). Performance can be useful for debugging but it is a main issue for virtualization. Kmview module provides optimizations to select the system call requests depending on the syscall number, the pathnames or the file descriptors. http://wiki.virtualsquare.org/index.php/KMview_module_interface_specifications Trying to add all the optimizations needed by different projects to ptrace is a never-ending nightmare: the LKML will continue to receive patch proposals for ptrace... The solution is that everybody can code his/her optimized kernel/user interface for tracing in his/her kernel module, i.e. utrace. renzo
Re: [RFC v2 00/19] utrace for 3.0 kernel
On Mon, Jul 11, 2011 at 06:19:33PM -0700, Josh Stone wrote: On 06/30/2011 05:20 PM, Oleg Nesterov wrote: TODO: - Testing. I ran the whole systemtap testsuite with a kernel built from your git tree, and did not see any utrace-specific issues. Thanks! I have got the git tree, too. I can confirm that also my kmview works on this version of utrace. Thank you Oleg. renzo
