Dear Roland, You are right when you say that the interface specification is a contract between utrace and the module writers. My goal is to use utrace for my virtual machines, your goal is to design utrace as a support for a wide range of applications. I hope your "wide range of applications" will include kmview.
In my perception utrace's support of multiple engines needs a supplement of investigation. I do not want my patches enter utrace code provided there is another fast/clean/easy to code way to reach the same results. It is not for kmview alone, I think this is an example for a range of virtualization application based on utrace. When utrace is used for debugging, "the faster, the better" invariant holds, but when you are dealing with virtualization the rule changes to "the slower, the useless!". Debugging is a temporary state of an application, while virtualization must be designed to be used as a standard environment. Sometimes a picture worth thousands of words. http://www.cs.unibo.it/~renzo/4roland20090323.pdf I have drawn some examples. This is actually a simplified view just to show the problems. The module unreal is a test module for kmview that virtualizes the /unreal subtree as a "copy" of the file system ("/unreal/x/y/z is the file /x/y/z). I know that a so simple transformation could have been implemented directly inside the report_syscall function but kmview is a general support for virtualization. unreal is just a simple test for it. kmview is composed by a kernel module and the "agent" in user space. In the first slide a user runs kmview and inside the vm he/she loads the unreal module and runs a cat command. When cat tries to open "/unreal/etc/passwd", unreal rewrites the path to /etc/passwd, the kernel runs an "open" system call but the arguments have been modified. The report_syscall_entry routine must send the path to kmview in userland and wait for the answer. The number on the arrows show the sequence of actions. The second slide shows a tracing/debugging tool used with virtualization. This is an example of multiple engines working on the same process. strace must read its data before the virtualization for report_syscall_entry. On the contrary the return value shown by strace must be the one returned by the kmview virtualization engine, thus the order for report_syscall_entry is the reverse of that used by report_syscall_exit. Note that if instead of "strace cat /unreal/etc/passwd" our user wrote "strace -f -o /tmp/xxx kmview bash" as the first command the order of the engine would have been inverted. strace in fact should show the system call trace as they appear "outside the virtualization" as one may expect from the command. The third slide shows a nested virtualization and the forth a debug tool running inside a nested virtualization. In all these examples I'd use UTRACE_STOP. Now let us discuss the details of the contract ;-) I set up two different implementations of kmview kernel module. In the standard one (#undefine KMVIEW_NEWSTOP) the report_syscall function returns UTRACE_STOP waiting for the answer from kmview application. The new one (#define KMVIEW_NEWSTOP) uses a semaphore to stop the execution inside the report_syscall function which always returns UTRACE_RESUME. -------------------------------------------------------------------- If you decide that the right implementation is the former (#undefine KMVIEW_NEWSTOP): - please tell me how to implement the example of page 3 if in the management of syscall_entry for kmview2 does not stop prior to call kmview1. Okay, you say kmview1's module receives a notification that another engine wants to stop reading its @action argument but it needs the state as modified by kmview2. - I could set up some kind of synchronization among kmview machines but the solution would be extremely weak. What about if kmview run nested with another virtualization/tracing application based on utracei e.g. strace? - You say "use UTRACE_REPORT" to wait for the other machines are done fiddling with it. The comment you wrote about UTRACE_REPORT says: * This is like %UTRACE_RESUME, but also ensures that there will be * a @report_quiesce or @report_signal callback made soon. If * @target had been stopped, then there will be a callback before it * resumes running normally. If another engine is keeping @target * stopped, then there might be no callbacks until all engines let * it resume. But if kmview1 and 2 have both stopped the report_syscall so no callback will be called until both finishes. Otherwise you may mean that kmview1 returns UTRACE_RESUME and when kmview1's report quiesce get called it returns UTRACE_STOP. In this way the management of the system call should be moved from the report_syscall_entry to report_quiesce but just for kmview1. Which one is the cleaner way to implement a service on utrace in you opinion? In my opinion the possibility to have the process blocked before calling the next report function leads to simpler code. Was this design choice chosen for efficiency? I feel that all this long sequence of report callbacks ends up slowing down the virtualization. "the slower the useless" I said.... Are you sure that each engine should examine by themselves what the other engines do, as utrace provides almost no synchronization rules between them? For sure you have in your mind examples where engines have to run concurrently when one or more return UTRACE_STOP. But there are other cases in which you need to stop a process before calling the next engine's report function. Instead of changing the semantics of UTRACE_STOP you could add a UTRACE_STOP_NOW return value to stop the engine before calling the next engine's report function ------------------------------------------------------------------- If you decide that the latter implementation is the right one (#define KMVIEW_NEWSTOP) - This means that I am not using UTRACE_STOP at all. I have implemented another way the support to stop a process. I don't think it is a good idea to stay in the report function for a long time, UTRACE_STOP was designed for that purpose. - The management of asynchronous events is harder as the process can be stopped in many "levels" of the architecture. - If you say that this is the right way to do it, I'll keep this code but I'll be wondering what is UTRACE_STOP for. ------------------------------------------------------------------- In both cases the order of report_syscall_entry report function must be reversed (with respect to all the other report functions) otherwise all the nested engine examples fail. ciao renzo Note: the actual kmview and unreal work in a slightly different way. This final note is useful if you want to read the code or run the examples otherwise it can be safely skipped. 1- kmview VMM (the agent) does not rewrite the path but open the file itself. 2- the nested kmview VMM itself runs in the space virtualized by the outer kmview. The drawings would have been more complex but the problem is the same, a process running in a nested kmview has one utrace engine for each kmview. 3- actual unreal provides two levels of /unreal. kmview + unreal provide /unreal and /unreal/unreal as copies of the file system kmview+unreal+kmview+unreal (nested) provide /unreal /unreal/unreal /unreal/unreal/unreal and /unreal/unreal/unreal/unreal.
