(missing the RFC :P) Hi everyone,
Introduce a complete new proposal for LTTng and UST tracing infrastructure and control.
This is the third revision (v0.3) of that proposal. Comments from Nils Carlson and Michel Dagenais were added. Nothing final, we are still in the development process.
Again, questions and comments are more than welcome! Cheers David -- David Goulet LTTng project, DORSAL Lab. PGP/GPG : 1024D/16BD8563 BE3C 672B 9331 9796 291A 14C6 4AF7 C14B 16BD 8563
RFC - New processes model for UST and LTTng Author: David Goulet <[email protected]> Contributors: * Mathieu Desnoyers <[email protected]> * Yannick Brosseau <[email protected]> * Nils Carlson <[email protected]> * Michel Dagenais <[email protected]> * Stefan Hajnoczi <[email protected]> Version: - v0.1: 17/01/2011 * Initial proposal - v0.2: 19/01/2011 After multiple reply from all the contributors above, here is the list of what has changed: * Change/Add Terminology elements from the initial model * New figures for four new scenarios * Add inprocess library section * LTTng kernel tracer support proposition * More details for the Model and Components * Improve the basic model. Quite different from the last one - v0.3: 28/01/2011 In response from Michel Dagenais and Nils Carlson comments: * Add scaling to reasons of this re-engineering * Purpose of the session ID * Explain why ltt-sessiond creates the tracing buffers * ust-consumerd interaction schema * Clarify inprocess library behavior Terminology ----------------- ltt-sessiond - Main daemon for trace session registry for UST and LTTng ust-consumerd - Daemon that consume UST buffers for a specific application ltt-consumerd - Daemon that consume LTTng buffers tracing session - A trace linked to a set of specific tracepoints tracing buffers - Buffers containing tracing data tracing data - Data created by tracing an application inprocess library - UST library linked with the application shared memory - system V shared memory Model ----------------- This RFC propose brand new UST and LTTng daemon model. This re-engineering was mostly driven by the need of better security in terms of access rights, having tracing session, scaling for the tracing infrastructure, LTTng and UST integration and networking such as streaming and remote control over different traces. The new model follows the basic principles of having a session registry (ltt-sessiond) and consumers for each tracing buffers (ust-consumerd and ltt-consumerd). With this proposal, LTTng and UST will share the same tracing session, be manage by the same tool and bring a complete integration between these two powerful tools. Note that this proposal does NOT makes UST dependent on LTTng and vice versa. Each component's roles are explained in the following sections. LTT-SESSIOND: The ltt-sessiond daemon acts as a session registry i.e. by keeping reference to all active session and, by active, it means a session in any state other than destroyed. Each entity we are keeping track of, here session, will have a unique identifier (ID) assign to it. The purpose of this ID is to be able for drtrace to apply any kind of actions (Ex: Attach, Destroy). It MUST be human readable in order to facilitate the session identification when listed by drtrace. This unique identifier SHOULD be a unique string consisting of the username, a session name and the number of already created session plus one. The last number makes sure that the ID is unique in the ltt-sessiond context. In the implementation, a simple number could be use for performance purpose instead of comparing string at each iteration. Ex: Username - "dave", Session name - "mysession", Num. sessions - fifth one --> ID: dave.mysession-5 This daemon is also responsible for tracing buffer creation. Two main reasons motivate this design: * The shared memory segment needs to be in the tracing group. We can only assume that ltt-sessiond, being root (UID = 0), is always able to do that. * The ltt-sessiond needs to keep track of all the shared memory segments in order to be able to give reference to any other consumer since it is this daemon that controls the access to all tracing buffers. A tracing session is associated to a user of the system (UID). Unless a user is in the tracing group, no other users can access the tracing data for that session. The trace roles of ltt-sessiond: Trace interaction - Create, Destroy, Pause, Stop, Start, Set options Registry - keep track of tracing session information: * shared memory location (only the keyid) * application PID * session name * UID/GID of the session user Session access control - allow or deny session interaction based on the user credentials and/or session ID. Buffers creation - creates shared memory for the tracing buffers. UST-CONSUMERD: The purpose of this daemon is to consume the UST trace buffers for only a specific session. The session MAY have several traces for example two different applications. The client tool, drtrace, is in charge of creating a ust-consumerd if NONE is available for that session. It is very important to understand that for a tracing session, there is only one ust-consumerd for all the traced applications. This daemon basically empty the tracing buffers when asked for and write that data to disk for future analysis using LTTv or/and TMF (Tracing Monitoring Frameworks). Upon creation, that daemon UID/GID is set to the user credentials and so are the data files on disk. Here is a flow of action to illustrate the ust-consumerd life span: 1) +-----------+ ops +--------------+ | drtrace A |<---------->| ltt-sessiond | +-----------+ +--------------+ drtrace ask for tracing an application using the PID and the session ID. If it is allowed, meaning that drtrace needs to be either in the tracing group or the same UID as the application, the shared memory reference is given to drtrace and the ust-consumerd communication socket if ust-consumerd already exist. 2a) If ust-consumerd EXIST +-----------+ | drtrace A | +-----------+ | mem ref. | +---------------+ read +------------+ +-->| ust-consumerd |--------->| shared mem | +---------------+ +------------+ In that case, drtrace only ask ust-consumerd to consume the buffers using the reference it previously got from ltt-sessiond. 2b) If ust-consumerd DOES NOT EXIST +-----------+ +--------------+ | drtrace A | +---->| ltt-sessiond | +-----------+ | +--------------+ | ID | | mem ref. | register | +---------------+ +-->| ust-consumerd | +---------------+ drtrace spawns the ust-consumerd for the session using the session ID in order for the daemon to register as a consumer to ltt-sessiond for that session. On the security side, the ust-consumerd gets UID/GID from the drtrace credentials since it was spawned by drtrace and so the files containing the tracing data will also be set to UID/GID of the drtrace client. The roles of ust-consumerd: Register to ltt-sessiond - Using a session ID and credentials (UID/GID) Consume buffers - Write data to a file descriptor (on disk, network, ...) Communicate with the application for buffers options LTT-CONSUMERD: The purpose of this daemon is to consume the LTTng trace buffers for only a specific session. For that kernel consumer, ltt-sessiond will pass different anonymous file descriptors to the ltt-consumerd using a Unix socket. From these file desriptors, it will be able to get the data from a special function export by the LTTng kernel. ltt-consumerd will be manage by the exact same way as ust-consumerd. However, in order to trace the kernel, you are either root (UID=0) or in the tracing group. The roles of ltt-consumerd: Register to ltt-sessiond - Using a session ID and credentials (UID/GID) Consume buffers - Write data to a file descriptor (on disk, network, ...) Communicate with the kernel for buffers information and with ltt-sessiond for command. UST INPROCESS LIBRARY: When the application starts, this library will check for the global name pipe of ltt-sessiond. If it is present, it MUST validate that root:root is the owner. This check is very important to prevent ltt-sessiond spoofing. If the pipe is root, we have the assurance that it's the privilege user that operates tracing. >From there, it will register as a valid traceable application using the apps credentials and will open a pipe in order to receive an eventual shared memory reference. If not present, the application opens a unix socket using the PID has unique identifier and wait on it for request from ltt-sessiond. If one bytes is written to that pipe, the application will try again to register to ltt-sessiond. Again, the application try to connect to ltt-sessiond and not the contrary since we don't want any unauthorized user being able to talk to the library and control tracing. (Future works, private ltt-sessiond if no global ltt-sessiond is found.) SHARED MEMORY For UST, this is the memory area where the tracing buffers will be held and given access in write mode for the inprocess library of the application and in readonly mode for the ust-consumerd. On the LTTng side (for ltt-consumerd), these buffers are in the kernel space and given access by opening a file in the debugfs file system. With an anonymous file desriptor, this consumer will be able to extract the data. This memory is ONLY used for the tracing data. No communication between components is done using that memory. A shared memory segment for tracing MUST be set with the tracing group GID for the UST buffers. This is the job of ltt-sessiond. PREREQUISITES: The ltt-sessiond daemon MUST always be running as "root" or an equivalent user having the same privilege as root (UID = 0). The ltt-sessiond daemon MUST be up and running at all time in order to trace a tracable application. (Future works will propose multiple daemon coexistence). The new drtrace command line tool MUST be use to interact with the ltt-sessiond registry daemon for every trace action needed by the user. A tracing group MUST be created. Whoever is in that group is able to access the tracing data of any buffers and is able to trace any application or the kernel. WARNING: This group name MAY interfere with other linux apps using the same group name. Care should be put at install time for that (from source and packages) Access Control ----------------- The next section illustrates different use cases using that new model. Use Cases ----------------- Each case considers these : * user A - UID: A; GID: A, tracing * user B - UID: B; GID: B, tracing Scenario 1 - Single user tracing app_1 This first scenario shows how user A will start a trace for application app_1 that is not running. 1) drtrace ask ltt-sessiond for a new session through a Unix socket. If allowed, ltt-sessiond returns a session ID to the client. (Ex: ops --> new session) +-----------+ ops +--------------+ | drtrace A |<---------->| ltt-sessiond | +-----------+ +--------------+ 2) The app_1 is spawned by drtrace having the user A credentials. Then, app_1 automatically register to ltt-sessiond has a "tracable apps" through the global name pipe of ltt-sessiond using the UID/GID and session ID. The shared memory is created with the app_1 UID (rw-) and tracing group GID (r--) and a reference is given back to app_1 +-----------+ +--------------+ | drtrace A | | ltt-sessiond | +-----------+ +--------------+ | ^ | | +-------+ | | +-------------+ +-->| app_1 |<--------+ +-->| shared mem. | +-------+ +-------------+ 3) app_1 connect to the shared memory and ust-consumerd is spawned with the session ID and drtrace credentials (user A). It then register to ltt-sessiond for a valid session to consume using the previous session ID and credentials. +-----------+ +--------------+ | drtrace A | +-->| ltt-sessiond | +-----------+ | +--------------+ | | | +---------------+ read +-->| ust-consumerd |---------+ +---------------+ v ^ | +-------------+ | v +------>| shared mem. | +-------+ | +-------------+ | app_1 |-------- +-------+ write Scenario 2 - Single user tracing already running app_1 1) drtrace ask ltt-sessiond for a new session through a Unix socket. If allowed, ltt-sessiond returns a session ID to the client. +-----------+ ops +--------------+ | drtrace A |<---------->| ltt-sessiond | +-----------+ +--------------+ ^ +-------+ read | | app_1 |----------+ +-------+ NOTE: At this stage, since app_1 is already running, the registration of app_1 to ltt-sessiond has already been done. However, the shared memory segment is not allocated yet until a trace session is initiated. Having no shared memory, the inprocess library of app_1 will wait on a name pipe connected to ltt-sessiond for the reference. +-----------+ +--------------+ | drtrace A | | ltt-sessiond | +-----------+ +--------------+ ^ | +-------+ | | +-------------+ | app_1 |<--------+ +-->| shared mem. | +-------+ +-------------+ | ^ +---------- write ----------+ 2) drtrace spawns a ust-consumerd for the session. We get the same figure as step 3 in the first scenario. There is a small difference though. The application MAY NOT be using the same credentials as user A (drtrace). However, the shared memory is always GID of the tracing group. So, in order for user A to trace app_1, is MUST be in the tracing group otherwise, if the application is not set with the user credentials, user A will not be able to trace app_1 Scenario 3 - Multiple users tracing the same running application 1) Session are created for the two users. Using the same exact mechanism as before, the shared memory and consumers are created. Two users, two sessions, two consumers and two shared memories for the same application. +-----------+ +--------------+ | drtrace A |-------- ops ------->| ltt-sessiond | +-----------+ ^ +--------------+ | +-----------+ | +-------+ | drtrace B |------+ +--->| app_1 |------- write -----+ +-----------+ | +-------+ | | | +-----------------+ | +-------------+ | | ust-consumerd A |--O--- read ----->| shared mem. |<-+ +-----------------+ | +-------------+ | | | +-----------------+ v +-------------+ | | ust-consumerd B |--+--- read ----->| shared mem. |<-+ +-----------------+ +-------------+ NOTE: No link was made to ltt-sessiond, but all consumers are connected to the session daemon ust-consumerd A - UID: user A (rw-), GID: tracing (r--) ust-consumerd B - UID: user B (rw-), GID: tracing (r--) Scenario 4 - User not in the tracing group For this particular case, it's all goes back to the first scenario. The user MUST start the application using his credentials. The session will be created by ltt-sessiond but he will not be able to trace anything that the user does not owned.
_______________________________________________ ltt-dev mailing list [email protected] http://lists.casi.polymtl.ca/cgi-bin/mailman/listinfo/ltt-dev
