Here is a vague start at the directions I have in mind for a user-level
interface that I've been calling "ntrace". It's not a real specific plan
at the literal interface level. It's an overview of what the components
are that the bit-level definition of the user-level interface sits atop.
I'll start with a couple of terms that I'll use later throughout the
discussion.
A tracing session is one independent use of the interface, e.g. one
debugger application might have one session to handle many debugees.
Different sessions don't interact or know about each other directly.
A session ends when its users go away, so it always ends automatically
when the debugger applications die suddenly and so forth. When a
session ends, all its state goes away ceases to mean anything.
Everything I'll talk about later is meant within the context of one
session. (At some point in the future we might deal with nuances
beyond this, but ignore that for now.)
A tracing "channel" is the term I'll use to encompass the whole subject
of transport. We'll leave the details of transport aside here, and just
say what the essential characteristics of a channel are. In the long
run, there can be multiple kinds of channels and there can be multiple
channels in one session. To start with, we'll only worry about a single
channel per session.
A channel supplies a security context. (I'll talk separately about the
security model, and we can leave it aside for now.) A channel can be a
source of commands. A channel can send data back to the user.
For sending data, a channel might have various buffering options and
characteristics available. Sending to the channel per se has to be
nonblocking in the kernel. So e.g., there may be fill-and-drop or
circular buffer options. For lossless buffering, a channel may need to
ask the sending thread to wait for a buffer drain. Internally, this may
mean using UTRACE_STOP to hold a thread in check until a channel-drained
callback lets it resume to generate more data. We don't have to worry
about all that immediately. But these are some of the subtleties
between the "channel" layer and the "core" of the thing.
I think of the interface as asynchronous at base. There may at some
point be some synchronous calls to optimize the round trips. But we
know that by its nature an interface for handling many threads at once
has to be asynchronous, because an event notification can always be
occurring while you are doing an operation on another thread. So what
keeping it simple means for the first version is that we only worry
about the asynchronous model. Think of it like a network protocol
between the application and the "tracing server" inside the kernel.
(Don't get hung up on that metaphor if it sounds like a complication to
you.)
One thing we know we need is to send multiple commands to the interface
in a single channel transaction (i.e. system call). In the general
case, we might have both datagram and stream channels. But we might as
well only consider the stream case since we have to anyway; i.e., we
can't rely on any intrinsic packetization, we have to recognize
boundaries between separate requests in-band in the command channel.
I think what is the first most important thing about the interface is
not anything much about what's in it. It's rapid prototyping. It needs
to be simple on the kernel side to write a new function, with a minimum
of boilerplate, add something to at most two tables, and recompile the
module, to have your new command available in the user-level interface.
As for encoding, I think everything can be composed of "words" and
"blobs". A word is long/unsigned long in kernel-land. A blob means a
raw chunk of bytes. To keep words aligned, I'd encode e.g.
{word=1,blob=A,word=2,blob=B} as:
word 0: 1
word 1: <length of blob A>
word 2: 2
word 3: <length of blob B>
: <blob A contents><blob B contents>
: <padding if lenA+lenB unaligned>
word n: next unrelated part of the stream
This keeps it straightforward, both on the kernel side and in user-level
interface front ends, to add a new command or report-back that takes n
words and m blobs meaning something or other.
I tend to over-engineer these things. So I'd been thinking about a
discoverable framework for the command vocabulary. Down the road I do
want to have some dynamic extensibility. But we shouldn't get bogged
down in that now.
Job #1 is to get something hobbling along that is easy to hack on, easy
to change, easy to play with, where you pull the cat's tail in user mode
and he's meowing in the kernel module (except there is no cat--ok you
can use a cat if you want to, I said I wasn't addressing transport).
In http://sourceware.org/ml/systemtap/2008-q2/msg00459.html I introduced
"tracing groups" (sorry, it's about half way down that long message). It
was slightly elaborated in that thread. I will talk all about groups in
another posting (sorry, will be another bit of delay before I can have that
all written up).
Some ideas about groups have a lot to do with how I'd like to structure the
command set. But you don't need to wait to hash that out. Get started
with whatever commands make sense and are straightforward to use to
demonstrate using the fundaments of the interface. Don't worry that
everything will get reorganized as we figure it all out (it will, just
don't worry about it).
Thanks,
Roland
