Merged, and I pushed a symlink from the .txt -> .md. Thanks!
Mathieu ----- Original Message ----- > From: "Philippe Proulx" <[email protected]> > To: [email protected] > Sent: Wednesday, January 21, 2015 11:51:16 PM > Subject: [lttng-dev] [PATCH ctf] Modernize CTF specification using Markdown > > Content is unchanged, with the exception of a few > minor typos fixed here and there. > > Signed-off-by: Philippe Proulx <[email protected]> > --- > common-trace-format-specification.md | 2035 > +++++++++++++++++++++++++++++++++ > common-trace-format-specification.txt | 1823 ----------------------------- > 2 files changed, 2035 insertions(+), 1823 deletions(-) > create mode 100644 common-trace-format-specification.md > delete mode 100644 common-trace-format-specification.txt > > diff --git a/common-trace-format-specification.md > b/common-trace-format-specification.md > new file mode 100644 > index 0000000..d984176 > --- /dev/null > +++ b/common-trace-format-specification.md > @@ -0,0 +1,2035 @@ > +# Common Trace Format (CTF) Specification (v1.8.2) > + > +**Author**: Mathieu Desnoyers, [EfficiOS Inc.](http://www.efficios.com/) > + > +The goal of the present document is to specify a trace format that suits > +the needs of the embedded, telecom, high-performance and kernel > +communities. It is based on the > +[Common Trace Format Requirements > (v1.4)](http://git.efficios.com/?p=ctf.git;a=blob_plain;f=common-trace-format-reqs.txt;hb=master) > +document. It is designed to allow traces to be natively generated by the > +Linux kernel, Linux user space applications written in C/C++, and > +hardware components. One major element of CTF is the Trace Stream > +Description Language (TSDL) which flexibility enables description of > +various binary trace stream layouts. > + > +The latest version of this document can be found at: > + > + * Git: `git clone git://git.efficios.com/ctf.git` > + * [Gitweb](http://git.efficios.com/?p=ctf.git) > + > +A reference implementation of a library to read and write this trace > +format is being implemented within the > +[Babeltrace](http://www.efficios.com/babeltrace) project, a converter > +between trace formats. The development tree is available at: > + > + * Git: `git clone git://git.efficios.com/babeltrace.git` > + * [Gitweb](http://git.efficios.com/?p=babeltrace.git) > + > +The [CE > Workgroup](http://www.linuxfoundation.org/collaborate/workgroups/celf) > +of the Linux Foundation, [Ericsson](http://www.ericsson.com/), and > +[EfficiOS](http://www.efficios.com/) have sponsored this work. > + > +**Contents**: > + > + 1. Preliminary definitions > + 2. High-level representation of a trace > + 3. Event stream > + 4. Types > + 4.1 Basic types > + 4.1.1 Type inheritance > + 4.1.2 Alignment > + 4.1.3 Byte order > + 4.1.4 Size > + 4.1.5 Integers > + 4.1.6 GNU/C bitfields > + 4.1.7 Floating point > + 4.1.8 Enumerations > + 4.2 Compound types > + 4.2.1 Structures > + 4.2.2 Variants (discriminated/tagged unions) > + 4.2.3 Arrays > + 4.2.4 Sequences > + 4.2.5 Strings > + 5. Event packet header > + 5.1 Event packet header description > + 5.2 Event packet context description > + 6. Event structure > + 6.1 Event header > + 6.1.1 Type 1: few event IDs > + 6.1.2 Type 2: many event IDs > + 6.2 Stream event context and event context > + 6.3 Event payload > + 6.3.1 Padding > + 6.3.2 Alignment > + 7. Trace Stream Description Language (TSDL) > + 7.1 Metadata > + 7.2 Declaration vs definition > + 7.3 TSDL scopes > + 7.3.1 Lexical scope > + 7.3.2 Static and dynamic scopes > + 7.4 TSDL examples > + 8. Clocks > + A. Helper macros > + B. Stream header rationale > + C. TSDL Grammar > + C.1 Lexical grammar > + C.1.1 Lexical elements > + C.1.2 Keywords > + C.1.3 Identifiers > + C.1.4 Universal character names > + C.1.5 Constants > + C.1.6 String literals > + C.1.7 Punctuators > + C.2 Phrase structure grammar > + C.2.2 Declarations: > + C.2.3 CTF-specific declarations > + > + > +## 1. Preliminary definitions > + > + * **Event trace**: an ordered sequence of events. > + * **Event stream**: an ordered sequence of events, containing a > + subset of the trace event types. > + * **Event packet**: a sequence of physically contiguous events within > + an event stream. > + * **Event**: this is the basic entry in a trace. Also known as > + a _trace record_. > + * An **event identifier** (ID) relates to the class (a type) of > + event within an event stream, e.g. event `irq_entry`. > + * An **event** (or event record) relates to a specific instance of > + an event class, e.g. event `irq_entry`, at time _X_, on CPU _Y_. > + * Source architecture: architecture writing the trace. > + * Reader architecture: architecture reading the trace. > + > + > +## 2. High-level representation of a trace > + > +A _trace_ is divided into multiple event streams. Each event stream > +contains a subset of the trace event types. > + > +The final output of the trace, after its generation and optional > +transport over the network, is expected to be either on permanent or > +temporary storage in a virtual file system. Because each event stream > +is appended to while a trace is being recorded, each is associated with > +a distinct set of files for output. Therefore, a stored trace can be > +represented as a directory containing zero, one or more files > +per stream. > + > +Metadata description associated with the trace contains information on > +trace event types expressed in the _Trace Stream Description Language_ > +(TSDL). This language describes: > + > + * Trace version > + * Types available > + * Per-trace event header description > + * Per-stream event header description > + * Per-stream event context description > + * Per-event > + * Event type to stream mapping > + * Event type to name mapping > + * Event type to ID mapping > + * Event context description > + * Event fields description > + > + > +## 3. Event stream > + > +An _event stream_ can be divided into contiguous event packets of > +variable size. An event packet can contain a certain amount of padding > +at the end. The stream header is repeated at the beginning of each > +event packet. The rationale for the event stream design choices is > +explained in [Stream header rationale](#specB). > + > +The event stream header will therefore be referred to as the > +_event packet header_ throughout the rest of this document. > + > + > +## 4. Types > + > +Types are organized as type classes. Each type class belong to either > +of two kind of types: _basic types_ or _compound types_. > + > + > +### 4.1 Basic types > + > +A basic type is a scalar type, as described in this section. It > +includes integers, GNU/C bitfields, enumerations, and floating > +point values. > + > + > +#### 4.1.1 Type inheritance > + > +Type specifications can be inherited to allow deriving types from a > +type class. For example, see the uint32_t named type derived from the > +[_integer_ type](#spec4.1.5) class. Types have a precise binary > +representation in the trace. A type class has methods to read and write > +these types, but must be derived into a type to be usable in an event > +field. > + > + > +#### 4.1.2 Alignment > + > +We define _byte-packed_ types as aligned on the byte size, namely 8-bit. > +We define _bit-packed_ types as following on the next bit, as defined > +by the [Integers](#spec4.1.5) section. > + > +Each basic type must specify its alignment, in bits. Examples of > +possible alignments are: bit-packed (`align = 1`), byte-packed > +(`align = 8`), or word-aligned (e.g. `align = 32` or `align = 64`). > +The choice depends on the architecture preference and compactness vs > +performance trade-offs of the implementation. Architectures providing > +fast unaligned write byte-packed basic types to save space, aligning > +each type on byte boundaries (8-bit). Architectures with slow unaligned > +writes align types on specific alignment values. If no specific > +alignment is declared for a type, it is assumed to be bit-packed for > +integers with size not multiple of 8 bits and for gcc bitfields. All > +other basic types are byte-packed by default. It is however recommended > +to always specify the alignment explicitly. Alignment values must be > +power of two. Compound types are aligned as specified in their > +individual specification. > + > +The base offset used for field alignment is the start of the packet > +containing the field. For instance, a field aligned on 32-bit needs to > +be at an offset multiple of 32-bit from the start of the packet that > +contains it. > + > +TSDL metadata attribute representation of a specific alignment: > + > +~~~ tsdl > +align = /* value in bits */; > +~~~ > + > +#### 4.1.3 Byte order > + > +By default, byte order of a basic type is the byte order described in > +the trace description. It can be overridden by specifying a > +`byte_order` attribute for a basic type. Typical use-case is to specify > +the network byte order (big endian: `be`) to save data captured from > +the network into the trace without conversion. > + > +TSDL metadata representation: > + > +~~~ tsdl > +/* network and be are aliases */ > +byte_order = /* native OR network OR be OR le */; > +~~~ > + > +The `native` keyword selects the byte order described in the trace > +description. The `network` byte order is an alias for big endian. > + > +Even though the trace description section is not per se a type, for > +sake of clarity, it should be noted that `native` and `network` byte > +orders are only allowed within type declaration. The `byte_order` > +specified in the trace description section only accepts `be` or `le` > +values. > + > + > +#### 4.1.4 Size > + > +Type size, in bits, for integers and floats is that returned by > +`sizeof()` in C multiplied by `CHAR_BIT`. We require the size of `char` > +and `unsigned char` types (`CHAR_BIT`) to be fixed to 8 bits for > +cross-endianness compatibility. > + > +TSDL metadata representation: > + > +~~~ tsdl > +size = /* value is in bits */; > +~~~ > + > + > +#### 4.1.5 Integers > + > +Signed integers are represented in two-complement. Integer alignment, > +size, signedness and byte ordering are defined in the TSDL metadata. > +Integers aligned on byte size (8-bit) and with length multiple of byte > +size (8-bit) correspond to the C99 standard integers. In addition, > +integers with alignment and/or size that are _not_ a multiple of the > +byte size are permitted; these correspond to the C99 standard bitfields, > +with the added specification that the CTF integer bitfields have a fixed > +binary representation. Integer size needs to be a positive integer. > +Integers of size 0 are **forbidden**. An MIT-licensed reference > +implementation of the CTF portable bitfields is available > +[here](http://git.efficios.com/?p=babeltrace.git;a=blob;f=include/babeltrace/bitfield.h). > + > +Binary representation of integers: > + > + * On little and big endian: > + * Within a byte, high bits correspond to an integer high bits, and > + low bits correspond to low bits > + * On little endian: > + * Integer across multiple bytes are placed from the less significant > + to the most significant > + * Consecutive integers are placed from lower bits to higher bits > + (even within a byte) > + * On big endian: > + * Integer across multiple bytes are placed from the most significant > + to the less significant > + * Consecutive integers are placed from higher bits to lower bits > + (even within a byte) > + > +This binary representation is derived from the bitfield implementation > +in GCC for little and big endian. However, contrary to what GCC does, > +integers can cross units boundaries (no padding is required). Padding > +can be [explicitly added](#spec4.1.6) to follow the GCC layout if needed. > + > +TSDL metadata representation: > + > +~~~ tsdl > +integer { > + signed = /* true OR false */; /* default: false */ > + byte_order = /* native OR network OR be OR le */; /* default: native */ > + size = /* value in bits */; /* no default */ > + align = /* value in bits */; > + > + /* base used for pretty-printing output; default: decimal */ > + base = /* decimal OR dec OR d OR i OR u OR 10 OR hexadecimal OR hex > + OR x OR X OR p OR 16 OR octal OR oct OR o OR 8 OR binary > + OR b OR 2 */; > + > + /* character encoding */ > + encoding = /* none or UTF8 or ASCII */; /* default: none */ > +} > +~~~ > + > +Example of type inheritance (creation of a `uint32_t` named type): > + > +~~~ tsdl > +typealias integer { > + size = 32; > + signed = false; > + align = 32; > +} := uint32_t; > +~~~ > + > +Definition of a named 5-bit signed bitfield: > + > +~~~ tsdl > +typealias integer { > + size = 5; > + signed = true; > + align = 1; > +} := int5_t; > +~~~ > + > +The character encoding field can be used to specify that the integer > +must be printed as a text character when read. e.g.: > + > +~~~ tsdl > +typealias integer { > + size = 8; > + align = 8; > + signed = false; > + encoding = UTF8; > +} := utf_char; > +~~~ > + > +#### 4.1.6 GNU/C bitfields > + > +The GNU/C bitfields follow closely the integer representation, with a > +particularity on alignment: if a bitfield cannot fit in the current > +unit, the unit is padded and the bitfield starts at the following unit. > +The unit size is defined by the size of the type `unit_type`. > + > +TSDL metadata representation: > + > +~~~ tsdl > +unit_type name:size; > +~~~ > + > +As an example, the following structure declared in C compiled by GCC: > + > +~~~ tsdl > +struct example { > + short a:12; > + short b:5; > +}; > +~~~ > + > +The example structure is aligned on the largest element (short). The > +second bitfield would be aligned on the next unit boundary, because it > +would not fit in the current unit. > + > + > +#### 4.1.7 Floating point > + > +The floating point values byte ordering is defined in the TSDL metadata. > + > +Floating point values follow the IEEE 754-2008 standard interchange > +formats. Description of the floating point values include the exponent > +and mantissa size in bits. Some requirements are imposed on the > +floating point values: > + > +* `FLT_RADIX` must be 2. > +* `mant_dig` is the number of digits represented in the mantissa. It is > + specified by the ISO C99 standard, section 5.2.4, as `FLT_MANT_DIG`, > + `DBL_MANT_DIG` and `LDBL_MANT_DIG` as defined by `<float.h>`. > +* `exp_dig` is the number of digits represented in the exponent. Given > + that `mant_dig` is one bit more than its actual size in bits (leading > + 1 is not needed) and also given that the sign bit always takes one > + bit, `exp_dig` can be specified as: > + * `sizeof(float) * CHAR_BIT - FLT_MANT_DIG` > + * `sizeof(double) * CHAR_BIT - DBL_MANT_DIG` > + * `sizeof(long double) * CHAR_BIT - LDBL_MANT_DIG` > + > +TSDL metadata representation: > + > +~~~ tsdl > +floating_point { > + exp_dig = /* value */; > + mant_dig = /* value */; > + byte_order = /* native OR network OR be OR le */; > + align = /* value */; > +} > +~~~ > + > +Example of type inheritance: > + > +~~~ tsdl > +typealias floating_point { > + exp_dig = 8; /* sizeof(float) * CHAR_BIT - FLT_MANT_DIG */ > + mant_dig = 24; /* FLT_MANT_DIG */ > + byte_order = native; > + align = 32; > +} := float; > +~~~ > + > +TODO: define NaN, +inf, -inf behavior. > + > +Bit-packed, byte-packed or larger alignments can be used for floating > +point values, similarly to integers. > + > + > +#### 4.1.8 Enumerations > + > +Enumerations are a mapping between an integer type and a table of > +strings. The numerical representation of the enumeration follows the > +integer type specified by the metadata. The enumeration mapping table > +is detailed in the enumeration description within the metadata. The > +mapping table maps inclusive value ranges (or single values) to strings. > +Instead of being limited to simple `value -> string` mappings, these > +enumerations map `[ start_value ... end_value ] -> string`, which map > +inclusive ranges of values to strings. An enumeration from the C > +language can be represented in this format by having the same > +`start_value` and `end_value` for each mapping, which is in fact a > +range of size 1. This single-value range is supported without repeating > +the start and end values with the `value = string` declaration. > +Enumerations need to contain at least one entry. > + > +~~~ tsdl > +enum name : integer_type { > + somestring = /* start_value1 */ ... /* end_value1 */, > + "other string" = /* start_value2 */ ... /* end_value2 */, > + yet_another_string, /* will be assigned to end_value2 + 1 */ > + "some other string" = /* value */, > + /* ... */ > +} > +~~~ > + > +If the values are omitted, the enumeration starts at 0 and increment > +of 1 for each entry. An entry with omitted value that follows a range > +entry takes as value the `end_value` of the previous range + 1: > + > +~~~ tsdl > +enum name : unsigned int { > + ZERO, > + ONE, > + TWO, > + TEN = 10, > + ELEVEN, > +} > +~~~ > + > +Overlapping ranges within a single enumeration are implementation > +defined. > + > +A nameless enumeration can be declared as a field type or as part of > +a `typedef`: > + > +~~~ tsdl > +enum : integer_type { > + /* ... */ > +} > +~~~ > + > +Enumerations omitting the container type `: integer_type` use the `int` > +type (for compatibility with C99). The `int` type _must be_ previously > +declared, e.g.: > + > +~~~ tsdl > +typealias integer { size = 32; align = 32; signed = true; } := int; > + > +enum { > + /* ... */ > +} > +~~~ > + > +### 4.2 Compound types > + > +Compound are aggregation of type declarations. Compound types include > +structures, variant, arrays, sequences, and strings. > + > + > +#### 4.2.1 Structures > + > +Structures are aligned on the largest alignment required by basic types > +contained within the structure. (This follows the ISO/C standard for > +structures) > + > +TSDL metadata representation of a named structure: > + > +~~~ tsdl > +struct name { > + field_type field_name; > + field_type field_name; > + /* ... */ > +}; > +~~~ > + > +Example: > + > +~~~ tsdl > +struct example { > + integer { /* nameless type */ > + size = 16; > + signed = true; > + align = 16; > + } first_field_name; > + uint64_t second_field_name; /* named type declared in the metadata */ > +}; > +~~~ > + > +The fields are placed in a sequence next to each other. They each > +possess a field name, which is a unique identifier within the structure. > +The identifier is not allowed to use any [reserved keyword](#specC.1.2). > +Replacing reserved keywords with underscore-prefixed field names is > +**recommended**. Fields starting with an underscore should have their > +leading underscore removed by the CTF trace readers. > + > +A nameless structure can be declared as a field type or as part of > +a `typedef`: > + > +~~~ tsdl > +struct { > + /* ... */ > +} > +~~~ > + > +Alignment for a structure compound type can be forced to a minimum > +value by adding an `align` specifier after the declaration of a > +structure body. This attribute is read as: `align(value)`. The value is > +specified in bits. The structure will be aligned on the maximum value > +between this attribute and the alignment required by the basic types > +contained within the structure. e.g. > + > +~~~ tsdl > +struct { > + /* ... */ > +} align(32) > +~~~ > + > +#### 4.2.2 Variants (discriminated/tagged unions) > + > +A CTF variant is a selection between different types. A CTF variant must > +always be defined within the scope of a structure or within fields > +contained within a structure (defined recursively). A _tag_ enumeration > +field must appear in either the same static scope, prior to the variant > +field (in field declaration order), in an upper static scope, or in an > +upper dynamic scope (see [Static and dynamic scopes](#spec7.3.2)). > +The type selection is indicated by the mapping from the enumeration > +value to the string used as variant type selector. The field to use as > +tag is specified by the `tag_field`, specified between `< >` after the > +`variant` keyword for unnamed variants, and after _variant name_ for > +named variants. It is not required that each enumeration mapping appears > +as variant type tag field. It is also not required that each variant > +type tag appears as enumeration mapping. However, it is required that > +any enumeration mapping encountered within a stream has a matching > +variant type tag field. > + > +The alignment of the variant is the alignment of the type as selected > +by the tag value for the specific instance of the variant. The size of > +the variant is the size as selected by the tag value for the specific > +instance of the variant. > + > +The alignment of the type containing the variant is independent of the > +variant alignment. For instance, if a structure contains two fields, a > +32-bit integer, aligned on 32 bits, and a variant, which contains two > +choices: either a 32-bit field, aligned on 32 bits, or a 64-bit field, > +aligned on 64 bits, the alignment of the outmost structure will be > +32-bit (the alignment of its largest field, disregarding the alignment > +of the variant). The alignment of the variant will depend on the > +selector: if the variant's 32-bit field is selected, its alignment will > +be 32-bit, or 64-bit otherwise. It is important to note that variants > +are specifically tailored for compactness in a stream. Therefore, the > +relative offsets of compound type fields can vary depending on the > +offset at which the compound type starts if it contains a variant > +that itself contains a type with alignment larger than the largest field > +contained within the compound type. This is caused by the fact that the > +compound type may contain the enumeration that select the variant's > +choice, and therefore the alignment to be applied to the compound type > +cannot be determined before encountering the enumeration. > + > +Each variant type selector possess a field name, which is a unique > +identifier within the variant. The identifier is not allowed to use any > +[reserved keyword](#C.1.2). Replacing reserved keywords with > +underscore-prefixed field names is recommended. Fields starting with an > +underscore should have their leading underscore removed by the CTF trace > +readers. > + > +A named variant declaration followed by its definition within a > +structure declaration: > + > +~~~ tsdl > +variant name { > + field_type sel1; > + field_type sel2; > + field_type sel3; > + /* ... */ > +}; > + > +struct { > + enum : integer_type { sel1, sel2, sel3, /* ... */ } tag_field; > + /* ... */ > + variant name <tag_field> v; > +} > +~~~ > + > +An unnamed variant definition within a structure is expressed by the > +following TSDL metadata: > + > +~~~ tsdl > +struct { > + enum : integer_type { sel1, sel2, sel3, /* ... */ } tag_field; > + /* ... */ > + variant <tag_field> { > + field_type sel1; > + field_type sel2; > + field_type sel3; > + /* ... */ > + } v; > +} > +~~~ > + > +Example of a named variant within a sequence that refers to a single > +tag field: > + > +~~~ tsdl > +variant example { > + uint32_t a; > + uint64_t b; > + short c; > +}; > + > +struct { > + enum : uint2_t { a, b, c } choice; > + unsigned int seqlen; > + variant example <choice> v[seqlen]; > +} > +~~~ > + > +Example of an unnamed variant: > + > +~~~ tsdl > +struct { > + enum : uint2_t { a, b, c, d } choice; > + > + /* Unrelated fields can be added between the variant and its tag */ > + int32_t somevalue; > + variant <choice> { > + uint32_t a; > + uint64_t b; > + short c; > + struct { > + unsigned int field1; > + uint64_t field2; > + } d; > + } s; > +} > +~~~ > + > +Example of an unnamed variant within an array: > + > +~~~ tsdl > +struct { > + enum : uint2_t { a, b, c } choice; > + variant <choice> { > + uint32_t a; > + uint64_t b; > + short c; > + } v[10]; > +} > +~~~ > + > +Example of a variant type definition within a structure, where the > +defined type is then declared within an array of structures. This > +variant refers to a tag located in an upper static scope. This example > +clearly shows that a variant type definition referring to the tag `x` > +uses the closest preceding field from the static scope of the type > +definition. > + > +~~~ tsdl > +struct { > + enum : uint2_t { a, b, c, d } x; > + > + /* > + * "x" refers to the preceding "x" enumeration in the > + * static scope of the type definition. > + */ > + typedef variant <x> { > + uint32_t a; > + uint64_t b; > + short c; > + } example_variant; > + > + struct { > + enum : int { x, y, z } x; /* This enumeration is not used by "v". */ > + > + /* "v" uses the "enum : uint2_t { a, b, c, d }" tag. */ > + example_variant v; > + } a[10]; > +} > +~~~ > + > + > +#### 4.2.3 Arrays > + > +Arrays are fixed-length. Their length is declared in the type > +declaration within the metadata. They contain an array of _inner type_ > +elements, which can refer to any type not containing the type of the > +array being declared (no circular dependency). The length is the number > +of elements in an array. > + > +TSDL metadata representation of a named array: > + > +~~~ tsdl > +typedef elem_type name[/* length */]; > +~~~ > + > +A nameless array can be declared as a field type within a > +structure, e.g.: > + > +~~~ tsdl > +uint8_t field_name[10]; > +~~~ > + > +Arrays are always aligned on their element alignment requirement. > + > + > +#### 4.2.4 Sequences > + > +Sequences are dynamically-sized arrays. They refer to a _length_ > +unsigned integer field, which must appear in either the same static > +scope, prior to the sequence field (in field declaration order), > +in an upper static scope, or in an upper dynamic scope > +(see [Static and dynamic scopes](#spec7.3.2)). This length field represents > +the number of elements in the sequence. The sequence per se is an > +array of _inner type_ elements. > + > +TSDL metadata representation for a sequence type definition: > + > +~~~ tsdl > +struct { > + unsigned int length_field; > + typedef elem_type typename[length_field]; > + typename seq_field_name; > +} > +~~~ > + > +A sequence can also be declared as a field type, e.g.: > + > +~~~ tsdl > +struct { > + unsigned int length_field; > + long seq_field_name[length_field]; > +} > +~~~ > + > +Multiple sequences can refer to the same length field, and these length > +fields can be in a different upper dynamic scope, e.g., assuming the > +`stream.event.header` defines: > + > +~~~ tsdl > +stream { > + /* ... */ > + id = 1; > + event.header := struct { > + uint16_t seq_len; > + }; > +}; > + > +event { > + /* ... */ > + stream_id = 1; > + fields := struct { > + long seq_a[stream.event.header.seq_len]; > + char seq_b[stream.event.header.seq_len]; > + }; > +}; > +~~~ > + > +The sequence elements follow the [array](#spec4.2.3) specifications. > + > + > +#### 4.2.5 Strings > + > +Strings are an array of _bytes_ of variable size and are terminated by > +a `'\0'` "NULL" character. Their encoding is described in the TSDL > +metadata. In absence of encoding attribute information, the default > +encoding is UTF-8. > + > +TSDL metadata representation of a named string type: > + > +~~~ tsdl > +typealias string { > + encoding = /* UTF8 OR ASCII */; > +} := name; > +~~~ > + > +A nameless string type can be declared as a field type: > + > +~~~ tsdl > +string field_name; /* use default UTF8 encoding */ > +~~~ > + > +Strings are always aligned on byte size. > + > + > +## 5. Event packet header > + > +The event packet header consists of two parts: the > +_event packet header_ is the same for all streams of a trace. The > +second part, the _event packet context_, is described on a per-stream > +basis. Both are described in the TSDL metadata. > + > +Event packet header (all fields are optional, specified by > +TSDL metadata): > + > + * **Magic number** (CTF magic number: 0xC1FC1FC1) specifies that this is > + a CTF packet. This magic number is optional, but when present, it > + should come at the very beginning of the packet. > + * **Trace UUID**, used to ensure the event packet match the metadata used. > + Note: we cannot use a metadata checksum in every cases instead of a > + UUID because metadata can be appended to while tracing is active. > + This field is optional. > + * **Stream ID**, used as reference to stream description in metadata. > + This field is optional if there is only one stream description in > + the metadata, but becomes required if there are more than one > + stream in the TSDL metadata description. > + > +Event packet context (all fields are optional, specified by > +TSDL metadata): > + > + * Event packet **content size** (in bits). > + * Event packet **size** (in bits, includes padding). > + * Event packet content checksum. Checksum excludes the event packet > + header. > + * Per-stream event **packet sequence count** (to deal with UDP packet > + loss). The number of significant sequence counter bits should also > + be present, so wrap-arounds are dealt with correctly. > + * Time-stamp at the beginning and timestamp at the end of the event > + packet. Both timestamps are written in the packet header, but > + sampled respectively while (or before) writing the first event and > + while (or after) writing the last event in the packet. The inclusive > + range between these timestamps should include all event timestamps > + assigned to events contained within the packet. The timestamp at the > + beginning of an event packet is guaranteed to be below or equal the > + timestamp at the end of that event packet. The timestamp at the end > + of an event packet is guaranteed to be below or equal the > + timestamps at the end of any following packet within the same stream. > + See [Clocks](#spec8) for more detail. > + * **Events discarded count**. Snapshot of a per-stream > + free-running counter, counting the number of events discarded that > + were supposed to be written in the stream after the last event in > + the event packet. Note: producer-consumer buffer full condition can > + fill the current event packet with padding so we know exactly where > + events have been discarded. However, if the buffer full condition > + chooses not to fill the current event packet with padding, all we > + know about the timestamp range in which the events have been > + discarded is that it is somewhere between the beginning and the end > + of the packet. > + * Lossless **compression scheme** used for the event packet content. > + Applied directly to raw data. New types of compression can be added > + in following versions of the format. > + * 0: no compression scheme > + * 1: bzip2 > + * 2: gzip > + * 3: xz > + * **Cypher** used for the event packet content. Applied after > + compression. > + * 0: no encryption > + * 1: AES > + * **Checksum scheme** used for the event packet content. Applied after > + encryption. > + * 0: no checksum > + * 1: md5 > + * 2: sha1 > + * 3: crc32 > + > + > +### 5.1 Event packet header description > + > +The event packet header layout is indicated by the > +`trace.packet.header` field. Here is a recommended structure type for > +the packet header with the fields typically expected (although these > +fields are each optional): > + > +~~~ tsdl > +struct event_packet_header { > + uint32_t magic; > + uint8_t uuid[16]; > + uint32_t stream_id; > +}; > + > +trace { > + /* ... */ > + packet.header := struct event_packet_header; > +}; > +~~~ > + > +If the magic number is not present, tools such as `file` will have no > +mean to discover the file type. > + > +If the uuid is not present, no validation that the metadata actually > +corresponds to the stream is performed. > + > +If the stream_id packet header field is missing, the trace can only > +contain a single stream. Its `id` field can be left out, and its events > +don't need to declare a `stream_id` field. > + > + > +### 5.2 Event packet context description > + > +Event packet context example. These are declared within the stream > +declaration in the metadata. All these fields are optional. If the > +packet size field is missing, the whole stream only contains a single > +packet. If the content size field is missing, the packet is filled > +(no padding). The content and packet sizes include all headers. > + > +An example event packet context type: > + > +~~~ tsdl > +struct event_packet_context { > + uint64_t timestamp_begin; > + uint64_t timestamp_end; > + uint32_t checksum; > + uint32_t stream_packet_count; > + uint32_t events_discarded; > + uint32_t cpu_id; > + uint64_t content_size; > + uint64_t packet_size; > + uint8_t compression_scheme; > + uint8_t encryption_scheme; > + uint8_t checksum_scheme; > +}; > +~~~ > + > + > +## 6. Event Structure > + > +The overall structure of an event is: > + > + 1. Event header (as specified by the stream metadata) > + 2. Stream event context (as specified by the stream metadata) > + 3. Event context (as specified by the event metadata) > + 4. Event payload (as specified by the event metadata) > + > +This structure defines an implicit dynamic scoping, where variants > +located in inner structures (those with a higher number in the listing > +above) can refer to the fields of outer structures (with lower number > +in the listing above). See [TSDL scopes](#spec7.3) for more detail. > + > +The total length of an event is defined as the difference between the > +end of its event payload and the end of the previous event's event > +payload. Therefore, it includes the event header alignment padding, and > +all its fields and their respective alignment padding. Events of length > +0 are forbidden. > + > + > +### 6.1 Event header > + > +Event headers can be described within the metadata. We hereby propose, > +as an example, two types of events headers. Type 1 accommodates streams > +with less than 31 event IDs. Type 2 accommodates streams with 31 or > +more event IDs. > + > +One major factor can vary between streams: the number of event IDs > +assigned to a stream. Luckily, this information tends to stay > +relatively constant (modulo event registration while trace is being > +recorded), so we can specify different representations for streams > +containing few event IDs and streams containing many event IDs, so we > +end up representing the event ID and timestamp as densely as possible > +in each case. > + > +The header is extended in the rare occasions where the information > +cannot be represented in the ranges available in the standard event > +header. They are also used in the rare occasions where the data > +required for a field could not be collected: the flag corresponding to > +the missing field within the `missing_fields` array is then set to 1. > + > +Types `uintX_t` represent an `X`-bit unsigned integer, as declared with > +either: > + > +~~~ tsdl > +typealias integer { > + size = /* X */; > + align = /* X */; > + signed = false; > +} := uintX_t; > +~~~ > + > +or > + > +~~~ tsdl > +typealias integer { > + size = /* X */; > + align = 1; > + signed = false; > +} := uintX_t; > +~~~ > + > +For more information about timestamp fields, see [Clocks](#spec8). > + > + > +#### 6.1.1 Type 1: few event IDs > + > + * Aligned on 32-bit (or 8-bit if byte-packed, depending on the > + architecture preference) > + * Native architecture byte ordering > + * For `compact` selection, fixed size of 32 bits > + * For "extended" selection, size depends on the architecture and > + variant alignment > + > +~~~ tsdl > +struct event_header_1 { > + /* > + * id: range: 0 - 30. > + * id 31 is reserved to indicate an extended header. > + */ > + enum : uint5_t { compact = 0 ... 30, extended = 31 } id; > + variant <id> { > + struct { > + uint27_t timestamp; > + } compact; > + struct { > + uint32_t id; /* 32-bit event IDs */ > + uint64_t timestamp; /* 64-bit timestamps */ > + } extended; > + } v; > +} align(32); /* or align(8) */ > +~~~ > + > + > +#### 6.1.2 Type 2: many event IDs > + > + * Aligned on 16-bit (or 8-bit if byte-packed, depending on the > + architecture preference) > + * Native architecture byte ordering > + * For `compact` selection, size depends on the architecture and > + variant alignment > + * For `extended` selection, size depends on the architecture and > + variant alignment > + > +~~~ tsdl > +struct event_header_2 { > + /* > + * id: range: 0 - 65534. > + * id 65535 is reserved to indicate an extended header. > + */ > + enum : uint16_t { compact = 0 ... 65534, extended = 65535 } id; > + variant <id> { > + struct { > + uint32_t timestamp; > + } compact; > + struct { > + uint32_t id; /* 32-bit event IDs */ > + uint64_t timestamp; /* 64-bit timestamps */ > + } extended; > + } v; > +} align(16); /* or align(8) */ > +~~~ > + > + > +### 6.2 Stream event context and event context > + > +The event context contains information relative to the current event. > +The choice and meaning of this information is specified by the TSDL > +stream and event metadata descriptions. The stream context is applied > +to all events within the stream. The stream context structure follows > +the event header. The event context is applied to specific events. Its > +structure follows the stream context structure. > + > +An example of stream-level event context is to save the event payload > +size with each event, or to save the current PID with each event. > +These are declared within the stream declaration within the metadata: > + > +~~~ tsdl > +stream { > + /* ... */ > + event.context := struct { > + uint pid; > + uint16_t payload_size; > + }; > +}; > +~~~ > + > +An example of event-specific event context is to declare a bitmap of > +missing fields, only appended after the stream event context if the > +extended event header is selected. `NR_FIELDS` is the number of fields > +within the event (a numeric value). > + > +~~~ tsdl > +event { > + context := struct { > + variant <id> { > + struct { } compact; > + struct { > + /* missing event fields bitmap */ > + uint1_t missing_fields[NR_FIELDS]; > + } extended; > + } v; > + }; > + /* ... */ > +} > +~~~ > + > + > +### 6.3 Event payload > + > +An event payload contains fields specific to a given event type. The > +fields belonging to an event type are described in the event-specific > +metadata within a structure type. > + > + > +#### 6.3.1 Padding > + > +No padding at the end of the event payload. This differs from the ISO/C > +standard for structures, but follows the CTF standard for structures. > +In a trace, even though it makes sense to align the beginning of a > +structure, it really makes no sense to add padding at the end of the > +structure, because structures are usually not followed by a structure > +of the same type. > + > +This trick can be done by adding a zero-length `end` field at the end > +of the C structures, and by using the offset of this field rather than > +using `sizeof()` when calculating the size of a structure > +(see [Helper macros](#specA)). > + > + > +#### 6.3.2 Alignment > + > +The event payload is aligned on the largest alignment required by types > +contained within the payload. This follows the ISO/C standard for > +structures. > + > + > +## 7. Trace Stream Description Language (TSDL) > + > +The Trace Stream Description Language (TSDL) allows expression of the > +binary trace streams layout in a C99-like Domain Specific Language > +(DSL). > + > + > +### 7.1 Meta-data > + > +The trace stream layout description is located in the trace metadata. > +The metadata is itself located in a stream identified by its name: > +`metadata`. > + > +The metadata description can be expressed in two different formats: > +text-only and packet-based. The text-only description facilitates > +generation of metadata and provides a convenient way to enter the > +metadata information by hand. The packet-based metadata provides the > +CTF stream packet facilities (checksumming, compression, encryption, > +network-readiness) for metadata stream generated and transported by a > +tracer. > + > +The text-only metadata file is a plain-text TSDL description. This file > +must begin with the following characters to identify the file as a CTF > +TSDL text-based metadata file (without the double-quotes): > + > +~~~ text > +"/* CTF" > +~~~ > + > +It must be followed by a space, and the version of the specification > +followed by the CTF trace, e.g.: > + > +~~~ text > +" 1.8" > +~~~ > + > +These characters allow automated discovery of file type and CTF > +specification version. They are interpreted as a the beginning of a > +comment by the TSDL metadata parser. The comment can be continued to > +contain extra commented characters before it is closed. > + > +The packet-based metadata is made of _metadata packets_, which each > +start with a metadata packet header. The packet-based metadata > +description is detected by reading the magic number 0x75D11D57 at the > +beginning of the file. This magic number is also used to detect the > +endianness of the architecture by trying to read the CTF magic number > +and its counterpart in reversed endianness. The events within the > +metadata stream have no event header nor event context. Each event only > +contains a special _sequence_ payload, which is a sequence of bits which > +length is implicitly calculated by using the > +`trace.packet.header.content_size` field, minus the packet header size. > +The formatting of this sequence of bits is a plain-text representation > +of the TSDL description. Each metadata packet start with a special > +packet header, specific to the metadata stream, which contains, > +exactly: > + > +~~~ tsdl > +struct metadata_packet_header { > + uint32_t magic; /* 0x75D11D57 */ > + uint8_t uuid[16]; /* Unique Universal Identifier */ > + uint32_t checksum; /* 0 if unused */ > + uint32_t content_size; /* in bits */ > + uint32_t packet_size; /* in bits */ > + uint8_t compression_scheme; /* 0 if unused */ > + uint8_t encryption_scheme; /* 0 if unused */ > + uint8_t checksum_scheme; /* 0 if unused */ > + uint8_t major; /* CTF spec version major number */ > + uint8_t minor; /* CTF spec version minor number */ > +}; > +~~~ > + > +The packet-based metadata can be converted to a text-only metadata by > +concatenating all the strings it contains. > + > +In the textual representation of the metadata, the text contained > +within `/*` and `*/`, as well as within `//` and end of line, are > +treated as comments. Boolean values can be represented as `true`, > +`TRUE`, or `1` for true, and `false`, `FALSE`, or `0` for false. Within > +the string-based metadata description, the trace UUID is represented as > +a string of hexadecimal digits and dashes `-`. In the event packet > +header, the trace UUID is represented as an array of bytes. > + > + > +### 7.2 Declaration vs definition > + > +A declaration associates a layout to a type, without specifying where > +this type is located in the event [structure hierarchy](#spec6). > +This therefore includes `typedef`, `typealias`, as well as all type > +specifiers. In certain circumstances (`typedef`, structure field and > +variant field), a declaration is followed by a declarator, which specify > +the newly defined type name (for `typedef`), or the field name (for > +declarations located within structure and variants). Array and sequence, > +declared with square brackets (`[` `]`), are part of the declarator, > +similarly to C99. The enumeration base type is specified by > +`: enum_base`, which is part of the type specifier. The variant tag > +name, specified between `<` `>`, is also part of the type specifier. > + > +A definition associates a type to a location in the event > +[structure hierarchy](#spec6). This association is denoted by `:=`, > +as shown in [TSDL scopes](#spec7.3). > + > + > +### 7.3 TSDL scopes > + > +TSDL uses three different types of scoping: a lexical scope is used for > +declarations and type definitions, and static and dynamic scopes are > +used for variants references to tag fields (with relative and absolute > +path lookups) and for sequence references to length fields. > + > + > +#### 7.3.1 Lexical Scope > + > +Each of `trace`, `env`, `stream`, `event`, `struct` and `variant` have > +their own nestable declaration scope, within which types can be declared > +using `typedef` and `typealias`. A root declaration scope also contains > +all declarations located outside of any of the aforementioned > +declarations. An inner declaration scope can refer to type declared > +within its container lexical scope prior to the inner declaration scope. > +Redefinition of a typedef or typealias is not valid, although hiding an > +upper scope typedef or typealias is allowed within a sub-scope. > + > + > +#### 7.3.2 Static and dynamic scopes > + > +A local static scope consists in the scope generated by the declaration > +of fields within a compound type. A static scope is a local static scope > +augmented with the nested sub-static-scopes it contains. > + > +A dynamic scope consists in the static scope augmented with the > +implicit [event structure](#spec6) definition hierarchy. > + > +Multiple declarations of the same field name within a local static scope > +is not valid. It is however valid to re-use the same field name in > +different local scopes. > + > +Nested static and dynamic scopes form lookup paths. These are used for > +variant tag and sequence length references. They are used at the variant > +and sequence definition site to look up the location of the tag field > +associated with a variant, and to lookup up the location of the length > +field associated with a sequence. > + > +Variants and sequences can refer to a tag field either using a relative > +path or an absolute path. The relative path is relative to the scope in > +which the variant or sequence performing the lookup is located. > +Relative paths are only allowed to lookup within the same static scope, > +which includes its nested static scopes. Lookups targeting parent static > +scopes need to be performed with an absolute path. > + > +Absolute path lookups use the full path including the dynamic scope > +followed by a `.` and then the static scope. Therefore, variants (or > +sequences) in lower levels in the dynamic scope (e.g., event context) > +can refer to a tag (or length) field located in upper levels > +(e.g., in the event header) by specifying, in this case, the associated > +tag with `<stream.event.header.field_name>`. This allows, for instance, > +the event context to define a variant referring to the `id` field of > +the event header as selector. > + > +The dynamic scope prefixes are thus: > + > + * Trace environment: `<env. >` > + * Trace packet header: `<trace.packet.header. >` > + * Stream packet context: `<stream.packet.context. >` > + * Event header: `<stream.event.header. >` > + * Stream event context: `<stream.event.context. >` > + * Event context: `<event.context. >` > + * Event payload: `<event.fields. >` > + > +The target dynamic scope must be specified explicitly when referring to > +a field outside of the static scope (absolute scope reference). No > +conflict can occur between relative and dynamic paths, because the > +keywords `trace`, `stream`, and `event` are reserved, and thus not > +permitted as field names. It is recommended that field names clashing > +with CTF and C99 reserved keywords use an underscore prefix to > +eliminate the risk of generating a description containing an invalid > +field name. Consequently, fields starting with an underscore should have > +their leading underscore removed by the CTF trace readers. > + > +The information available in the dynamic scopes can be thought of as the > +current tracing context. At trace production, information about the > +current context is saved into the specified scope field levels. At trace > +consumption, for each event, the current trace context is therefore > +readable by accessing the upper dynamic scopes. > + > + > +### 7.4 TSDL examples > + > +The grammar representing the TSDL metadata is presented in > +[TSDL grammar](#specC). This section presents a rather lighter reading that > +consists in examples of TSDL metadata, with template values. > + > +The stream ID can be left out if there is only one stream in the > +trace. The event `id` field can be left out if there is only one event > +in a stream. > + > +~~~ tsdl > +trace { > + major = /* value */; /* CTF spec version major number */ > + minor = /* value */; /* CTF spec version minor number */ > + uuid = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; /* Trace UUID */ > + byte_order = /* be OR le */; /* Endianness (required) */ > + packet.header := struct { > + uint32_t magic; > + uint8_t uuid[16]; > + uint32_t stream_id; > + }; > +}; > + > +/* > + * The "env" (environment) scope contains assignment expressions. The > + * field names and content are implementation-defined. > + */ > +env { > + pid = /* value */; /* example */ > + proc_name = "name"; /* example */ > + /* ... */ > +}; > + > +stream { > + id = /* stream_id */; > + /* Type 1 - Few event IDs; Type 2 - Many event IDs. See section 6.1. */ > + event.header := /* event_header_1 OR event_header_2 */; > + event.context := struct { > + /* ... */ > + }; > + packet.context := struct { > + /* ... */ > + }; > +}; > + > +event { > + name = "event_name"; > + id = /* value */; /* Numeric identifier within the stream */ > + stream_id = /* stream_id */; > + loglevel = /* value */; > + model.emf.uri = "string"; > + context := struct { > + /* ... */ > + }; > + fields := struct { > + /* ... */ > + }; > +}; > + > +callsite { > + name = "event_name"; > + func = "func_name"; > + file = "myfile.c"; > + line = 39; > + ip = 0x40096c; > +}; > +~~~ > + > +More detail on [types](#spec4): > + > +~~~ tsdl > +/* > + * Named types: > + * > + * Type declarations behave similarly to the C standard. > + */ > + > +typedef aliased_type_specifiers new_type_declarators; > + > +/* e.g.: typedef struct example new_type_name[10]; */ > + > +/* > + * typealias > + * > + * The "typealias" declaration can be used to give a name (including > + * pointer declarator specifier) to a type. It should also be used to > + * map basic C types (float, int, unsigned long, ...) to a CTF type. > + * Typealias is a superset of "typedef": it also allows assignment of a > + * simple variable identifier to a type. > + */ > + > +typealias type_class { > + /* ... */ > +} := type_specifiers type_declarator; > + > +/* > + * e.g.: > + * typealias integer { > + * size = 32; > + * align = 32; > + * signed = false; > + * } := struct page *; > + * > + * typealias integer { > + * size = 32; > + * align = 32; > + * signed = true; > + * } := int; > + */ > + > +struct name { > + /* ... */ > +}; > + > +variant name { > + /* ... */ > +}; > + > +enum name : integer_type { > + /* ... */ > +}; > +~~~ > + > +Unnamed types, contained within compound type fields, `typedef` or > +`typealias`: > + > +~~~ tsdl > +struct { > + /* ... */ > +} > +~~~ > + > +~~~ tsdl > +struct { > + /* ... */ > +} align(value) > +~~~ > + > +~~~ tsdl > +variant { > + /* ... */ > +} > +~~~ > + > +~~~ tsdl > +enum : integer_type { > + /* ... */ > +} > +~~~ > + > +~~~ tsdl > +typedef type new_type[length]; > + > +struct { > + type field_name[length]; > +} > +~~~ > + > +~~~ tsdl > +typedef type new_type[length_type]; > + > +struct { > + type field_name[length_type]; > +} > +~~~ > + > +~~~ tsdl > +integer { > + /* ... */ > +} > +~~~ > + > +~~~ tsdl > +floating_point { > + /* ... */ > +} > +~~~ > + > +~~~ tsdl > +struct { > + integer_type field_name:size; /* GNU/C bitfield */ > +} > +~~~ > + > +~~~ tsdl > +struct { > + string field_name; > +} > +~~~ > + > + > +## 8. Clocks > + > +Clock metadata allows to describe the clock topology of the system, as > +well as to detail each clock parameter. In absence of clock description, > +it is assumed that all fields named `timestamp` use the same clock > +source, which increments once per nanosecond. > + > +Describing a clock and how it is used by streams is threefold: first, > +the clock and clock topology should be described in a `clock` > +description block, e.g.: > + > +~~~ tsdl > +clock { > + name = cycle_counter_sync; > + uuid = "62189bee-96dc-11e0-91a8-cfa3d89f3923"; > + description = "Cycle counter synchronized across CPUs"; > + freq = 1000000000; /* frequency, in Hz */ > + /* precision in seconds is: 1000 * (1/freq) */ > + precision = 1000; > + /* > + * clock value offset from Epoch is: > + * offset_s + (offset * (1/freq)) > + */ > + offset_s = 1326476837; > + offset = 897235420; > + absolute = FALSE; > +}; > +~~~ > + > +The mandatory `name` field specifies the name of the clock identifier, > +which can later be used as a reference. The optional field `uuid` is > +the unique identifier of the clock. It can be used to correlate > +different traces that use the same clock. An optional textual > +description string can be added with the `description` field. The > +`freq` field is the initial frequency of the clock, in Hz. If the > +`freq` field is not present, the frequency is assumed to be 1000000000 > +(providing clock increment of 1 ns). The optional `precision` field > +details the uncertainty on the clock measurements, in (1/freq) units. > +The `offset_s` and `offset` fields indicate the offset from > +POSIX.1 Epoch, 1970-01-01 00:00:00 +0000 (UTC), to the zero of value > +of the clock. The `offset_s` field is in seconds. The `offset` field is > +in (1/freq) units. If any of the `offset_s` or `offset` field is not > +present, it is assigned the 0 value. The field `absolute` is `TRUE` if > +the clock is a global reference across different clock UUID > +(e.g. NTP time). Otherwise, `absolute` is `FALSE`, and the clock can > +be considered as synchronized only with other clocks that have the same > +UUID. > + > +Secondly, a reference to this clock should be added within an integer > +type: > + > +~~~ tsdl > +typealias integer { > + size = 64; align = 1; signed = false; > + map = clock.cycle_counter_sync.value; > +} := uint64_ccnt_t; > +~~~ > + > +Thirdly, stream declarations can reference the clock they use as a > +timestamp source: > + > +~~~ tsdl > +struct packet_context { > + uint64_ccnt_t ccnt_begin; > + uint64_ccnt_t ccnt_end; > + /* ... */ > +}; > + > +stream { > + /* ... */ > + event.header := struct { > + uint64_ccnt_t timestamp; > + /* ... */ > + }; > + packet.context := struct packet_context; > +}; > +~~~ > + > +For a N-bit integer type referring to a clock, if the integer overflows > +compared to the N low order bits of the clock prior value found in the > +same stream, then it is assumed that one, and only one, overflow > +occurred. It is therefore important that events encoding time on a small > +number of bits happen frequently enough to detect when more than one > +N-bit overflow occurs. > + > +In a packet context, clock field names ending with `_begin` and `_end` > +have a special meaning: this refers to the timestamps at, respectively, > +the beginning and the end of each packet. > + > + > +## A. Helper macros > + > +The two following macros keep track of the size of a GNU/C structure > +without padding at the end by placing HEADER_END as the last field. > +A one byte end field is used for C90 compatibility (C99 flexible arrays > +could be used here). Note that this does not affect the effective > +structure size, which should always be calculated with the > +`header_sizeof()` helper. > + > +~~~ c > +#define HEADER_END char end_field > +#define header_sizeof(type) offsetof(typeof(type), end_field) > +~~~ > + > +## B. Stream header rationale > + > +An event stream is divided in contiguous event packets of variable > +size. These subdivisions allow the trace analyzer to perform a fast > +binary search by time within the stream (typically requiring to index > +only the event packet headers) without reading the whole stream. These > +subdivisions have a variable size to eliminate the need to transfer the > +event packet padding when partially filled event packets must be sent > +when streaming a trace for live viewing/analysis. An event packet can > +contain a certain amount of padding at the end. Dividing streams into > +event packets is also useful for network streaming over UDP and flight > +recorder mode tracing (a whole event packet can be swapped out of the > +buffer atomically for reading). > + > +The stream header is repeated at the beginning of each event packet to > +allow flexibility in terms of: > + > + * streaming support > + * allowing arbitrary buffers to be discarded without making the trace > + unreadable > + * allow UDP packet loss handling by either dealing with missing event > packet > + or asking for re-transmission > + * transparently support flight recorder mode > + * transparently support crash dump > + > + > +## C. TSDL Grammar > + > +~~~ c > +/* > + * Common Trace Format (CTF) Trace Stream Description Language (TSDL) > Grammar. > + * > + * Inspired from the C99 grammar: > + * http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf (Annex A) > + * and c++1x grammar (draft) > + * http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3291.pdf (Annex > A) > + * > + * Specialized for CTF needs by including only constant and declarations > from > + * C99 (excluding function declarations), and by adding support for > variants, > + * sequences and CTF-specific specifiers. Enumeration container types > + * semantic is inspired from c++1x enum-base. > + */ > +~~~ > + > + > +### C.1 Lexical grammar > + > + > +#### C.1.1 Lexical elements > + > +~~~ text > +token: > + keyword > + identifier > + constant > + string-literal > + punctuator > +~~~ > + > +#### C.1.2 Keywords > + > +~~~ text > +keyword: is one of > + > +align > +callsite > +const > +char > +clock > +double > +enum > +env > +event > +floating_point > +float > +integer > +int > +long > +short > +signed > +stream > +string > +struct > +trace > +typealias > +typedef > +unsigned > +variant > +void > +_Bool > +_Complex > +_Imaginary > +~~~ > + > + > +#### C.1.3 Identifiers > + > +~~~ text > +identifier: > + identifier-nondigit > + identifier identifier-nondigit > + identifier digit > + > +identifier-nondigit: > + nondigit > + universal-character-name > + any other implementation-defined characters > + > +nondigit: > + _ > + [a-zA-Z] /* regular expression */ > + > +digit: > + [0-9] /* regular expression */ > +~~~ > + > + > +#### C.1.4 Universal character names > + > +~~~ text > +universal-character-name: > + \u hex-quad > + \U hex-quad hex-quad > + > +hex-quad: > + hexadecimal-digit hexadecimal-digit hexadecimal-digit hexadecimal-digit > +~~~ > + > + > +##### C.1.5 Constants > + > +~~~ text > +constant: > + integer-constant > + enumeration-constant > + character-constant > + > +integer-constant: > + decimal-constant integer-suffix-opt > + octal-constant integer-suffix-opt > + hexadecimal-constant integer-suffix-opt > + > +decimal-constant: > + nonzero-digit > + decimal-constant digit > + > +octal-constant: > + 0 > + octal-constant octal-digit > + > +hexadecimal-constant: > + hexadecimal-prefix hexadecimal-digit > + hexadecimal-constant hexadecimal-digit > + > +hexadecimal-prefix: > + 0x > + 0X > + > +nonzero-digit: > + [1-9] > + > +integer-suffix: > + unsigned-suffix long-suffix-opt > + unsigned-suffix long-long-suffix > + long-suffix unsigned-suffix-opt > + long-long-suffix unsigned-suffix-opt > + > +unsigned-suffix: > + u > + U > + > +long-suffix: > + l > + L > + > +long-long-suffix: > + ll > + LL > + > +enumeration-constant: > + identifier > + string-literal > + > +character-constant: > + ' c-char-sequence ' > + L' c-char-sequence ' > + > +c-char-sequence: > + c-char > + c-char-sequence c-char > + > +c-char: > + any member of source charset except single-quote ('), backslash > + (\), or new-line character. > + escape-sequence > + > +escape-sequence: > + simple-escape-sequence > + octal-escape-sequence > + hexadecimal-escape-sequence > + universal-character-name > + > +simple-escape-sequence: one of > + \' \" \? \\ \a \b \f \n \r \t \v > + > +octal-escape-sequence: > + \ octal-digit > + \ octal-digit octal-digit > + \ octal-digit octal-digit octal-digit > + > +hexadecimal-escape-sequence: > + \x hexadecimal-digit > + hexadecimal-escape-sequence hexadecimal-digit > +~~~ > + > + > +#### C.1.6 String literals > + > +~~~ text > +string-literal: > + " s-char-sequence-opt " > + L" s-char-sequence-opt " > + > +s-char-sequence: > + s-char > + s-char-sequence s-char > + > +s-char: > + any member of source charset except double-quote ("), backslash > + (\), or new-line character. > + escape-sequence > +~~~ > + > + > +#### C.1.7 Punctuators > + > +~~~ text > +punctuator: one of > + [ ] ( ) { } . -> * + - < > : ; ... = , > +~~~ > + > + > +### C.2 Phrase structure grammar > + > +~~~ text > +primary-expression: > + identifier > + constant > + string-literal > + ( unary-expression ) > + > +postfix-expression: > + primary-expression > + postfix-expression [ unary-expression ] > + postfix-expression . identifier > + postfix-expressoin -> identifier > + > +unary-expression: > + postfix-expression > + unary-operator postfix-expression > + > +unary-operator: one of > + + - > + > +assignment-operator: > + = > + > +type-assignment-operator: > + := > + > +constant-expression-range: > + unary-expression ... unary-expression > +~~~ > + > + > +#### C.2.2 Declarations: > + > +~~~ text > +declaration: > + declaration-specifiers declarator-list-opt ; > + ctf-specifier ; > + > +declaration-specifiers: > + storage-class-specifier declaration-specifiers-opt > + type-specifier declaration-specifiers-opt > + type-qualifier declaration-specifiers-opt > + > +declarator-list: > + declarator > + declarator-list , declarator > + > +abstract-declarator-list: > + abstract-declarator > + abstract-declarator-list , abstract-declarator > + > +storage-class-specifier: > + typedef > + > +type-specifier: > + void > + char > + short > + int > + long > + float > + double > + signed > + unsigned > + _Bool > + _Complex > + _Imaginary > + struct-specifier > + variant-specifier > + enum-specifier > + typedef-name > + ctf-type-specifier > + > +align-attribute: > + align ( unary-expression ) > + > +struct-specifier: > + struct identifier-opt { struct-or-variant-declaration-list-opt } > align-attribute-opt > + struct identifier align-attribute-opt > + > +struct-or-variant-declaration-list: > + struct-or-variant-declaration > + struct-or-variant-declaration-list struct-or-variant-declaration > + > +struct-or-variant-declaration: > + specifier-qualifier-list struct-or-variant-declarator-list ; > + declaration-specifiers-opt storage-class-specifier > declaration-specifiers-opt declarator-list ; > + typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declaration-specifiers abstract-declarator-list ; > + typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declarator-list ; > + > +specifier-qualifier-list: > + type-specifier specifier-qualifier-list-opt > + type-qualifier specifier-qualifier-list-opt > + > +struct-or-variant-declarator-list: > + struct-or-variant-declarator > + struct-or-variant-declarator-list , struct-or-variant-declarator > + > +struct-or-variant-declarator: > + declarator > + declarator-opt : unary-expression > + > +variant-specifier: > + variant identifier-opt variant-tag-opt { > struct-or-variant-declaration-list } > + variant identifier variant-tag > + > +variant-tag: > + < unary-expression > > + > +enum-specifier: > + enum identifier-opt { enumerator-list } > + enum identifier-opt { enumerator-list , } > + enum identifier > + enum identifier-opt : declaration-specifiers { enumerator-list } > + enum identifier-opt : declaration-specifiers { enumerator-list , } > + > +enumerator-list: > + enumerator > + enumerator-list , enumerator > + > +enumerator: > + enumeration-constant > + enumeration-constant assignment-operator unary-expression > + enumeration-constant assignment-operator constant-expression-range > + > +type-qualifier: > + const > + > +declarator: > + pointer-opt direct-declarator > + > +direct-declarator: > + identifier > + ( declarator ) > + direct-declarator [ unary-expression ] > + > +abstract-declarator: > + pointer-opt direct-abstract-declarator > + > +direct-abstract-declarator: > + identifier-opt > + ( abstract-declarator ) > + direct-abstract-declarator [ unary-expression ] > + direct-abstract-declarator [ ] > + > +pointer: > + * type-qualifier-list-opt > + * type-qualifier-list-opt pointer > + > +type-qualifier-list: > + type-qualifier > + type-qualifier-list type-qualifier > + > +typedef-name: > + identifier > +~~~ > + > + > +#### C.2.3 CTF-specific declarations > + > +~~~ text > +ctf-specifier: > + clock { ctf-assignment-expression-list-opt } > + event { ctf-assignment-expression-list-opt } > + stream { ctf-assignment-expression-list-opt } > + env { ctf-assignment-expression-list-opt } > + trace { ctf-assignment-expression-list-opt } > + callsite { ctf-assignment-expression-list-opt } > + typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declaration-specifiers abstract-declarator-list > + typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declarator-list > + > +ctf-type-specifier: > + floating_point { ctf-assignment-expression-list-opt } > + integer { ctf-assignment-expression-list-opt } > + string { ctf-assignment-expression-list-opt } > + string > + > +ctf-assignment-expression-list: > + ctf-assignment-expression ; > + ctf-assignment-expression-list ctf-assignment-expression ; > + > +ctf-assignment-expression: > + unary-expression assignment-operator unary-expression > + unary-expression type-assignment-operator type-specifier > + declaration-specifiers-opt storage-class-specifier > declaration-specifiers-opt declarator-list > + typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declaration-specifiers abstract-declarator-list > + typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declarator-list > +~~~ > diff --git a/common-trace-format-specification.txt > b/common-trace-format-specification.txt > deleted file mode 100644 > index 5568a27..0000000 > --- a/common-trace-format-specification.txt > +++ /dev/null > @@ -1,1823 +0,0 @@ > -Common Trace Format (CTF) Specification (v1.8.2) > - > -Mathieu Desnoyers, EfficiOS Inc. > - > -The goal of the present document is to specify a trace format that suits the > -needs of the embedded, telecom, high-performance and kernel communities. It > is > -based on the Common Trace Format Requirements (v1.4) document. It is > designed to > -allow traces to be natively generated by the Linux kernel, Linux user-space > -applications written in C/C++, and hardware components. One major element of > -CTF is the Trace Stream Description Language (TSDL) which flexibility > -enables description of various binary trace stream layouts. > - > -The latest version of this document can be found at: > - > - git tree: git://git.efficios.com/ctf.git > - gitweb: http://git.efficios.com/?p=ctf.git > - > -A reference implementation of a library to read and write this trace format > is > -being implemented within the BabelTrace project, a converter between trace > -formats. The development tree is available at: > - > - git tree: git://git.efficios.com/babeltrace.git > - gitweb: http://git.efficios.com/?p=babeltrace.git > - > -The CE Workgroup of the Linux Foundation, Ericsson, and EfficiOS have > -sponsored this work. > - > - > -Table of Contents > - > -1. Preliminary definitions > -2. High-level representation of a trace > -3. Event stream > -4. Types > - 4.1 Basic types > - 4.1.1 Type inheritance > - 4.1.2 Alignment > - 4.1.3 Byte order > - 4.1.4 Size > - 4.1.5 Integers > - 4.1.6 GNU/C bitfields > - 4.1.7 Floating point > - 4.1.8 Enumerations > - 4.2 Compound types > - 4.2.1 Structures > - 4.2.2 Variants (Discriminated/Tagged Unions) > - 4.2.3 Arrays > - 4.2.4 Sequences > - 4.2.5 Strings > -5. Event Packet Header > - 5.1 Event Packet Header Description > - 5.2 Event Packet Context Description > -6. Event Structure > - 6.1 Event Header > - 6.1.1 Type 1 - Few event IDs > - 6.1.2 Type 2 - Many event IDs > - 6.2 Stream Event Context and Event Context > - 6.3 Event Payload > - 6.3.1 Padding > - 6.3.2 Alignment > -7. Trace Stream Description Language (TSDL) > - 7.1 Meta-data > - 7.2 Declaration vs Definition > - 7.3 TSDL Scopes > - 7.3.1 Lexical Scope > - 7.3.2 Static and Dynamic Scopes > - 7.4 TSDL Examples > -8. Clocks > - > - > -1. Preliminary definitions > - > - - Event Trace: An ordered sequence of events. > - - Event Stream: An ordered sequence of events, containing a subset of the > - trace event types. > - - Event Packet: A sequence of physically contiguous events within an event > - stream. > - - Event: This is the basic entry in a trace. (aka: a trace record). > - - An event identifier (ID) relates to the class (a type) of event within > - an event stream. > - e.g. event: irq_entry. > - - An event (or event record) relates to a specific instance of an event > - class. > - e.g. event: irq_entry, at time X, on CPU Y > - - Source Architecture: Architecture writing the trace. > - - Reader Architecture: Architecture reading the trace. > - > - > -2. High-level representation of a trace > - > -A trace is divided into multiple event streams. Each event stream contains a > -subset of the trace event types. > - > -The final output of the trace, after its generation and optional transport > over > -the network, is expected to be either on permanent or temporary storage in a > -virtual file system. Because each event stream is appended to while a trace > is > -being recorded, each is associated with a distinct set of files for > -output. Therefore, a stored trace can be represented as a directory > -containing zero, one or more files per stream. > - > -Meta-data description associated with the trace contains information on > -trace event types expressed in the Trace Stream Description Language > -(TSDL). This language describes: > - > -- Trace version. > -- Types available. > -- Per-trace event header description. > -- Per-stream event header description. > -- Per-stream event context description. > -- Per-event > - - Event type to stream mapping. > - - Event type to name mapping. > - - Event type to ID mapping. > - - Event context description. > - - Event fields description. > - > - > -3. Event stream > - > -An event stream can be divided into contiguous event packets of variable > -size. An event packet can contain a certain amount of padding at the > -end. The stream header is repeated at the beginning of each event > -packet. The rationale for the event stream design choices is explained > -in Appendix B. Stream Header Rationale. > - > -The event stream header will therefore be referred to as the "event packet > -header" throughout the rest of this document. > - > - > -4. Types > - > -Types are organized as type classes. Each type class belong to either of two > -kind of types: basic types or compound types. > - > -4.1 Basic types > - > -A basic type is a scalar type, as described in this section. It includes > -integers, GNU/C bitfields, enumerations, and floating point values. > - > -4.1.1 Type inheritance > - > -Type specifications can be inherited to allow deriving types from a > -type class. For example, see the uint32_t named type derived from the > "integer" > -type class below ("Integers" section). Types have a precise binary > -representation in the trace. A type class has methods to read and write > these > -types, but must be derived into a type to be usable in an event field. > - > -4.1.2 Alignment > - > -We define "byte-packed" types as aligned on the byte size, namely 8-bit. > -We define "bit-packed" types as following on the next bit, as defined by the > -"Integers" section. > - > -Each basic type must specify its alignment, in bits. Examples of > -possible alignments are: bit-packed (align = 1), byte-packed (align = > -8), or word-aligned (e.g. align = 32 or align = 64). The choice depends > -on the architecture preference and compactness vs performance trade-offs > -of the implementation. Architectures providing fast unaligned write > -byte-packed basic types to save space, aligning each type on byte > -boundaries (8-bit). Architectures with slow unaligned writes align types > -on specific alignment values. If no specific alignment is declared for a > -type, it is assumed to be bit-packed for integers with size not multiple > -of 8 bits and for gcc bitfields. All other basic types are byte-packed > -by default. It is however recommended to always specify the alignment > -explicitly. Alignment values must be power of two. Compound types are > -aligned as specified in their individual specification. > - > -The base offset used for field alignment is the start of the packet > -containing the field. For instance, a field aligned on 32-bit needs to > -be at an offset multiple of 32-bit from the start of the packet that > -contains it. > - > -TSDL meta-data attribute representation of a specific alignment: > - > - align = value; /* value in bits */ > - > -4.1.3 Byte order > - > -By default, byte order of a basic type is the byte order described in > -the trace description. It can be overridden by specifying a > -"byte_order" attribute for a basic type. Typical use-case is to specify > -the network byte order (big endian: "be") to save data captured from the > -network into the trace without conversion. > - > -TSDL meta-data representation: > - > - byte_order = native OR network OR be OR le; /* network and be are > aliases > */ > - > -The "native" keyword selects the byte order described in the trace > -description. The "network" byte order is an alias for big endian. > - > -Even though the trace description section is not per se a type, for sake > -of clarity, it should be noted that "native" and "network" byte orders > -are only allowed within type declaration. The byte_order specified in > -the trace description section only accepts "be" or "le" values. > - > -4.1.4 Size > - > -Type size, in bits, for integers and floats is that returned by "sizeof()" > in C > -multiplied by CHAR_BIT. > -We require the size of "char" and "unsigned char" types (CHAR_BIT) to be > fixed > -to 8 bits for cross-endianness compatibility. > - > -TSDL meta-data representation: > - > - size = value; (value is in bits) > - > -4.1.5 Integers > - > -Signed integers are represented in two-complement. Integer alignment, > -size, signedness and byte ordering are defined in the TSDL meta-data. > -Integers aligned on byte size (8-bit) and with length multiple of byte > -size (8-bit) correspond to the C99 standard integers. In addition, > -integers with alignment and/or size that are _not_ a multiple of the > -byte size are permitted; these correspond to the C99 standard bitfields, > -with the added specification that the CTF integer bitfields have a fixed > -binary representation. Integer size needs to be a positive integer. > -Integers of size 0 are forbidden. A MIT-licensed reference > -implementation of the CTF portable bitfields is available at: > - > - > http://git.efficios.com/?p=babeltrace.git;a=blob;f=include/babeltrace/bitfield.h > - > -Binary representation of integers: > - > -- On little and big endian: > - - Within a byte, high bits correspond to an integer high bits, and low > bits > - correspond to low bits. > -- On little endian: > - - Integer across multiple bytes are placed from the less significant to > the > - most significant. > - - Consecutive integers are placed from lower bits to higher bits (even > within > - a byte). > -- On big endian: > - - Integer across multiple bytes are placed from the most significant to > the > - less significant. > - - Consecutive integers are placed from higher bits to lower bits (even > within > - a byte). > - > -This binary representation is derived from the bitfield implementation in > GCC > -for little and big endian. However, contrary to what GCC does, integers can > -cross units boundaries (no padding is required). Padding can be explicitly > -added (see 4.1.6 GNU/C bitfields) to follow the GCC layout if needed. > - > -TSDL meta-data representation: > - > - integer { > - signed = true OR false; /* default false */ > - byte_order = native OR network OR be OR le; /* default native */ > - size = value; /* value in bits, no default > */ > - align = value; /* value in bits */ > - /* based used for pretty-printing output, default: decimal. */ > - base = decimal OR dec OR d OR i OR u OR 10 OR hexadecimal OR hex OR x OR > X OR p OR 16 > - OR octal OR oct OR o OR 8 OR binary OR b OR 2; > - /* character encoding, default: none */ > - encoding = none or UTF8 or ASCII; > - } > - > -Example of type inheritance (creation of a uint32_t named type): > - > -typealias integer { > - size = 32; > - signed = false; > - align = 32; > -} := uint32_t; > - > -Definition of a named 5-bit signed bitfield: > - > -typealias integer { > - size = 5; > - signed = true; > - align = 1; > -} := int5_t; > - > -The character encoding field can be used to specify that the integer > -must be printed as a text character when read. e.g.: > - > -typealias integer { > - size = 8; > - align = 8; > - signed = false; > - encoding = UTF8; > -} := utf_char; > - > - > -4.1.6 GNU/C bitfields > - > -The GNU/C bitfields follow closely the integer representation, with a > -particularity on alignment: if a bitfield cannot fit in the current unit, > the > -unit is padded and the bitfield starts at the following unit. The unit size > is > -defined by the size of the type "unit_type". > - > -TSDL meta-data representation: > - > - unit_type name:size; > - > -As an example, the following structure declared in C compiled by GCC: > - > -struct example { > - short a:12; > - short b:5; > -}; > - > -The example structure is aligned on the largest element (short). The second > -bitfield would be aligned on the next unit boundary, because it would not > fit in > -the current unit. > - > -4.1.7 Floating point > - > -The floating point values byte ordering is defined in the TSDL meta-data. > - > -Floating point values follow the IEEE 754-2008 standard interchange formats. > -Description of the floating point values include the exponent and mantissa > size > -in bits. Some requirements are imposed on the floating point values: > - > -- FLT_RADIX must be 2. > -- mant_dig is the number of digits represented in the mantissa. It is > specified > - by the ISO C99 standard, section 5.2.4, as FLT_MANT_DIG, DBL_MANT_DIG and > - LDBL_MANT_DIG as defined by <float.h>. > -- exp_dig is the number of digits represented in the exponent. Given that > - mant_dig is one bit more than its actual size in bits (leading 1 is not > - needed) and also given that the sign bit always takes one bit, exp_dig can > be > - specified as: > - > - - sizeof(float) * CHAR_BIT - FLT_MANT_DIG > - - sizeof(double) * CHAR_BIT - DBL_MANT_DIG > - - sizeof(long double) * CHAR_BIT - LDBL_MANT_DIG > - > -TSDL meta-data representation: > - > -floating_point { > - exp_dig = value; > - mant_dig = value; > - byte_order = native OR network OR be OR le; > - align = value; > -} > - > -Example of type inheritance: > - > -typealias floating_point { > - exp_dig = 8; /* sizeof(float) * CHAR_BIT - FLT_MANT_DIG */ > - mant_dig = 24; /* FLT_MANT_DIG */ > - byte_order = native; > - align = 32; > -} := float; > - > -TODO: define NaN, +inf, -inf behavior. > - > -Bit-packed, byte-packed or larger alignments can be used for floating > -point values, similarly to integers. > - > -4.1.8 Enumerations > - > -Enumerations are a mapping between an integer type and a table of strings. > The > -numerical representation of the enumeration follows the integer type > specified > -by the meta-data. The enumeration mapping table is detailed in the > enumeration > -description within the meta-data. The mapping table maps inclusive value > -ranges (or single values) to strings. Instead of being limited to simple > -"value -> string" mappings, these enumerations map > -"[ start_value ... end_value ] -> string", which map inclusive ranges of > -values to strings. An enumeration from the C language can be represented in > -this format by having the same start_value and end_value for each > -mapping, which is in fact a range of size 1. This single-value range is > -supported without repeating the start and end values with the value = > -string declaration. Enumerations need to contain at least one entry. > - > -enum name : integer_type { > - somestring = start_value1 ... end_value1, > - "other string" = start_value2 ... end_value2, > - yet_another_string, /* will be assigned to end_value2 + 1 */ > - "some other string" = value, > - ... > -}; > - > -If the values are omitted, the enumeration starts at 0 and increment of 1 > for > -each entry. An entry with omitted value that follows a range entry takes > -as value the end_value of the previous range + 1: > - > -enum name : unsigned int { > - ZERO, > - ONE, > - TWO, > - TEN = 10, > - ELEVEN, > -}; > - > -Overlapping ranges within a single enumeration are implementation defined. > - > -A nameless enumeration can be declared as a field type or as part of a > typedef: > - > -enum : integer_type { > - ... > -} > - > -Enumerations omitting the container type ": integer_type" use the "int" > -type (for compatibility with C99). The "int" type must be previously > -declared. E.g.: > - > -typealias integer { size = 32; align = 32; signed = true; } := int; > - > -enum { > - ... > -} > - > - > -4.2 Compound types > - > -Compound are aggregation of type declarations. Compound types include > -structures, variant, arrays, sequences, and strings. > - > -4.2.1 Structures > - > -Structures are aligned on the largest alignment required by basic types > -contained within the structure. (This follows the ISO/C standard for > structures) > - > -TSDL meta-data representation of a named structure: > - > -struct name { > - field_type field_name; > - field_type field_name; > - ... > -}; > - > -Example: > - > -struct example { > - integer { /* Nameless type */ > - size = 16; > - signed = true; > - align = 16; > - } first_field_name; > - uint64_t second_field_name; /* Named type declared in the meta-data */ > -}; > - > -The fields are placed in a sequence next to each other. They each > -possess a field name, which is a unique identifier within the structure. > -The identifier is not allowed to use any reserved keyword > -(see Section C.1.2). Replacing reserved keywords with > -underscore-prefixed field names is recommended. Fields starting with an > -underscore should have their leading underscore removed by the CTF trace > -readers. > - > -A nameless structure can be declared as a field type or as part of a > typedef: > - > -struct { > - ... > -} > - > -Alignment for a structure compound type can be forced to a minimum value > -by adding an "align" specifier after the declaration of a structure > -body. This attribute is read as: align(value). The value is specified in > -bits. The structure will be aligned on the maximum value between this > -attribute and the alignment required by the basic types contained within > -the structure. e.g. > - > -struct { > - ... > -} align(32) > - > -4.2.2 Variants (Discriminated/Tagged Unions) > - > -A CTF variant is a selection between different types. A CTF variant must > -always be defined within the scope of a structure or within fields > -contained within a structure (defined recursively). A "tag" enumeration > -field must appear in either the same static scope, prior to the variant > -field (in field declaration order), in an upper static scope, or in an > -upper dynamic scope (see Section 7.3.2). The type selection is indicated > -by the mapping from the enumeration value to the string used as variant > -type selector. The field to use as tag is specified by the "tag_field", > -specified between "< >" after the "variant" keyword for unnamed > -variants, and after "variant name" for named variants. It is not > -required that each enumeration mapping appears as variant type tag > -field. It is also not required that each variant type tag appears as > -enumeration mapping. However, it is required that any enumeration > -mapping encountered within a stream has a matching variant type tag > -field. > - > -The alignment of the variant is the alignment of the type as selected by > -the tag value for the specific instance of the variant. The size of the > -variant is the size as selected by the tag value for the specific > -instance of the variant. > - > -The alignment of the type containing the variant is independent of the > -variant alignment. For instance, if a structure contains two fields, a > -32-bit integer, aligned on 32 bits, and a variant, which contains two > -choices: either a 32-bit field, aligned on 32 bits, or a 64-bit field, > -aligned on 64 bits, the alignment of the outmost structure will be > -32-bit (the alignment of its largest field, disregarding the alignment > -of the variant). The alignment of the variant will depend on the > -selector: if the variant's 32-bit field is selected, its alignment will > -be 32-bit, or 64-bit otherwise. It is important to note that variants > -are specifically tailored for compactness in a stream. Therefore, the > -relative offsets of compound type fields can vary depending on > -the offset at which the compound type starts if it contains a variant > -that itself contains a type with alignment larger than the largest field > -contained within the compound type. This is caused by the fact that the > -compound type may contain the enumeration that select the variant's > -choice, and therefore the alignment to be applied to the compound type > -cannot be determined before encountering the enumeration. > - > -Each variant type selector possess a field name, which is a unique > -identifier within the variant. The identifier is not allowed to use any > -reserved keyword (see Section C.1.2). Replacing reserved keywords with > -underscore-prefixed field names is recommended. Fields starting with an > -underscore should have their leading underscore removed by the CTF trace > -readers. > - > - > -A named variant declaration followed by its definition within a structure > -declaration: > - > -variant name { > - field_type sel1; > - field_type sel2; > - field_type sel3; > - ... > -}; > - > -struct { > - enum : integer_type { sel1, sel2, sel3, ... } tag_field; > - ... > - variant name <tag_field> v; > -} > - > -An unnamed variant definition within a structure is expressed by the > following > -TSDL meta-data: > - > -struct { > - enum : integer_type { sel1, sel2, sel3, ... } tag_field; > - ... > - variant <tag_field> { > - field_type sel1; > - field_type sel2; > - field_type sel3; > - ... > - } v; > -} > - > -Example of a named variant within a sequence that refers to a single tag > field: > - > -variant example { > - uint32_t a; > - uint64_t b; > - short c; > -}; > - > -struct { > - enum : uint2_t { a, b, c } choice; > - unsigned int seqlen; > - variant example <choice> v[seqlen]; > -} > - > -Example of an unnamed variant: > - > -struct { > - enum : uint2_t { a, b, c, d } choice; > - /* Unrelated fields can be added between the variant and its tag */ > - int32_t somevalue; > - variant <choice> { > - uint32_t a; > - uint64_t b; > - short c; > - struct { > - unsigned int field1; > - uint64_t field2; > - } d; > - } s; > -} > - > -Example of an unnamed variant within an array: > - > -struct { > - enum : uint2_t { a, b, c } choice; > - variant <choice> { > - uint32_t a; > - uint64_t b; > - short c; > - } v[10]; > -} > - > -Example of a variant type definition within a structure, where the defined > type > -is then declared within an array of structures. This variant refers to a tag > -located in an upper static scope. This example clearly shows that a variant > -type definition referring to the tag "x" uses the closest preceding field > from > -the static scope of the type definition. > - > -struct { > - enum : uint2_t { a, b, c, d } x; > - > - typedef variant <x> { /* > - * "x" refers to the preceding "x" enumeration in the > - * static scope of the type definition. > - */ > - uint32_t a; > - uint64_t b; > - short c; > - } example_variant; > - > - struct { > - enum : int { x, y, z } x; /* This enumeration is not used by "v". > */ > - example_variant v; /* > - * "v" uses the "enum : uint2_t { a, b, c, d }" > - * tag. > - */ > - } a[10]; > -} > - > -4.2.3 Arrays > - > -Arrays are fixed-length. Their length is declared in the type > -declaration within the meta-data. They contain an array of "inner type" > -elements, which can refer to any type not containing the type of the > -array being declared (no circular dependency). The length is the number > -of elements in an array. > - > -TSDL meta-data representation of a named array: > - > -typedef elem_type name[length]; > - > -A nameless array can be declared as a field type within a structure, e.g.: > - > - uint8_t field_name[10]; > - > -Arrays are always aligned on their element alignment requirement. > - > -4.2.4 Sequences > - > -Sequences are dynamically-sized arrays. They refer to a "length" > -unsigned integer field, which must appear in either the same static scope, > -prior to the sequence field (in field declaration order), in an upper > -static scope, or in an upper dynamic scope (see Section 7.3.2). This > -length field represents the number of elements in the sequence. The > -sequence per se is an array of "inner type" elements. > - > -TSDL meta-data representation for a sequence type definition: > - > -struct { > - unsigned int length_field; > - typedef elem_type typename[length_field]; > - typename seq_field_name; > -} > - > -A sequence can also be declared as a field type, e.g.: > - > -struct { > - unsigned int length_field; > - long seq_field_name[length_field]; > -} > - > -Multiple sequences can refer to the same length field, and these length > -fields can be in a different upper dynamic scope: > - > -e.g., assuming the stream.event.header defines: > - > -stream { > - ... > - id = 1; > - event.header := struct { > - uint16_t seq_len; > - }; > -}; > - > -event { > - ... > - stream_id = 1; > - fields := struct { > - long seq_a[stream.event.header.seq_len]; > - char seq_b[stream.event.header.seq_len]; > - }; > -}; > - > -The sequence elements follow the "array" specifications. > - > -4.2.5 Strings > - > -Strings are an array of bytes of variable size and are terminated by a '\0' > -"NULL" character. Their encoding is described in the TSDL meta-data. In > -absence of encoding attribute information, the default encoding is > -UTF-8. > - > -TSDL meta-data representation of a named string type: > - > -typealias string { > - encoding = UTF8 OR ASCII; > -} := name; > - > -A nameless string type can be declared as a field type: > - > -string field_name; /* Use default UTF8 encoding */ > - > -Strings are always aligned on byte size. > - > -5. Event Packet Header > - > -The event packet header consists of two parts: the "event packet header" > -is the same for all streams of a trace. The second part, the "event > -packet context", is described on a per-stream basis. Both are described > -in the TSDL meta-data. > - > -Event packet header (all fields are optional, specified by TSDL meta-data): > - > -- Magic number (CTF magic number: 0xC1FC1FC1) specifies that this is a > - CTF packet. This magic number is optional, but when present, it should > - come at the very beginning of the packet. > -- Trace UUID, used to ensure the event packet match the meta-data used. > - (note: we cannot use a meta-data checksum in every cases instead of a > - UUID because meta-data can be appended to while tracing is active) > - This field is optional. > -- Stream ID, used as reference to stream description in meta-data. > - This field is optional if there is only one stream description in the > - meta-data, but becomes required if there are more than one stream in > - the TSDL meta-data description. > - > -Event packet context (all fields are optional, specified by TSDL meta-data): > - > -- Event packet content size (in bits). > -- Event packet size (in bits, includes padding). > -- Event packet content checksum. Checksum excludes the event packet > - header. > -- Per-stream event packet sequence count (to deal with UDP packet loss). The > - number of significant sequence counter bits should also be present, so > - wrap-arounds are dealt with correctly. > -- Time-stamp at the beginning and time-stamp at the end of the event packet. > - Both timestamps are written in the packet header, but sampled respectively > - while (or before) writing the first event and while (or after) writing the > - last event in the packet. The inclusive range between these timestamps > should > - include all event timestamps assigned to events contained within the > packet. > - The timestamp at the beginning of an event packet is guaranteed to be > - below or equal the timestamp at the end of that event packet. > - The timestamp at the end of an event packet is guaranteed to be below > - or equal the timestamps at the end of any following packet within the > - same stream. See Section 8. Clocks for more detail. > -- Events discarded count > - - Snapshot of a per-stream free-running counter, counting the number of > - events discarded that were supposed to be written in the stream after > - the last event in the event packet. > - * Note: producer-consumer buffer full condition can fill the current > - event packet with padding so we know exactly where events have > been > - discarded. However, if the buffer full condition chooses not > - to fill the current event packet with padding, all we know > - about the timestamp range in which the events have been > - discarded is that it is somewhere between the beginning and > - the end of the packet. > -- Lossless compression scheme used for the event packet content. Applied > - directly to raw data. New types of compression can be added in following > - versions of the format. > - 0: no compression scheme > - 1: bzip2 > - 2: gzip > - 3: xz > -- Cypher used for the event packet content. Applied after compression. > - 0: no encryption > - 1: AES > -- Checksum scheme used for the event packet content. Applied after > encryption. > - 0: no checksum > - 1: md5 > - 2: sha1 > - 3: crc32 > - > -5.1 Event Packet Header Description > - > -The event packet header layout is indicated by the trace packet.header > -field. Here is a recommended structure type for the packet header with > -the fields typically expected (although these fields are each optional): > - > -struct event_packet_header { > - uint32_t magic; > - uint8_t uuid[16]; > - uint32_t stream_id; > -}; > - > -trace { > - ... > - packet.header := struct event_packet_header; > -}; > - > -If the magic number is not present, tools such as "file" will have no > -mean to discover the file type. > - > -If the uuid is not present, no validation that the meta-data actually > -corresponds to the stream is performed. > - > -If the stream_id packet header field is missing, the trace can only > -contain a single stream. Its "id" field can be left out, and its events > -don't need to declare a "stream_id" field. > - > - > -5.2 Event Packet Context Description > - > -Event packet context example. These are declared within the stream > declaration > -in the meta-data. All these fields are optional. If the packet size field is > -missing, the whole stream only contains a single packet. If the content > -size field is missing, the packet is filled (no padding). The content > -and packet sizes include all headers. > - > -An example event packet context type: > - > -struct event_packet_context { > - uint64_t timestamp_begin; > - uint64_t timestamp_end; > - uint32_t checksum; > - uint32_t stream_packet_count; > - uint32_t events_discarded; > - uint32_t cpu_id; > - uint64_t/uint32_t/uint16_t content_size; > - uint64_t/uint32_t/uint16_t packet_size; > - uint8_t compression_scheme; > - uint8_t encryption_scheme; > - uint8_t checksum_scheme; > -}; > - > - > -6. Event Structure > - > -The overall structure of an event is: > - > -1 - Event Header (as specified by the stream meta-data) > - 2 - Stream Event Context (as specified by the stream meta-data) > - 3 - Event Context (as specified by the event meta-data) > - 4 - Event Payload (as specified by the event meta-data) > - > -This structure defines an implicit dynamic scoping, where variants > -located in inner structures (those with a higher number in the listing > -above) can refer to the fields of outer structures (with lower number in > -the listing above). See Section 7.3 TSDL Scopes for more detail. > - > -The total length of an event is defined as the difference between the > -end of its Event Payload and the end of the previous event's Event > -Payload. Therefore, it includes the event header alignment padding, and > -all its fields and their respective alignment padding. Events of length > -0 are forbidden. > - > -6.1 Event Header > - > -Event headers can be described within the meta-data. We hereby propose, as > an > -example, two types of events headers. Type 1 accommodates streams with less > than > -31 event IDs. Type 2 accommodates streams with 31 or more event IDs. > - > -One major factor can vary between streams: the number of event IDs assigned > to > -a stream. Luckily, this information tends to stay relatively constant > (modulo > -event registration while trace is being recorded), so we can specify > different > -representations for streams containing few event IDs and streams containing > -many event IDs, so we end up representing the event ID and time-stamp as > -densely as possible in each case. > - > -The header is extended in the rare occasions where the information cannot be > -represented in the ranges available in the standard event header. They are > also > -used in the rare occasions where the data required for a field could not be > -collected: the flag corresponding to the missing field within the > missing_fields > -array is then set to 1. > - > -Types uintX_t represent an X-bit unsigned integer, as declared with > -either: > - > - typealias integer { size = X; align = X; signed = false; } := uintX_t; > - > - or > - > - typealias integer { size = X; align = 1; signed = false; } := uintX_t; > - > -For more information about timestamp fields, see Section 8. Clocks. > - > -6.1.1 Type 1 - Few event IDs > - > - - Aligned on 32-bit (or 8-bit if byte-packed, depending on the > architecture > - preference). > - - Native architecture byte ordering. > - - For "compact" selection > - - Fixed size: 32 bits. > - - For "extended" selection > - - Size depends on the architecture and variant alignment. > - > -struct event_header_1 { > - /* > - * id: range: 0 - 30. > - * id 31 is reserved to indicate an extended header. > - */ > - enum : uint5_t { compact = 0 ... 30, extended = 31 } id; > - variant <id> { > - struct { > - uint27_t timestamp; > - } compact; > - struct { > - uint32_t id; /* 32-bit event IDs */ > - uint64_t timestamp; /* 64-bit timestamps */ > - } extended; > - } v; > -} align(32); /* or align(8) */ > - > - > -6.1.2 Type 2 - Many event IDs > - > - - Aligned on 16-bit (or 8-bit if byte-packed, depending on the > architecture > - preference). > - - Native architecture byte ordering. > - - For "compact" selection > - - Size depends on the architecture and variant alignment. > - - For "extended" selection > - - Size depends on the architecture and variant alignment. > - > -struct event_header_2 { > - /* > - * id: range: 0 - 65534. > - * id 65535 is reserved to indicate an extended header. > - */ > - enum : uint16_t { compact = 0 ... 65534, extended = 65535 } id; > - variant <id> { > - struct { > - uint32_t timestamp; > - } compact; > - struct { > - uint32_t id; /* 32-bit event IDs */ > - uint64_t timestamp; /* 64-bit timestamps */ > - } extended; > - } v; > -} align(16); /* or align(8) */ > - > - > -6.2 Stream Event Context and Event Context > - > -The event context contains information relative to the current event. > -The choice and meaning of this information is specified by the TSDL > -stream and event meta-data descriptions. The stream context is applied > -to all events within the stream. The stream context structure follows > -the event header. The event context is applied to specific events. Its > -structure follows the stream context structure. > - > -An example of stream-level event context is to save the event payload size > with > -each event, or to save the current PID with each event. These are declared > -within the stream declaration within the meta-data: > - > - stream { > - ... > - event.context := struct { > - uint pid; > - uint16_t payload_size; > - }; > - }; > - > -An example of event-specific event context is to declare a bitmap of missing > -fields, only appended after the stream event context if the extended event > -header is selected. NR_FIELDS is the number of fields within the event (a > -numeric value). > - > - event { > - context := struct { > - variant <id> { > - struct { } compact; > - struct { > - uint1_t missing_fields[NR_FIELDS]; /* missing event fields bitmap > */ > - } extended; > - } v; > - }; > - ... > - } > - > -6.3 Event Payload > - > -An event payload contains fields specific to a given event type. The fields > -belonging to an event type are described in the event-specific meta-data > -within a structure type. > - > -6.3.1 Padding > - > -No padding at the end of the event payload. This differs from the ISO/C > standard > -for structures, but follows the CTF standard for structures. In a trace, > even > -though it makes sense to align the beginning of a structure, it really makes > no > -sense to add padding at the end of the structure, because structures are > usually > -not followed by a structure of the same type. > - > -This trick can be done by adding a zero-length "end" field at the end of the > C > -structures, and by using the offset of this field rather than using sizeof() > -when calculating the size of a structure (see Appendix "A. Helper macros"). > - > -6.3.2 Alignment > - > -The event payload is aligned on the largest alignment required by types > -contained within the payload. (This follows the ISO/C standard for > structures) > - > - > -7. Trace Stream Description Language (TSDL) > - > -The Trace Stream Description Language (TSDL) allows expression of the > -binary trace streams layout in a C99-like Domain Specific Language > -(DSL). > - > - > -7.1 Meta-data > - > -The trace stream layout description is located in the trace meta-data. > -The meta-data is itself located in a stream identified by its name: > -"metadata". > - > -The meta-data description can be expressed in two different formats: > -text-only and packet-based. The text-only description facilitates > -generation of meta-data and provides a convenient way to enter the > -meta-data information by hand. The packet-based meta-data provides the > -CTF stream packet facilities (checksumming, compression, encryption, > -network-readiness) for meta-data stream generated and transported by a > -tracer. > - > -The text-only meta-data file is a plain-text TSDL description. This file > -must begin with the following characters to identify the file as a CTF > -TSDL text-based metadata file (without the double-quotes) : > - > -"/* CTF" > - > -It must be followed by a space, and the version of the specification > -followed by the CTF trace, e.g.: > - > -" 1.8" > - > -These characters allow automated discovery of file type and CTF > -specification version. They are interpreted as a the beginning of a > -comment by the TSDL metadata parser. The comment can be continued to > -contain extra commented characters before it is closed. > - > -The packet-based meta-data is made of "meta-data packets", which each > -start with a meta-data packet header. The packet-based meta-data > -description is detected by reading the magic number "0x75D11D57" at the > -beginning of the file. This magic number is also used to detect the > -endianness of the architecture by trying to read the CTF magic number > -and its counterpart in reversed endianness. The events within the > -meta-data stream have no event header nor event context. Each event only > -contains a special "sequence" payload, which is a sequence of bits which > -length is implicitly calculated by using the > -"trace.packet.header.content_size" field, minus the packet header size. > -The formatting of this sequence of bits is a plain-text representation > -of the TSDL description. Each meta-data packet start with a special > -packet header, specific to the meta-data stream, which contains, > -exactly: > - > -struct metadata_packet_header { > - uint32_t magic; /* 0x75D11D57 */ > - uint8_t uuid[16]; /* Unique Universal Identifier */ > - uint32_t checksum; /* 0 if unused */ > - uint32_t content_size; /* in bits */ > - uint32_t packet_size; /* in bits */ > - uint8_t compression_scheme; /* 0 if unused */ > - uint8_t encryption_scheme; /* 0 if unused */ > - uint8_t checksum_scheme; /* 0 if unused */ > - uint8_t major; /* CTF spec version major number */ > - uint8_t minor; /* CTF spec version minor number */ > -}; > - > -The packet-based meta-data can be converted to a text-only meta-data by > -concatenating all the strings it contains. > - > -In the textual representation of the meta-data, the text contained > -within "/*" and "*/", as well as within "//" and end of line, are > -treated as comments. Boolean values can be represented as true, TRUE, > -or 1 for true, and false, FALSE, or 0 for false. Within the string-based > -meta-data description, the trace UUID is represented as a string of > -hexadecimal digits and dashes "-". In the event packet header, the trace > -UUID is represented as an array of bytes. > - > - > -7.2 Declaration vs Definition > - > -A declaration associates a layout to a type, without specifying where > -this type is located in the event structure hierarchy (see Section 6). > -This therefore includes typedef, typealias, as well as all type > -specifiers. In certain circumstances (typedef, structure field and > -variant field), a declaration is followed by a declarator, which specify > -the newly defined type name (for typedef), or the field name (for > -declarations located within structure and variants). Array and sequence, > -declared with square brackets ("[" "]"), are part of the declarator, > -similarly to C99. The enumeration base type is specified by > -": enum_base", which is part of the type specifier. The variant tag > -name, specified between "<" ">", is also part of the type specifier. > - > -A definition associates a type to a location in the event structure > -hierarchy (see Section 6). This association is denoted by ":=", as shown > -in Section 7.3. > - > - > -7.3 TSDL Scopes > - > -TSDL uses three different types of scoping: a lexical scope is used for > -declarations and type definitions, and static and dynamic scopes are > -used for variants references to tag fields (with relative and absolute > -path lookups) and for sequence references to length fields. > - > -7.3.1 Lexical Scope > - > -Each of "trace", "env", "stream", "event", "struct" and "variant" have > -their own nestable declaration scope, within which types can be declared > -using "typedef" and "typealias". A root declaration scope also contains > -all declarations located outside of any of the aforementioned > -declarations. An inner declaration scope can refer to type declared > -within its container lexical scope prior to the inner declaration scope. > -Redefinition of a typedef or typealias is not valid, although hiding an > -upper scope typedef or typealias is allowed within a sub-scope. > - > -7.3.2 Static and Dynamic Scopes > - > -A local static scope consists in the scope generated by the declaration > -of fields within a compound type. A static scope is a local static scope > -augmented with the nested sub-static-scopes it contains. > - > -A dynamic scope consists in the static scope augmented with the > -implicit event structure definition hierarchy presented at Section 6. > - > -Multiple declarations of the same field name within a local static scope > -is not valid. It is however valid to re-use the same field name in > -different local scopes. > - > -Nested static and dynamic scopes form lookup paths. These are used for > -variant tag and sequence length references. They are used at the variant > -and sequence definition site to look up the location of the tag field > -associated with a variant, and to lookup up the location of the length > -field associated with a sequence. > - > -Variants and sequences can refer to a tag field either using a relative > -path or an absolute path. The relative path is relative to the scope in > -which the variant or sequence performing the lookup is located. > -Relative paths are only allowed to lookup within the same static scope, > -which includes its nested static scopes. Lookups targeting parent static > -scopes need to be performed with an absolute path. > - > -Absolute path lookups use the full path including the dynamic scope > -followed by a "." and then the static scope. Therefore, variants (or > -sequences) in lower levels in the dynamic scope (e.g. event context) can > -refer to a tag (or length) field located in upper levels (e.g. in the > -event header) by specifying, in this case, the associated tag with > -<stream.event.header.field_name>. This allows, for instance, the event > -context to define a variant referring to the "id" field of the event > -header as selector. > - > -The dynamic scope prefixes are thus: > - > - - Trace Environment: <env. >, > - - Trace Packet Header: <trace.packet.header. >, > - - Stream Packet Context: <stream.packet.context. >, > - - Event Header: <stream.event.header. >, > - - Stream Event Context: <stream.event.context. >, > - - Event Context: <event.context. >, > - - Event Payload: <event.fields. >. > - > - > -The target dynamic scope must be specified explicitly when referring to > -a field outside of the static scope (absolute scope reference). No > -conflict can occur between relative and dynamic paths, because the > -keywords "trace", "stream", and "event" are reserved, and thus > -not permitted as field names. It is recommended that field names > -clashing with CTF and C99 reserved keywords use an underscore prefix to > -eliminate the risk of generating a description containing an invalid > -field name. Consequently, fields starting with an underscore should have > -their leading underscore removed by the CTF trace readers. > - > - > -The information available in the dynamic scopes can be thought of as the > -current tracing context. At trace production, information about the > -current context is saved into the specified scope field levels. At trace > -consumption, for each event, the current trace context is therefore > -readable by accessing the upper dynamic scopes. > - > - > -7.4 TSDL Examples > - > -The grammar representing the TSDL meta-data is presented in Appendix C. > -TSDL Grammar. This section presents a rather lighter reading that > -consists in examples of TSDL meta-data, with template values. > - > -The stream "id" can be left out if there is only one stream in the > -trace. The event "id" field can be left out if there is only one event > -in a stream. > - > -trace { > - major = value; /* CTF spec version major number */ > - minor = value; /* CTF spec version minor number */ > - uuid = "aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa"; /* Trace UUID */ > - byte_order = be OR le; /* Endianness (required) */ > - packet.header := struct { > - uint32_t magic; > - uint8_t uuid[16]; > - uint32_t stream_id; > - }; > -}; > - > -/* > - * The "env" (environment) scope contains assignment expressions. The > - * field names and content are implementation-defined. > - */ > -env { > - pid = value; /* example */ > - proc_name = "name"; /* example */ > - ... > -}; > - > -stream { > - id = stream_id; > - /* Type 1 - Few event IDs; Type 2 - Many event IDs. See section 6.1. */ > - event.header := event_header_1 OR event_header_2; > - event.context := struct { > - ... > - }; > - packet.context := struct { > - ... > - }; > -}; > - > -event { > - name = "event_name"; > - id = value; /* Numeric identifier within the stream > */ > - stream_id = stream_id; > - loglevel = value; > - model.emf.uri = "string"; > - context := struct { > - ... > - }; > - fields := struct { > - ... > - }; > -}; > - > -callsite { > - name = "event_name"; > - func = "func_name"; > - file = "myfile.c"; > - line = 39; > - ip = 0x40096c; > -}; > - > -/* More detail on types in section 4. Types */ > - > -/* > - * Named types: > - * > - * Type declarations behave similarly to the C standard. > - */ > - > -typedef aliased_type_specifiers new_type_declarators; > - > -/* e.g.: typedef struct example new_type_name[10]; */ > - > -/* > - * typealias > - * > - * The "typealias" declaration can be used to give a name (including > - * pointer declarator specifier) to a type. It should also be used to > - * map basic C types (float, int, unsigned long, ...) to a CTF type. > - * Typealias is a superset of "typedef": it also allows assignment of a > - * simple variable identifier to a type. > - */ > - > -typealias type_class { > - ... > -} := type_specifiers type_declarator; > - > -/* > - * e.g.: > - * typealias integer { > - * size = 32; > - * align = 32; > - * signed = false; > - * } := struct page *; > - * > - * typealias integer { > - * size = 32; > - * align = 32; > - * signed = true; > - * } := int; > - */ > - > -struct name { > - ... > -}; > - > -variant name { > - ... > -}; > - > -enum name : integer_type { > - ... > -}; > - > - > -/* > - * Unnamed types, contained within compound type fields, typedef or > typealias. > - */ > - > -struct { > - ... > -} > - > -struct { > - ... > -} align(value) > - > -variant { > - ... > -} > - > -enum : integer_type { > - ... > -} > - > -typedef type new_type[length]; > - > -struct { > - type field_name[length]; > -} > - > -typedef type new_type[length_type]; > - > -struct { > - type field_name[length_type]; > -} > - > -integer { > - ... > -} > - > -floating_point { > - ... > -} > - > -struct { > - integer_type field_name:size; /* GNU/C bitfield */ > -} > - > -struct { > - string field_name; > -} > - > - > -8. Clocks > - > -Clock metadata allows to describe the clock topology of the system, as > -well as to detail each clock parameter. In absence of clock description, > -it is assumed that all fields named "timestamp" use the same clock > -source, which increments once per nanosecond. > - > -Describing a clock and how it is used by streams is threefold: first, > -the clock and clock topology should be described in a "clock" > -description block, e.g.: > - > -clock { > - name = cycle_counter_sync; > - uuid = "62189bee-96dc-11e0-91a8-cfa3d89f3923"; > - description = "Cycle counter synchronized across CPUs"; > - freq = 1000000000; /* frequency, in Hz */ > - /* precision in seconds is: 1000 * (1/freq) */ > - precision = 1000; > - /* > - * clock value offset from Epoch is: > - * offset_s + (offset * (1/freq)) > - */ > - offset_s = 1326476837; > - offset = 897235420; > - absolute = FALSE; > -}; > - > -The mandatory "name" field specifies the name of the clock identifier, > -which can later be used as a reference. The optional field "uuid" is the > -unique identifier of the clock. It can be used to correlate different > -traces that use the same clock. An optional textual description string > -can be added with the "description" field. The "freq" field is the > -initial frequency of the clock, in Hz. If the "freq" field is not > -present, the frequency is assumed to be 1000000000 (providing clock > -increment of 1 ns). The optional "precision" field details the > -uncertainty on the clock measurements, in (1/freq) units. The "offset_s" > -and "offset" fields indicate the offset from POSIX.1 Epoch, 1970-01-01 > -00:00:00 +0000 (UTC), to the zero of value of the clock. The "offset_s" > -field is in seconds. The "offset" field is in (1/freq) units. If any of > -the "offset_s" or "offset" field is not present, it is assigned the 0 > -value. The field "absolute" is TRUE if the clock is a global reference > -across different clock uuid (e.g. NTP time). Otherwise, "absolute" is > -FALSE, and the clock can be considered as synchronized only with other > -clocks that have the same uuid. > - > - > -Secondly, a reference to this clock should be added within an integer > -type: > - > -typealias integer { > - size = 64; align = 1; signed = false; > - map = clock.cycle_counter_sync.value; > -} := uint64_ccnt_t; > - > -Thirdly, stream declarations can reference the clock they use as a > -time-stamp source: > - > -struct packet_context { > - uint64_ccnt_t ccnt_begin; > - uint64_ccnt_t ccnt_end; > - /* ... */ > -}; > - > -stream { > - /* ... */ > - event.header := struct { > - uint64_ccnt_t timestamp; > - /* ... */ > - }; > - packet.context := struct packet_context; > -}; > - > -For a N-bit integer type referring to a clock, if the integer overflows > -compared to the N low order bits of the clock prior value found in the > -same stream, then it is assumed that one, and only one, overflow > -occurred. It is therefore important that events encoding time on a small > -number of bits happen frequently enough to detect when more than one > -N-bit overflow occurs. > - > -In a packet context, clock field names ending with "_begin" and "_end" > -have a special meaning: this refers to the time-stamps at, respectively, > -the beginning and the end of each packet. > - > - > -A. Helper macros > - > -The two following macros keep track of the size of a GNU/C structure without > -padding at the end by placing HEADER_END as the last field. A one byte end > field > -is used for C90 compatibility (C99 flexible arrays could be used here). Note > -that this does not affect the effective structure size, which should always > be > -calculated with the header_sizeof() helper. > - > -#define HEADER_END char end_field > -#define header_sizeof(type) offsetof(typeof(type), end_field) > - > - > -B. Stream Header Rationale > - > -An event stream is divided in contiguous event packets of variable size. > These > -subdivisions allow the trace analyzer to perform a fast binary search by > time > -within the stream (typically requiring to index only the event packet > headers) > -without reading the whole stream. These subdivisions have a variable size to > -eliminate the need to transfer the event packet padding when partially > filled > -event packets must be sent when streaming a trace for live viewing/analysis. > -An event packet can contain a certain amount of padding at the end. Dividing > -streams into event packets is also useful for network streaming over UDP and > -flight recorder mode tracing (a whole event packet can be swapped out of the > -buffer atomically for reading). > - > -The stream header is repeated at the beginning of each event packet to allow > -flexibility in terms of: > - > - - streaming support, > - - allowing arbitrary buffers to be discarded without making the trace > - unreadable, > - - allow UDP packet loss handling by either dealing with missing event > packet > - or asking for re-transmission. > - - transparently support flight recorder mode, > - - transparently support crash dump. > - > - > -C. TSDL Grammar > - > -/* > - * Common Trace Format (CTF) Trace Stream Description Language (TSDL) > Grammar. > - * > - * Inspired from the C99 grammar: > - * http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1124.pdf (Annex A) > - * and c++1x grammar (draft) > - * http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2011/n3291.pdf (Annex > A) > - * > - * Specialized for CTF needs by including only constant and declarations > from > - * C99 (excluding function declarations), and by adding support for > variants, > - * sequences and CTF-specific specifiers. Enumeration container types > - * semantic is inspired from c++1x enum-base. > - */ > - > -1) Lexical grammar > - > -1.1) Lexical elements > - > -token: > - keyword > - identifier > - constant > - string-literal > - punctuator > - > -1.2) Keywords > - > -keyword: is one of > - > -align > -callsite > -const > -char > -clock > -double > -enum > -env > -event > -floating_point > -float > -integer > -int > -long > -short > -signed > -stream > -string > -struct > -trace > -typealias > -typedef > -unsigned > -variant > -void > -_Bool > -_Complex > -_Imaginary > - > - > -1.3) Identifiers > - > -identifier: > - identifier-nondigit > - identifier identifier-nondigit > - identifier digit > - > -identifier-nondigit: > - nondigit > - universal-character-name > - any other implementation-defined characters > - > -nondigit: > - _ > - [a-zA-Z] /* regular expression */ > - > -digit: > - [0-9] /* regular expression */ > - > -1.4) Universal character names > - > -universal-character-name: > - \u hex-quad > - \U hex-quad hex-quad > - > -hex-quad: > - hexadecimal-digit hexadecimal-digit hexadecimal-digit hexadecimal-digit > - > -1.5) Constants > - > -constant: > - integer-constant > - enumeration-constant > - character-constant > - > -integer-constant: > - decimal-constant integer-suffix-opt > - octal-constant integer-suffix-opt > - hexadecimal-constant integer-suffix-opt > - > -decimal-constant: > - nonzero-digit > - decimal-constant digit > - > -octal-constant: > - 0 > - octal-constant octal-digit > - > -hexadecimal-constant: > - hexadecimal-prefix hexadecimal-digit > - hexadecimal-constant hexadecimal-digit > - > -hexadecimal-prefix: > - 0x > - 0X > - > -nonzero-digit: > - [1-9] > - > -integer-suffix: > - unsigned-suffix long-suffix-opt > - unsigned-suffix long-long-suffix > - long-suffix unsigned-suffix-opt > - long-long-suffix unsigned-suffix-opt > - > -unsigned-suffix: > - u > - U > - > -long-suffix: > - l > - L > - > -long-long-suffix: > - ll > - LL > - > -enumeration-constant: > - identifier > - string-literal > - > -character-constant: > - ' c-char-sequence ' > - L' c-char-sequence ' > - > -c-char-sequence: > - c-char > - c-char-sequence c-char > - > -c-char: > - any member of source charset except single-quote ('), backslash > - (\), or new-line character. > - escape-sequence > - > -escape-sequence: > - simple-escape-sequence > - octal-escape-sequence > - hexadecimal-escape-sequence > - universal-character-name > - > -simple-escape-sequence: one of > - \' \" \? \\ \a \b \f \n \r \t \v > - > -octal-escape-sequence: > - \ octal-digit > - \ octal-digit octal-digit > - \ octal-digit octal-digit octal-digit > - > -hexadecimal-escape-sequence: > - \x hexadecimal-digit > - hexadecimal-escape-sequence hexadecimal-digit > - > -1.6) String literals > - > -string-literal: > - " s-char-sequence-opt " > - L" s-char-sequence-opt " > - > -s-char-sequence: > - s-char > - s-char-sequence s-char > - > -s-char: > - any member of source charset except double-quote ("), backslash > - (\), or new-line character. > - escape-sequence > - > -1.7) Punctuators > - > -punctuator: one of > - [ ] ( ) { } . -> * + - < > : ; ... = , > - > - > -2) Phrase structure grammar > - > -primary-expression: > - identifier > - constant > - string-literal > - ( unary-expression ) > - > -postfix-expression: > - primary-expression > - postfix-expression [ unary-expression ] > - postfix-expression . identifier > - postfix-expressoin -> identifier > - > -unary-expression: > - postfix-expression > - unary-operator postfix-expression > - > -unary-operator: one of > - + - > - > -assignment-operator: > - = > - > -type-assignment-operator: > - := > - > -constant-expression-range: > - unary-expression ... unary-expression > - > -2.2) Declarations: > - > -declaration: > - declaration-specifiers declarator-list-opt ; > - ctf-specifier ; > - > -declaration-specifiers: > - storage-class-specifier declaration-specifiers-opt > - type-specifier declaration-specifiers-opt > - type-qualifier declaration-specifiers-opt > - > -declarator-list: > - declarator > - declarator-list , declarator > - > -abstract-declarator-list: > - abstract-declarator > - abstract-declarator-list , abstract-declarator > - > -storage-class-specifier: > - typedef > - > -type-specifier: > - void > - char > - short > - int > - long > - float > - double > - signed > - unsigned > - _Bool > - _Complex > - _Imaginary > - struct-specifier > - variant-specifier > - enum-specifier > - typedef-name > - ctf-type-specifier > - > -align-attribute: > - align ( unary-expression ) > - > -struct-specifier: > - struct identifier-opt { struct-or-variant-declaration-list-opt } > align-attribute-opt > - struct identifier align-attribute-opt > - > -struct-or-variant-declaration-list: > - struct-or-variant-declaration > - struct-or-variant-declaration-list struct-or-variant-declaration > - > -struct-or-variant-declaration: > - specifier-qualifier-list struct-or-variant-declarator-list ; > - declaration-specifiers-opt storage-class-specifier > declaration-specifiers-opt declarator-list ; > - typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declaration-specifiers abstract-declarator-list ; > - typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declarator-list ; > - > -specifier-qualifier-list: > - type-specifier specifier-qualifier-list-opt > - type-qualifier specifier-qualifier-list-opt > - > -struct-or-variant-declarator-list: > - struct-or-variant-declarator > - struct-or-variant-declarator-list , struct-or-variant-declarator > - > -struct-or-variant-declarator: > - declarator > - declarator-opt : unary-expression > - > -variant-specifier: > - variant identifier-opt variant-tag-opt { > struct-or-variant-declaration-list > } > - variant identifier variant-tag > - > -variant-tag: > - < unary-expression > > - > -enum-specifier: > - enum identifier-opt { enumerator-list } > - enum identifier-opt { enumerator-list , } > - enum identifier > - enum identifier-opt : declaration-specifiers { enumerator-list } > - enum identifier-opt : declaration-specifiers { enumerator-list , } > - > -enumerator-list: > - enumerator > - enumerator-list , enumerator > - > -enumerator: > - enumeration-constant > - enumeration-constant assignment-operator unary-expression > - enumeration-constant assignment-operator constant-expression-range > - > -type-qualifier: > - const > - > -declarator: > - pointer-opt direct-declarator > - > -direct-declarator: > - identifier > - ( declarator ) > - direct-declarator [ unary-expression ] > - > -abstract-declarator: > - pointer-opt direct-abstract-declarator > - > -direct-abstract-declarator: > - identifier-opt > - ( abstract-declarator ) > - direct-abstract-declarator [ unary-expression ] > - direct-abstract-declarator [ ] > - > -pointer: > - * type-qualifier-list-opt > - * type-qualifier-list-opt pointer > - > -type-qualifier-list: > - type-qualifier > - type-qualifier-list type-qualifier > - > -typedef-name: > - identifier > - > -2.3) CTF-specific declarations > - > -ctf-specifier: > - clock { ctf-assignment-expression-list-opt } > - event { ctf-assignment-expression-list-opt } > - stream { ctf-assignment-expression-list-opt } > - env { ctf-assignment-expression-list-opt } > - trace { ctf-assignment-expression-list-opt } > - callsite { ctf-assignment-expression-list-opt } > - typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declaration-specifiers abstract-declarator-list > - typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declarator-list > - > -ctf-type-specifier: > - floating_point { ctf-assignment-expression-list-opt } > - integer { ctf-assignment-expression-list-opt } > - string { ctf-assignment-expression-list-opt } > - string > - > -ctf-assignment-expression-list: > - ctf-assignment-expression ; > - ctf-assignment-expression-list ctf-assignment-expression ; > - > -ctf-assignment-expression: > - unary-expression assignment-operator unary-expression > - unary-expression type-assignment-operator type-specifier > - declaration-specifiers-opt storage-class-specifier > declaration-specifiers-opt declarator-list > - typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declaration-specifiers abstract-declarator-list > - typealias declaration-specifiers abstract-declarator-list > type-assignment-operator declarator-list > -- > 2.2.1 > > > _______________________________________________ > lttng-dev mailing list > [email protected] > http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev > > -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com _______________________________________________ lttng-dev mailing list [email protected] http://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
