Thanks very much Mathieu. Appreciate the help. -Brian
From: Mathieu Desnoyers [mailto:mathieu.desnoy...@efficios.com] Sent: Wednesday, March 16, 2016 5:16 PM To: Brian Robbins <brian...@microsoft.com> Cc: lttng-dev <lttng-dev@lists.lttng.org>; Lee Culver <lecul...@microsoft.com> Subject: Re: [lttng-dev] CTF Compact Header Timestamps Hi Brian, You should think about those 27 bits as the "new lowest significant bits" of the 64-bit clock. Quoting the CTF spec to understand how to deal with 27-bit overflow between two consecutive time-stamps in the same stream (here N = 27): "For a N-bit integer type referring to a clock, if the integer overflows compared to the N low order bits of the clock prior value found in the same stream, then it is assumed that one, and only one, overflow occurred. It is therefore important that events encoding time on a small number of bits happen frequently enough to detect when more than one N-bit overflow occurs." Those 27 bits are _not_ an offset from the previous timestamp. They update the current notion of time (based on all prior timestamps read for this stream) by setting the low 27 bits to the new value. If that value happens to be lower than the previous 27 low bits of the current time, this means we had one 27-bit overflow, so we need to increment the 28th bit by one. In each packet header, you will find the full 64-bit timestamps, so we always have a full 64-bit base from which to start. It is up to the tracer (lttng) to ensure that there is never more than a single 27-bit overflow preceding a 27-bit compact timestamp. If lttng detects that there would be more than a single 27-bit overflow prior to a timestamp, it will use an extended header which stores the full 64-bit of the new timestamp. We rely on this time-compaction scheme rather than offsetting from the prior 64-bit timestamp because it allows us to weaken the sequential inter-dependency between consecutive events, which is useful for lock-free ring buffers. Let me know if you still have questions, Thanks, Mathieu ----- On Mar 16, 2016, at 6:55 PM, Brian Robbins <brian...@microsoft.com<mailto:brian...@microsoft.com>> wrote: Hello, I’m forwarding this on-behalf of one of my colleagues who is trying to parse CTF. Any help you can provide is greatly appreciated. Thanks. -Brian From: Lee Culver Sent: Wednesday, March 16, 2016 11:06 AM To: Brian Robbins <brian...@microsoft.com<mailto:brian...@microsoft.com>> Subject: CTF Compact Header Timestamps Event headers in LTTng can have a 27-bit clock or a 64-bit clock. I have been able to properly map the 64-bit clock back to a real timestamp, but I’m not sure how to interpret the 27-bit clock value. I’ve dug through the CTF specification but can’t find the relevant text. I have pasted the relevant metadata at the bottom of this email. For 64-bit clocks, this calculation gets us to the proper DateTime in C#: var clock = _metadata.Clocks.Single(); var offset = new DateTime(1970, 1, 1, 0, 0, 0, 0, DateTimeKind.Utc).AddSeconds(clock.Offset / clock.Frequency); long ticks = (long)(header.Timestamp * 10000000.0 / clock.Frequency); var time = new DateTime(offset.Ticks + ticks, DateTimeKind.Utc); However, for the ‘compact’ variant (see below), we end up with a timestamp that’s far lower than the frequency, such as ‘0x55363b9’ and I’m simply not sure how to interpret this. It does NOT seem to be an offset from the previous event since if I treat it as such we get events that are out of order. One such example is the following: Event #11 - Extended header timestamp (64-bit): 0x00098a754555ea28 Event #12 - Compact header timestamp (27-bit): 0x04ecc38c Event #13 - Extended header timestamp (64-bit): 0x00098a75369cc5e4 As you can see, Event[11].Timestamp + Event[12].Timestamp > Event[13].Timestamp. Which means it cannot be a simple “offset from previous event”, since we would be getting events out of order. Here is the relevant metadata: clock { name = "monotonic"; uuid = "9fb57333-ada2-4c73-a7ce-334f46437c74"; description = "Monotonic Clock"; freq = 1000000000; /* Frequency, in Hz */ /* clock value offset from Epoch is: offset * (1/freq) */ offset = 1451946230228988436; }; typealias integer { size = 27; align = 1; signed = false; map = clock.monotonic.value; } := uint27_clock_monotonic_t; typealias integer { size = 64; align = 8; signed = false; map = clock.monotonic.value; } := uint64_clock_monotonic_t; struct event_header_compact { enum : uint5_t { compact = 0 ... 30, extended = 31 } id; variant <id> { struct { uint27_clock_monotonic_t timestamp; } compact; struct { uint32_t id; uint64_clock_monotonic_t timestamp; } extended; } v; } align(8); How should we be treating these uint27_clock_monotonic_t values? Thanks! -Lee Culver CLR Reliability and Performance _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org<mailto:lttng-dev@lists.lttng.org> https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev -- Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
_______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev
