Re: [PATCH v7 6/8] x86: tsc: Always Running Timer (ART) correlated clocksource
On Mon, 22 Feb 2016 18:49:10 -0800, Andy Lutomirski <l...@amacapital.net> wrote:
>> an
>> earlier related thread Peter Zijlstra asserts that TSC adjust "had
>> better" be
>> 0.(http://lkml.iu.edu/hypermail/linux/kernel/1507.3/03734.html).
The comment more or less reinforces your description below of such a
BIOS as "crappy"
> There are three interesting cases that I can think of:
>
> 1. Crappy BIOS that sets TSC_ADJUST. As the not-so-proud owner of a
> piece of crap motherboard that actively messes with the TSC, I don't
> trust BIOS.
I have added code that read this offset MSR at initialization
time. This isn't bulletproof, but it covers the most common case where
BIOS boots with TSC adjust set to something non-zero.
> 2. Hypervisors. What if we're running as a guest with an ART-using
> NIC passed through?
I don't see a lot of use cases where virtualization would be
used. Currently this feature (ART) isn't available on server
platforms. Without explicit support, there are a lot of ways this can
go wrong. For now, I've added an extra check that disables ART on a
VM.
> 3. Hypothetical improved future kernel that politely uses TSC_ADJUST
> to keep the TSC from going backwards across suspend/resume.
This seems like a non-trivial change. Without know how this code is
going to work, it would be difficult to test the ART solution.
[Removed errant carriage return]
*** Commit message below:
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 1]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Reviewed-by: Thomas Gleixner <t...@linutronix.de>
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Tweaked to fix build issue, also reworked math for
64bit division on 32bit systems]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 58 +++
3 files changed, 61 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 7ad8c94..ff557b4 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..174c421 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3d743da..a10cff1 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -43,6 +43,11 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+static u64 art_to_tsc_offset;
+struct clocksource *art_related_clocksource;
+
/*
* Use a ring-buffer like data structure, where a writer advances the head by
* writing a new data entry and a reader advances the tail when it observes a
@@ -949,6 +954,36 @@ static struct notifier_block time_cpufreq_notifier_block =
{
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+#define ART_MIN_DENOMINATOR (1)
+
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
+ return;
+
+ cpuid(ART_CPUID_LEAF, _to
Re: [PATCH v7 6/8] x86: tsc: Always Running Timer (ART) correlated clocksource
On Mon, 22 Feb 2016 18:49:10 -0800, Andy Lutomirski wrote:
>> an
>> earlier related thread Peter Zijlstra asserts that TSC adjust "had
>> better" be
>> 0.(http://lkml.iu.edu/hypermail/linux/kernel/1507.3/03734.html).
The comment more or less reinforces your description below of such a
BIOS as "crappy"
> There are three interesting cases that I can think of:
>
> 1. Crappy BIOS that sets TSC_ADJUST. As the not-so-proud owner of a
> piece of crap motherboard that actively messes with the TSC, I don't
> trust BIOS.
I have added code that read this offset MSR at initialization
time. This isn't bulletproof, but it covers the most common case where
BIOS boots with TSC adjust set to something non-zero.
> 2. Hypervisors. What if we're running as a guest with an ART-using
> NIC passed through?
I don't see a lot of use cases where virtualization would be
used. Currently this feature (ART) isn't available on server
platforms. Without explicit support, there are a lot of ways this can
go wrong. For now, I've added an extra check that disables ART on a
VM.
> 3. Hypothetical improved future kernel that politely uses TSC_ADJUST
> to keep the TSC from going backwards across suspend/resume.
This seems like a non-trivial change. Without know how this code is
going to work, it would be difficult to test the ART solution.
[Removed errant carriage return]
*** Commit message below:
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 1]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Reviewed-by: Thomas Gleixner
Signed-off-by: Christopher S. Hall
[jstultz: Tweaked to fix build issue, also reworked math for
64bit division on 32bit systems]
Signed-off-by: John Stultz
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 58 +++
3 files changed, 61 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 7ad8c94..ff557b4 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..174c421 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3d743da..a10cff1 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -43,6 +43,11 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+static u64 art_to_tsc_offset;
+struct clocksource *art_related_clocksource;
+
/*
* Use a ring-buffer like data structure, where a writer advances the head by
* writing a new data entry and a reader advances the tail when it observes a
@@ -949,6 +954,36 @@ static struct notifier_block time_cpufreq_notifier_block =
{
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+#define ART_MIN_DENOMINATOR (1)
+
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
+ return;
+
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ /* Don't enable ART in a VM, non-s
Re: [PATCH v7 6/8] x86: tsc: Always Running Timer (ART) correlated clocksource
On Mon, 22 Feb 2016 18:49:10 -0800, Andy Lutomirski <l...@amacapital.net> wrote:
>> an
>> earlier related thread Peter Zijlstra asserts that TSC adjust "had
>> better" be
>> 0.(http://lkml.iu.edu/hypermail/linux/kernel/1507.3/03734.html).
The comment more or less reinforces your description below of such a
BIOS as "crappy"
> There are three interesting cases that I can think of:
>
> 1. Crappy BIOS that sets TSC_ADJUST. As the not-so-proud owner of a
> piece of crap motherboard that actively messes with the TSC, I don't
> trust BIOS.
I have added code that read this offset MSR at initialization
time. This isn't bulletproof, but it covers the most common case where
BIOS boots with TSC adjust set to something non-zero.
> 2. Hypervisors. What if we're running as a guest with an ART-using
> NIC passed through?
I don't see a lot of use cases where virtualization would be
used. Currently this feature (ART) isn't available on server
platforms. Without explicit support, there are a lot of ways this can
go wrong. For now, I've added an extra check that disables ART on a
VM.
> 3. Hypothetical improved future kernel that politely uses TSC_ADJUST
> to keep the TSC from going backwards across suspend/resume.
This seems like a non-trivial change. Without know how this code is
going to work, it would be difficult to test the ART solution.
*** Commit message below:
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 1]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Reviewed-by: Thomas Gleixner <t...@linutronix.de>
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Tweaked to fix build issue, also reworked math for
64bit division on 32bit systems]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 59 +++
3 files changed, 62 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 7ad8c94..ff557b4 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..174c421 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3d743da..8644b47 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -43,6 +43,11 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+static u64 art_to_tsc_offset;
+struct clocksource *art_related_clocksource;
+
/*
* Use a ring-buffer like data structure, where a writer advances the head by
* writing a new data entry and a reader advances the tail when it observes a
@@ -949,10 +954,41 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+#define ART_MIN_DENOMINATOR (1)
+
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
+ return;
+
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_t
Re: [PATCH v7 6/8] x86: tsc: Always Running Timer (ART) correlated clocksource
On Mon, 22 Feb 2016 18:49:10 -0800, Andy Lutomirski wrote:
>> an
>> earlier related thread Peter Zijlstra asserts that TSC adjust "had
>> better" be
>> 0.(http://lkml.iu.edu/hypermail/linux/kernel/1507.3/03734.html).
The comment more or less reinforces your description below of such a
BIOS as "crappy"
> There are three interesting cases that I can think of:
>
> 1. Crappy BIOS that sets TSC_ADJUST. As the not-so-proud owner of a
> piece of crap motherboard that actively messes with the TSC, I don't
> trust BIOS.
I have added code that read this offset MSR at initialization
time. This isn't bulletproof, but it covers the most common case where
BIOS boots with TSC adjust set to something non-zero.
> 2. Hypervisors. What if we're running as a guest with an ART-using
> NIC passed through?
I don't see a lot of use cases where virtualization would be
used. Currently this feature (ART) isn't available on server
platforms. Without explicit support, there are a lot of ways this can
go wrong. For now, I've added an extra check that disables ART on a
VM.
> 3. Hypothetical improved future kernel that politely uses TSC_ADJUST
> to keep the TSC from going backwards across suspend/resume.
This seems like a non-trivial change. Without know how this code is
going to work, it would be difficult to test the ART solution.
*** Commit message below:
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 1]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Reviewed-by: Thomas Gleixner
Signed-off-by: Christopher S. Hall
[jstultz: Tweaked to fix build issue, also reworked math for
64bit division on 32bit systems]
Signed-off-by: John Stultz
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 59 +++
3 files changed, 62 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 7ad8c94..ff557b4 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..174c421 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3d743da..8644b47 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -43,6 +43,11 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+static u64 art_to_tsc_offset;
+struct clocksource *art_related_clocksource;
+
/*
* Use a ring-buffer like data structure, where a writer advances the head by
* writing a new data entry and a reader advances the tail when it observes a
@@ -949,10 +954,41 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+#define ART_MIN_DENOMINATOR (1)
+
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level < ART_CPUID_LEAF)
+ return;
+
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ /* Don't enable ART in a VM, non-stop TSC required */
+
[PATCH v8 4/8] time: Add driver cross timestamp interface for higher precision time synchronization
ACKNOWLEDGMENT: cross timestamp code was developed by Thomas Gleixner
<t...@linutronix.de>. It has changed considerably and any mistakes are
mine.
The precision with which events on multiple networked systems can be
synchronized using, as an example, PTP (IEEE 1588, 802.1AS) is limited
by the precision of the cross timestamps between the system clock and
the device (timestamp) clock. Precision here is the degree of
simultaneity when capturing the cross timestamp.
Currently the PTP cross timestamp is captured in software using the
PTP device driver ioctl PTP_SYS_OFFSET. Reads of the device clock are
interleaved with reads of the realtime clock. At best, the precision
of this cross timestamp is on the order of several microseconds due to
software latencies. Sub-microsecond precision is required for
industrial control and some media applications. To achieve this level
of precision hardware supported cross timestamping is needed.
The function get_device_system_crosstimestamp() allows device drivers
to return a cross timestamp with system time properly scaled to
nanoseconds. The realtime value is needed to discipline that clock
using PTP and the monotonic raw value is used for applications that
don't require a "real" time, but need an unadjusted clock time. The
get_device_system_crosstimestamp() code calls back into the driver to
ensure that the system counter is within the current timekeeping
update interval.
Modern Intel hardware provides an Always Running Timer (ART) which is
exactly related to TSC through a known frequency ratio. The ART is
routed to devices on the system and is used to precisely and
simultaneously capture the device clock with the ART.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Reworked to remove extra structures and simplify calling]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
include/linux/timekeeping.h | 35 +++
kernel/time/timekeeping.c | 58 +
2 files changed, 93 insertions(+)
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index af220e1..75bb836 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -280,6 +280,41 @@ struct system_time_snapshot {
};
/*
+ * struct system_device_crosststamp - system/device cross-timestamp
+ * (syncronized capture)
+ * @device:Device time
+ * @sys_realtime: Realtime simultaneous with device time
+ * @sys_monoraw: Monotonic raw simultaneous with device time
+ */
+struct system_device_crosststamp {
+ ktime_t device;
+ ktime_t sys_realtime;
+ ktime_t sys_monoraw;
+};
+
+/*
+ * struct system_counterval_t - system counter value with the pointer to the
+ * corresponding clocksource
+ * @cycles:System counter value
+ * @cs:Clocksource corresponding to system counter value. Used
by
+ * timekeeping code to verify comparibility of two cycle values
+ */
+struct system_counterval_t {
+ cycle_t cycles;
+ struct clocksource *cs;
+};
+
+/*
+ * Get cross timestamp between system clock and device clock
+ */
+extern int get_device_system_crosststamp(
+ int (*get_time_fn)(ktime_t *device_time,
+ struct system_counterval_t *system_counterval,
+ void *ctx),
+ void *ctx,
+ struct system_device_crosststamp *xtstamp);
+
+/*
* Simultaneously snapshot realtime and monotonic raw clocks
*/
extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index f1a1c97..8c53398 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -908,6 +908,64 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
EXPORT_SYMBOL_GPL(ktime_get_snapshot);
/**
+ * get_device_system_crosststamp - Synchronously capture system/device
timestamp
+ * @sync_devicetime: Callback to get simultaneous device time and
+ * system counter from the device driver
+ * @xtstamp: Receives simultaneously captured system and device time
+ *
+ * Reads a timestamp from a device and correlates it to system time
+ */
+int get_device_system_crosststamp(int (*get_time_fn)
+ (ktime_t *device_time,
+ struct system_counterval_t *sys_counterval,
+ void *ctx),
+ void *ctx,
+ struct system_device_crosststamp *xtstamp)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ struct system_counterval_t system_counterval;
+ ktime_t base_raw;
+ ktime_t base_real;
+ s64 nsec_raw;
+ s64 nsec_real;
+ int ret;
+
+ do {
+ seq = r
[PATCH v8 4/8] time: Add driver cross timestamp interface for higher precision time synchronization
ACKNOWLEDGMENT: cross timestamp code was developed by Thomas Gleixner
. It has changed considerably and any mistakes are
mine.
The precision with which events on multiple networked systems can be
synchronized using, as an example, PTP (IEEE 1588, 802.1AS) is limited
by the precision of the cross timestamps between the system clock and
the device (timestamp) clock. Precision here is the degree of
simultaneity when capturing the cross timestamp.
Currently the PTP cross timestamp is captured in software using the
PTP device driver ioctl PTP_SYS_OFFSET. Reads of the device clock are
interleaved with reads of the realtime clock. At best, the precision
of this cross timestamp is on the order of several microseconds due to
software latencies. Sub-microsecond precision is required for
industrial control and some media applications. To achieve this level
of precision hardware supported cross timestamping is needed.
The function get_device_system_crosstimestamp() allows device drivers
to return a cross timestamp with system time properly scaled to
nanoseconds. The realtime value is needed to discipline that clock
using PTP and the monotonic raw value is used for applications that
don't require a "real" time, but need an unadjusted clock time. The
get_device_system_crosstimestamp() code calls back into the driver to
ensure that the system counter is within the current timekeeping
update interval.
Modern Intel hardware provides an Always Running Timer (ART) which is
exactly related to TSC through a known frequency ratio. The ART is
routed to devices on the system and is used to precisely and
simultaneously capture the device clock with the ART.
Signed-off-by: Christopher S. Hall
[jstultz: Reworked to remove extra structures and simplify calling]
Signed-off-by: John Stultz
---
include/linux/timekeeping.h | 35 +++
kernel/time/timekeeping.c | 58 +
2 files changed, 93 insertions(+)
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index af220e1..75bb836 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -280,6 +280,41 @@ struct system_time_snapshot {
};
/*
+ * struct system_device_crosststamp - system/device cross-timestamp
+ * (syncronized capture)
+ * @device:Device time
+ * @sys_realtime: Realtime simultaneous with device time
+ * @sys_monoraw: Monotonic raw simultaneous with device time
+ */
+struct system_device_crosststamp {
+ ktime_t device;
+ ktime_t sys_realtime;
+ ktime_t sys_monoraw;
+};
+
+/*
+ * struct system_counterval_t - system counter value with the pointer to the
+ * corresponding clocksource
+ * @cycles:System counter value
+ * @cs:Clocksource corresponding to system counter value. Used
by
+ * timekeeping code to verify comparibility of two cycle values
+ */
+struct system_counterval_t {
+ cycle_t cycles;
+ struct clocksource *cs;
+};
+
+/*
+ * Get cross timestamp between system clock and device clock
+ */
+extern int get_device_system_crosststamp(
+ int (*get_time_fn)(ktime_t *device_time,
+ struct system_counterval_t *system_counterval,
+ void *ctx),
+ void *ctx,
+ struct system_device_crosststamp *xtstamp);
+
+/*
* Simultaneously snapshot realtime and monotonic raw clocks
*/
extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index f1a1c97..8c53398 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -908,6 +908,64 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
EXPORT_SYMBOL_GPL(ktime_get_snapshot);
/**
+ * get_device_system_crosststamp - Synchronously capture system/device
timestamp
+ * @sync_devicetime: Callback to get simultaneous device time and
+ * system counter from the device driver
+ * @xtstamp: Receives simultaneously captured system and device time
+ *
+ * Reads a timestamp from a device and correlates it to system time
+ */
+int get_device_system_crosststamp(int (*get_time_fn)
+ (ktime_t *device_time,
+ struct system_counterval_t *sys_counterval,
+ void *ctx),
+ void *ctx,
+ struct system_device_crosststamp *xtstamp)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ struct system_counterval_t system_counterval;
+ ktime_t base_raw;
+ ktime_t base_real;
+ s64 nsec_raw;
+ s64 nsec_real;
+ int ret;
+
+ do {
+ seq = read_seqcount_begin(_core.seq);
+ /*
+* Try to synchronously capt
[PATCH v8 5/8] time: Add history to cross timestamp interface supporting slower devices
Another representative use case of time sync and the correlated
clocksource (in addition to PTP noted above) is PTP synchronized
audio.
In a streaming application, as an example, samples will be sent and/or
received by multiple devices with a presentation time that is in terms
of the PTP master clock. Synchronizing the audio output on these
devices requires correlating the audio clock with the PTP master
clock. The more precise this correlation is, the better the audio
quality (i.e. out of sync audio sounds bad).
>From an application standpoint, to correlate the PTP master clock with
the audio device clock, the system clock is used as a intermediate
timebase. The transforms such an application would perform are:
System Clock <-> Audio clock
System Clock <-> Network Device Clock [<-> PTP Master Clock]
Modern Intel platforms can perform a more accurate cross timestamp in
hardware (ART,audio device clock). The audio driver requires
ART->system time transforms -- the same as required for the network
driver. These platforms offload audio processing (including
cross-timestamps) to a DSP which to ensure uninterrupted audio
processing, communicates and response to the host only once every
millsecond. As a result is takes up to a millisecond for the DSP to
receive a request, the request is processed by the DSP, the audio
output hardware is polled for completion, the result is copied into
shared memory, and the host is notified. All of these operation occur
on a millisecond cadence. This transaction requires about 2 ms, but
under heavier workloads it may take up to 4 ms.
Adding a history allows these slow devices the option of providing an
ART value outside of the current interval. In this case, the callback
provided is an accessor function for the previously obtained counter
value. If get_system_device_crosststamp() receives a counter value
previous to cycle_last, it consults the history provided as an
argument in history_ref and interpolates the realtime and monotonic
raw system time using the provided counter value. If there are any
clock discontinuities, e.g. from calling settimeofday(), the monotonic
raw time is interpolated in the usual way, but the realtime clock time
is adjusted by scaling the monotonic raw adjustment.
When an accessor function is used a history argument *must* be
provided. The history is initialized using ktime_get_snapshot() and
must be called before the counter values are read.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
include/linux/timekeeper_internal.h | 2 +
include/linux/timekeeping.h | 5 ++
kernel/time/timekeeping.c | 173 +++-
3 files changed, 179 insertions(+), 1 deletion(-)
diff --git a/include/linux/timekeeper_internal.h
b/include/linux/timekeeper_internal.h
index 2524722..e880054 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -50,6 +50,7 @@ struct tk_read_base {
* @offs_tai: Offset clock monotonic -> clock tai
* @tai_offset:The current UTC to TAI offset in seconds
* @clock_was_set_seq: The sequence number of clock was set events
+ * @cs_was_changed_seq:The sequence number of clocksource change events
* @next_leap_ktime: CLOCK_MONOTONIC time value of a pending leap-second
* @raw_time: Monotonic raw base time in timespec64 format
* @cycle_interval:Number of clock cycles in one NTP interval
@@ -91,6 +92,7 @@ struct timekeeper {
ktime_t offs_tai;
s32 tai_offset;
unsigned intclock_was_set_seq;
+ u8 cs_was_changed_seq;
ktime_t next_leap_ktime;
struct timespec64 raw_time;
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index 75bb836..8b90d06 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -272,11 +272,15 @@ extern void ktime_get_raw_and_real_ts64(struct timespec64
*ts_raw,
* @cycles:Clocksource counter value to produce the system times
* @real: Realtime system time
* @raw: Monotonic raw system time
+ * @clock_was_set_seq: The sequence number of clock was set events
+ * @cs_was_changed_seq:The sequence number of clocksource change events
*/
struct system_time_snapshot {
cycles_tcycles;
ktime_t real;
ktime_t raw;
+ unsigned intclock_was_set_seq;
+ u8 cs_was_changed_seq;
};
/*
@@ -312,6 +316,7 @@ extern int get_device_system_crosststamp(
struct system_counterval_t *system_counterval,
void *ctx),
void *ctx,
+ struct system_time_snapshot *history,
struct system_d
[PATCH v8 6/8] x86: tsc: Always Running Timer (ART) correlated clocksource
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 2]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Tweaked to fix build issue, also reworked math for
64bit division on 32bit systems]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 49 +++
3 files changed, 52 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 7ad8c94..ff557b4 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..174c421 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3d743da..3fa0f41 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -43,6 +43,10 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+struct clocksource *art_related_clocksource;
+
/*
* Use a ring-buffer like data structure, where a writer advances the head by
* writing a new data entry and a reader advances the tail when it observes a
@@ -949,10 +953,34 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+#define ART_MIN_DENOMINATOR (1)
+
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (boot_cpu_has(X86_FEATURE_NONSTOP_TSC) &&
+ art_to_tsc_denominator >= ART_MIN_DENOMINATOR)
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ detect_art();
+
if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
return 0;
cpufreq_register_notifier(_cpufreq_notifier_block,
@@ -1071,6 +1099,25 @@ int unsynchronized_tsc(void)
return 0;
}
+/*
+ * Convert ART to TSC given numerator/denominator found in detect_art()
+ */
+struct system_counterval_t convert_art_to_tsc(cycle_t art)
+{
+ u64 tmp, res, rem;
+
+ rem = do_div(art, art_to_tsc_denominator);
+
+ res = art * art_to_tsc_numerator;
+ tmp = rem * art_to_tsc_numerator;
+
+ do_div(tmp, art_to_tsc_denominator);
+ res += tmp;
+
+ return (struct system_counterval_t) {.cs = art_related_clocksource,
+ .cycles = res};
+}
+EXPORT_SYMBOL(convert_art_to_tsc);
static void tsc_refine_calibration_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(tsc_irqwork, tsc_refine_calibration_work);
@@ -1142,6 +1189,8 @@ static void tsc_refine_calibration_work(struct
work_struct *work)
(unsigned long)tsc_khz % 1000);
out:
+ if (boot_cpu_has(X86_FEATURE_ART))
+ art_related_clocksource = _tsc;
clocksource_register_khz(_tsc, tsc_khz);
}
--
2.1.4
[PATCH v8 5/8] time: Add history to cross timestamp interface supporting slower devices
Another representative use case of time sync and the correlated
clocksource (in addition to PTP noted above) is PTP synchronized
audio.
In a streaming application, as an example, samples will be sent and/or
received by multiple devices with a presentation time that is in terms
of the PTP master clock. Synchronizing the audio output on these
devices requires correlating the audio clock with the PTP master
clock. The more precise this correlation is, the better the audio
quality (i.e. out of sync audio sounds bad).
>From an application standpoint, to correlate the PTP master clock with
the audio device clock, the system clock is used as a intermediate
timebase. The transforms such an application would perform are:
System Clock <-> Audio clock
System Clock <-> Network Device Clock [<-> PTP Master Clock]
Modern Intel platforms can perform a more accurate cross timestamp in
hardware (ART,audio device clock). The audio driver requires
ART->system time transforms -- the same as required for the network
driver. These platforms offload audio processing (including
cross-timestamps) to a DSP which to ensure uninterrupted audio
processing, communicates and response to the host only once every
millsecond. As a result is takes up to a millisecond for the DSP to
receive a request, the request is processed by the DSP, the audio
output hardware is polled for completion, the result is copied into
shared memory, and the host is notified. All of these operation occur
on a millisecond cadence. This transaction requires about 2 ms, but
under heavier workloads it may take up to 4 ms.
Adding a history allows these slow devices the option of providing an
ART value outside of the current interval. In this case, the callback
provided is an accessor function for the previously obtained counter
value. If get_system_device_crosststamp() receives a counter value
previous to cycle_last, it consults the history provided as an
argument in history_ref and interpolates the realtime and monotonic
raw system time using the provided counter value. If there are any
clock discontinuities, e.g. from calling settimeofday(), the monotonic
raw time is interpolated in the usual way, but the realtime clock time
is adjusted by scaling the monotonic raw adjustment.
When an accessor function is used a history argument *must* be
provided. The history is initialized using ktime_get_snapshot() and
must be called before the counter values are read.
Signed-off-by: Christopher S. Hall
Signed-off-by: John Stultz
---
include/linux/timekeeper_internal.h | 2 +
include/linux/timekeeping.h | 5 ++
kernel/time/timekeeping.c | 173 +++-
3 files changed, 179 insertions(+), 1 deletion(-)
diff --git a/include/linux/timekeeper_internal.h
b/include/linux/timekeeper_internal.h
index 2524722..e880054 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -50,6 +50,7 @@ struct tk_read_base {
* @offs_tai: Offset clock monotonic -> clock tai
* @tai_offset:The current UTC to TAI offset in seconds
* @clock_was_set_seq: The sequence number of clock was set events
+ * @cs_was_changed_seq:The sequence number of clocksource change events
* @next_leap_ktime: CLOCK_MONOTONIC time value of a pending leap-second
* @raw_time: Monotonic raw base time in timespec64 format
* @cycle_interval:Number of clock cycles in one NTP interval
@@ -91,6 +92,7 @@ struct timekeeper {
ktime_t offs_tai;
s32 tai_offset;
unsigned intclock_was_set_seq;
+ u8 cs_was_changed_seq;
ktime_t next_leap_ktime;
struct timespec64 raw_time;
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index 75bb836..8b90d06 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -272,11 +272,15 @@ extern void ktime_get_raw_and_real_ts64(struct timespec64
*ts_raw,
* @cycles:Clocksource counter value to produce the system times
* @real: Realtime system time
* @raw: Monotonic raw system time
+ * @clock_was_set_seq: The sequence number of clock was set events
+ * @cs_was_changed_seq:The sequence number of clocksource change events
*/
struct system_time_snapshot {
cycles_tcycles;
ktime_t real;
ktime_t raw;
+ unsigned intclock_was_set_seq;
+ u8 cs_was_changed_seq;
};
/*
@@ -312,6 +316,7 @@ extern int get_device_system_crosststamp(
struct system_counterval_t *system_counterval,
void *ctx),
void *ctx,
+ struct system_time_snapshot *history,
struct system_device_crosststamp *xtstamp);
/*
diff --git a/kernel/time/
[PATCH v8 6/8] x86: tsc: Always Running Timer (ART) correlated clocksource
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 2]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Signed-off-by: Christopher S. Hall
[jstultz: Tweaked to fix build issue, also reworked math for
64bit division on 32bit systems]
Signed-off-by: John Stultz
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 49 +++
3 files changed, 52 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 7ad8c94..ff557b4 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..174c421 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3d743da..3fa0f41 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -43,6 +43,10 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+struct clocksource *art_related_clocksource;
+
/*
* Use a ring-buffer like data structure, where a writer advances the head by
* writing a new data entry and a reader advances the tail when it observes a
@@ -949,10 +953,34 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+#define ART_MIN_DENOMINATOR (1)
+
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (boot_cpu_has(X86_FEATURE_NONSTOP_TSC) &&
+ art_to_tsc_denominator >= ART_MIN_DENOMINATOR)
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ detect_art();
+
if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
return 0;
cpufreq_register_notifier(_cpufreq_notifier_block,
@@ -1071,6 +1099,25 @@ int unsynchronized_tsc(void)
return 0;
}
+/*
+ * Convert ART to TSC given numerator/denominator found in detect_art()
+ */
+struct system_counterval_t convert_art_to_tsc(cycle_t art)
+{
+ u64 tmp, res, rem;
+
+ rem = do_div(art, art_to_tsc_denominator);
+
+ res = art * art_to_tsc_numerator;
+ tmp = rem * art_to_tsc_numerator;
+
+ do_div(tmp, art_to_tsc_denominator);
+ res += tmp;
+
+ return (struct system_counterval_t) {.cs = art_related_clocksource,
+ .cycles = res};
+}
+EXPORT_SYMBOL(convert_art_to_tsc);
static void tsc_refine_calibration_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(tsc_irqwork, tsc_refine_calibration_work);
@@ -1142,6 +1189,8 @@ static void tsc_refine_calibration_work(struct
work_struct *work)
(unsigned long)tsc_khz % 1000);
out:
+ if (boot_cpu_has(X86_FEATURE_ART))
+ art_related_clocksource = _tsc;
clocksource_register_khz(_tsc, tsc_khz);
}
--
2.1.4
[PATCH v8 8/8] net: e1000e: Adds hardware supported cross timestamp on e1000e nic
Modern Intel systems supports cross timestamping of the network device
clock and Always Running Timer (ART) in hardware. This allows the
device time and system time to be precisely correlated. The timestamp
pair is returned through e1000e_phc_get_syncdevicetime() used by
get_system_device_crosststamp(). The hardware cross-timestamp result
is made available to applications through the PTP_SYS_OFFSET_PRECISE
ioctl which calls e1000e_phc_getcrosststamp().
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Reworked to use new interface, commit message tweaks]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
drivers/net/ethernet/intel/Kconfig | 9 +++
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 85 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
4 files changed, 103 insertions(+)
diff --git a/drivers/net/ethernet/intel/Kconfig
b/drivers/net/ethernet/intel/Kconfig
index fa593dd..3772f3a 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -83,6 +83,15 @@ config E1000E
To compile this driver as a module, choose M here. The module
will be called e1000e.
+config E1000E_HWTS
+ bool "Support HW cross-timestamp on PCH devices"
+ default y
+ depends on E1000E && X86
+ ---help---
+Say Y to enable hardware supported cross-timestamping on PCH
+devices. The cross-timestamp is available through the PTP clock
+driver precise cross-timestamp ioctl (PTP_SYS_OFFSET_PRECISE).
+
config IGB
tristate "Intel(R) 82575/82576 PCI-Express Gigabit Ethernet support"
depends on PCI
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index f7c7804..0641c00 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -528,6 +528,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..e2ff3ef 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -26,6 +26,12 @@
#include "e1000.h"
+#ifdef CONFIG_E1000E_HWTS
+#include
+#include
+#include
+#endif
+
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
* @ptp: ptp clock structure
@@ -98,6 +104,78 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#ifdef CONFIG_E1000E_HWTS
+#define MAX_HW_WAIT_COUNT (3)
+
+/**
+ * e1000e_phc_get_syncdevicetime - Callback given to timekeeping code reads
system/device registers
+ * @device: current device time
+ * @system: system counter value read synchronously with device time
+ * @ctx: context provided by timekeeping code
+ *
+ * Read device and system (ART) clock simultaneously and return the corrected
+ * clock values in ns.
+ **/
+static int e1000e_phc_get_syncdevicetime(ktime_t *device,
+struct system_counterval_t *system,
+void *ctx)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)ctx;
+ struct e1000_hw *hw = >hw;
+ unsigned long flags;
+ int i;
+ u32 tsync_ctrl;
+ cycle_t dev_cycles;
+ cycle_t sys_cycles;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT)
+ return -ETIMEDOUT;
+
+ dev_cycles = er32(SYSSTMPH);
+ dev_cycles <<= 32;
+ dev_cycles |= er32(SYSSTMPL);
+ spin_lock_irqsave(>systim_lock, flags);
+ *device = ns_to_ktime(timecounter_cyc2time(>tc, dev_cycles));
+ spin_unlock_irqrestore(>systim_lock, flags);
+
+ sys_cycles = er32(PLTSTMPH);
+ sys_cycles <<= 32;
+ sys_cycles |= er32(PLTSTMPL);
+ *system = convert_art_to_tsc(sys_cycles);
+
+ return 0;
+}
+
+/**
+ * e1000e_phc_
[PATCH v8 8/8] net: e1000e: Adds hardware supported cross timestamp on e1000e nic
Modern Intel systems supports cross timestamping of the network device
clock and Always Running Timer (ART) in hardware. This allows the
device time and system time to be precisely correlated. The timestamp
pair is returned through e1000e_phc_get_syncdevicetime() used by
get_system_device_crosststamp(). The hardware cross-timestamp result
is made available to applications through the PTP_SYS_OFFSET_PRECISE
ioctl which calls e1000e_phc_getcrosststamp().
Signed-off-by: Christopher S. Hall
[jstultz: Reworked to use new interface, commit message tweaks]
Signed-off-by: John Stultz
---
drivers/net/ethernet/intel/Kconfig | 9 +++
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 85 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
4 files changed, 103 insertions(+)
diff --git a/drivers/net/ethernet/intel/Kconfig
b/drivers/net/ethernet/intel/Kconfig
index fa593dd..3772f3a 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -83,6 +83,15 @@ config E1000E
To compile this driver as a module, choose M here. The module
will be called e1000e.
+config E1000E_HWTS
+ bool "Support HW cross-timestamp on PCH devices"
+ default y
+ depends on E1000E && X86
+ ---help---
+Say Y to enable hardware supported cross-timestamping on PCH
+devices. The cross-timestamp is available through the PTP clock
+driver precise cross-timestamp ioctl (PTP_SYS_OFFSET_PRECISE).
+
config IGB
tristate "Intel(R) 82575/82576 PCI-Express Gigabit Ethernet support"
depends on PCI
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index f7c7804..0641c00 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -528,6 +528,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..e2ff3ef 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -26,6 +26,12 @@
#include "e1000.h"
+#ifdef CONFIG_E1000E_HWTS
+#include
+#include
+#include
+#endif
+
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
* @ptp: ptp clock structure
@@ -98,6 +104,78 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#ifdef CONFIG_E1000E_HWTS
+#define MAX_HW_WAIT_COUNT (3)
+
+/**
+ * e1000e_phc_get_syncdevicetime - Callback given to timekeeping code reads
system/device registers
+ * @device: current device time
+ * @system: system counter value read synchronously with device time
+ * @ctx: context provided by timekeeping code
+ *
+ * Read device and system (ART) clock simultaneously and return the corrected
+ * clock values in ns.
+ **/
+static int e1000e_phc_get_syncdevicetime(ktime_t *device,
+struct system_counterval_t *system,
+void *ctx)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)ctx;
+ struct e1000_hw *hw = >hw;
+ unsigned long flags;
+ int i;
+ u32 tsync_ctrl;
+ cycle_t dev_cycles;
+ cycle_t sys_cycles;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT)
+ return -ETIMEDOUT;
+
+ dev_cycles = er32(SYSSTMPH);
+ dev_cycles <<= 32;
+ dev_cycles |= er32(SYSSTMPL);
+ spin_lock_irqsave(>systim_lock, flags);
+ *device = ns_to_ktime(timecounter_cyc2time(>tc, dev_cycles));
+ spin_unlock_irqrestore(>systim_lock, flags);
+
+ sys_cycles = er32(PLTSTMPH);
+ sys_cycles <<= 32;
+ sys_cycles |= er32(PLTSTMPL);
+ *system = convert_art_to_tsc(sys_cycles);
+
+ return 0;
+}
+
+/**
+ * e1000e_phc_getsynctime - Reads the current system/device cross timestamp
+
[PATCH v8 7/8] ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping
Currently, network /system cross-timestamping is performed in the
PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday() and
the gettime64() callback provided by the driver. The cross-timestamp
is best effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be
significant.
The getcrosststamp() callback and corresponding PTP_SYS_OFFSET_PRECISE
ioctl allows the driver to perform this device/system correlation when
for example cross timestamp hardware is available. Modern Intel
systems can do this for onboard Ethernet controllers using the ART
counter. There is virtually zero latency between captures of the ART
and network device clock.
The capabilities ioctl (PTP_CLOCK_GETCAPS), is augmented allowing
applications to query whether or not drivers implement the
getcrosststamp callback, providing more precise cross timestamping.
Acked-by: Richard Cochran <richardcoch...@gmail.com>
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Commit subject tweaks]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 27 +++
include/linux/ptp_clock_kernel.h | 8
include/uapi/linux/ptp_clock.h | 13 -
4 files changed, 51 insertions(+), 3 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 6c6247a..d99012f 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -277,13 +277,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..579fd65 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "ptp_private.h"
@@ -120,11 +121,13 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock_caps caps;
struct ptp_clock_request req;
struct ptp_sys_offset *sysoff = NULL;
+ struct ptp_sys_offset_precise precise_offset;
struct ptp_pin_desc pd;
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
struct timespec64 ts;
+ struct system_device_crosststamp xtstamp;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +141,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getcrosststamp != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -180,6 +184,29 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
err = ops->enable(ops, , enable);
break;
+ case PTP_SYS_OFFSET_PRECISE:
+ if (!ptp->info->getcrosststamp) {
+ err = -EOPNOTSUPP;
+ break;
+ }
+ err = ptp->info->getcrosststamp(ptp->info, );
+ if (err)
+ break;
+
+ ts = ktime_to_timespec64(xtstamp.device);
+ precise_offset.device.sec = ts.tv_sec;
+ precise_offset.device.nsec = ts.tv_nsec;
+ ts = ktime_to_timespec64(xtstamp.sys_realtime);
+ precise_offset.sys_realtime.sec = ts.tv_sec;
+ precise_offset.sys_realtime.nsec = ts.tv_nsec;
+ ts = ktime_to_timespec64(xtstamp.sys_monoraw);
+ precise_offset.sys_monoraw.sec = ts.tv_sec;
+ precise_offset.sys_monoraw.nsec = ts.tv_nsec;
+ if (copy_to_user((void __user *)arg, _offset,
+sizeof(precise_offset)))
+
[PATCH v8 7/8] ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping
Currently, network /system cross-timestamping is performed in the
PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday() and
the gettime64() callback provided by the driver. The cross-timestamp
is best effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be
significant.
The getcrosststamp() callback and corresponding PTP_SYS_OFFSET_PRECISE
ioctl allows the driver to perform this device/system correlation when
for example cross timestamp hardware is available. Modern Intel
systems can do this for onboard Ethernet controllers using the ART
counter. There is virtually zero latency between captures of the ART
and network device clock.
The capabilities ioctl (PTP_CLOCK_GETCAPS), is augmented allowing
applications to query whether or not drivers implement the
getcrosststamp callback, providing more precise cross timestamping.
Acked-by: Richard Cochran
Signed-off-by: Christopher S. Hall
[jstultz: Commit subject tweaks]
Signed-off-by: John Stultz
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 27 +++
include/linux/ptp_clock_kernel.h | 8
include/uapi/linux/ptp_clock.h | 13 -
4 files changed, 51 insertions(+), 3 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 6c6247a..d99012f 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -277,13 +277,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..579fd65 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "ptp_private.h"
@@ -120,11 +121,13 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock_caps caps;
struct ptp_clock_request req;
struct ptp_sys_offset *sysoff = NULL;
+ struct ptp_sys_offset_precise precise_offset;
struct ptp_pin_desc pd;
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
struct timespec64 ts;
+ struct system_device_crosststamp xtstamp;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +141,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getcrosststamp != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -180,6 +184,29 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
err = ops->enable(ops, , enable);
break;
+ case PTP_SYS_OFFSET_PRECISE:
+ if (!ptp->info->getcrosststamp) {
+ err = -EOPNOTSUPP;
+ break;
+ }
+ err = ptp->info->getcrosststamp(ptp->info, );
+ if (err)
+ break;
+
+ ts = ktime_to_timespec64(xtstamp.device);
+ precise_offset.device.sec = ts.tv_sec;
+ precise_offset.device.nsec = ts.tv_nsec;
+ ts = ktime_to_timespec64(xtstamp.sys_realtime);
+ precise_offset.sys_realtime.sec = ts.tv_sec;
+ precise_offset.sys_realtime.nsec = ts.tv_nsec;
+ ts = ktime_to_timespec64(xtstamp.sys_monoraw);
+ precise_offset.sys_monoraw.sec = ts.tv_sec;
+ precise_offset.sys_monoraw.nsec = ts.tv_nsec;
+ if (copy_to_user((void __user *)arg, _offset,
+sizeof(precise_offset)))
+ err = -EFAULT;
+ break;
+
case PTP_SYS_OFFSET:
sysoff = kmalloc(s
[PATCH v8 2/8] time: Add timekeeping snapshot code capturing system time and counter
In the current timekeeping code there isn't any interface to
atomically capture the current relationship between the system counter
and system time. ktime_get_snapshot() returns this triple (counter,
monotonic raw, realtime) in the system_time_snapshot struct.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Moved structure definitions around to clean things up]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
include/linux/timekeeping.h | 18 ++
kernel/time/timekeeping.c | 30 ++
2 files changed, 48 insertions(+)
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index ec89d84..af220e1 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -267,6 +267,24 @@ extern void ktime_get_raw_and_real_ts64(struct timespec64
*ts_raw,
struct timespec64 *ts_real);
/*
+ * struct system_time_snapshot - simultaneous raw/real time capture with
+ * counter value
+ * @cycles:Clocksource counter value to produce the system times
+ * @real: Realtime system time
+ * @raw: Monotonic raw system time
+ */
+struct system_time_snapshot {
+ cycles_tcycles;
+ ktime_t real;
+ ktime_t raw;
+};
+
+/*
+ * Simultaneously snapshot realtime and monotonic raw clocks
+ */
+extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
+
+/*
* Persistent clock related interfaces
*/
extern int persistent_clock_is_local;
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 4243d28..89b4695 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -874,6 +874,36 @@ time64_t __ktime_get_real_seconds(void)
return tk->xtime_sec;
}
+/**
+ * ktime_get_snapshot - snapshots the realtime/monotonic raw clocks with
counter
+ * @systime_snapshot: pointer to struct receiving the system time snapshot
+ */
+void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ ktime_t base_raw;
+ ktime_t base_real;
+ s64 nsec_raw;
+ s64 nsec_real;
+ cycle_t now;
+
+ do {
+ seq = read_seqcount_begin(_core.seq);
+
+ now = tk->tkr_mono.read(tk->tkr_mono.clock);
+ base_real = ktime_add(tk->tkr_mono.base,
+ tk_core.timekeeper.offs_real);
+ base_raw = tk->tkr_raw.base;
+ nsec_real = timekeeping_cycles_to_ns(>tkr_mono, now);
+ nsec_raw = timekeeping_cycles_to_ns(>tkr_raw, now);
+ } while (read_seqcount_retry(_core.seq, seq));
+
+ systime_snapshot->cycles = now;
+ systime_snapshot->real = ktime_add_ns(base_real, nsec_real);
+ systime_snapshot->raw = ktime_add_ns(base_raw, nsec_raw);
+}
+EXPORT_SYMBOL_GPL(ktime_get_snapshot);
#ifdef CONFIG_NTP_PPS
--
2.1.4
[PATCH v8 1/8] time: Add cycles to nanoseconds translation
The timekeeping code does not currently provide a way to translate
externally provided clocksource cycles to system time. The cycle count
is always provided by the result clocksource read() method internal to
the timekeeping code. The added function timekeeping_cycles_to_ns()
calculated a nanosecond value from a cycle count that can be added to
tk_read_base.base value yielding the current system time. This allows
clocksource cycle values external to the timekeeping code to provide a
cycle count that can be transformed to system time.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
kernel/time/timekeeping.c | 25 +
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 34b4ced..4243d28 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -298,17 +298,34 @@ u32 (*arch_gettimeoffset)(void) =
default_arch_gettimeoffset;
static inline u32 arch_gettimeoffset(void) { return 0; }
#endif
+static inline s64 timekeeping_delta_to_ns(struct tk_read_base *tkr,
+ cycle_t delta)
+{
+ s64 nsec;
+
+ nsec = delta * tkr->mult + tkr->xtime_nsec;
+ nsec >>= tkr->shift;
+
+ /* If arch requires, add in get_arch_timeoffset() */
+ return nsec + arch_gettimeoffset();
+}
+
static inline s64 timekeeping_get_ns(struct tk_read_base *tkr)
{
cycle_t delta;
- s64 nsec;
delta = timekeeping_get_delta(tkr);
+ return timekeeping_delta_to_ns(tkr, delta);
+}
- nsec = (delta * tkr->mult + tkr->xtime_nsec) >> tkr->shift;
+static inline s64 timekeeping_cycles_to_ns(struct tk_read_base *tkr,
+ cycle_t cycles)
+{
+ cycle_t delta;
- /* If arch requires, add in get_arch_timeoffset() */
- return nsec + arch_gettimeoffset();
+ /* calculate the delta since the last update_wall_time */
+ delta = clocksource_delta(cycles, tkr->cycle_last, tkr->mask);
+ return timekeeping_delta_to_ns(tkr, delta);
}
/**
--
2.1.4
[PATCH v8 2/8] time: Add timekeeping snapshot code capturing system time and counter
In the current timekeeping code there isn't any interface to
atomically capture the current relationship between the system counter
and system time. ktime_get_snapshot() returns this triple (counter,
monotonic raw, realtime) in the system_time_snapshot struct.
Signed-off-by: Christopher S. Hall
[jstultz: Moved structure definitions around to clean things up]
Signed-off-by: John Stultz
---
include/linux/timekeeping.h | 18 ++
kernel/time/timekeeping.c | 30 ++
2 files changed, 48 insertions(+)
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index ec89d84..af220e1 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -267,6 +267,24 @@ extern void ktime_get_raw_and_real_ts64(struct timespec64
*ts_raw,
struct timespec64 *ts_real);
/*
+ * struct system_time_snapshot - simultaneous raw/real time capture with
+ * counter value
+ * @cycles:Clocksource counter value to produce the system times
+ * @real: Realtime system time
+ * @raw: Monotonic raw system time
+ */
+struct system_time_snapshot {
+ cycles_tcycles;
+ ktime_t real;
+ ktime_t raw;
+};
+
+/*
+ * Simultaneously snapshot realtime and monotonic raw clocks
+ */
+extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
+
+/*
* Persistent clock related interfaces
*/
extern int persistent_clock_is_local;
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 4243d28..89b4695 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -874,6 +874,36 @@ time64_t __ktime_get_real_seconds(void)
return tk->xtime_sec;
}
+/**
+ * ktime_get_snapshot - snapshots the realtime/monotonic raw clocks with
counter
+ * @systime_snapshot: pointer to struct receiving the system time snapshot
+ */
+void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ ktime_t base_raw;
+ ktime_t base_real;
+ s64 nsec_raw;
+ s64 nsec_real;
+ cycle_t now;
+
+ do {
+ seq = read_seqcount_begin(_core.seq);
+
+ now = tk->tkr_mono.read(tk->tkr_mono.clock);
+ base_real = ktime_add(tk->tkr_mono.base,
+ tk_core.timekeeper.offs_real);
+ base_raw = tk->tkr_raw.base;
+ nsec_real = timekeeping_cycles_to_ns(>tkr_mono, now);
+ nsec_raw = timekeeping_cycles_to_ns(>tkr_raw, now);
+ } while (read_seqcount_retry(_core.seq, seq));
+
+ systime_snapshot->cycles = now;
+ systime_snapshot->real = ktime_add_ns(base_real, nsec_real);
+ systime_snapshot->raw = ktime_add_ns(base_raw, nsec_raw);
+}
+EXPORT_SYMBOL_GPL(ktime_get_snapshot);
#ifdef CONFIG_NTP_PPS
--
2.1.4
[PATCH v8 1/8] time: Add cycles to nanoseconds translation
The timekeeping code does not currently provide a way to translate
externally provided clocksource cycles to system time. The cycle count
is always provided by the result clocksource read() method internal to
the timekeeping code. The added function timekeeping_cycles_to_ns()
calculated a nanosecond value from a cycle count that can be added to
tk_read_base.base value yielding the current system time. This allows
clocksource cycle values external to the timekeeping code to provide a
cycle count that can be transformed to system time.
Signed-off-by: Christopher S. Hall
Signed-off-by: John Stultz
---
kernel/time/timekeeping.c | 25 +
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 34b4ced..4243d28 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -298,17 +298,34 @@ u32 (*arch_gettimeoffset)(void) =
default_arch_gettimeoffset;
static inline u32 arch_gettimeoffset(void) { return 0; }
#endif
+static inline s64 timekeeping_delta_to_ns(struct tk_read_base *tkr,
+ cycle_t delta)
+{
+ s64 nsec;
+
+ nsec = delta * tkr->mult + tkr->xtime_nsec;
+ nsec >>= tkr->shift;
+
+ /* If arch requires, add in get_arch_timeoffset() */
+ return nsec + arch_gettimeoffset();
+}
+
static inline s64 timekeeping_get_ns(struct tk_read_base *tkr)
{
cycle_t delta;
- s64 nsec;
delta = timekeeping_get_delta(tkr);
+ return timekeeping_delta_to_ns(tkr, delta);
+}
- nsec = (delta * tkr->mult + tkr->xtime_nsec) >> tkr->shift;
+static inline s64 timekeeping_cycles_to_ns(struct tk_read_base *tkr,
+ cycle_t cycles)
+{
+ cycle_t delta;
- /* If arch requires, add in get_arch_timeoffset() */
- return nsec + arch_gettimeoffset();
+ /* calculate the delta since the last update_wall_time */
+ delta = clocksource_delta(cycles, tkr->cycle_last, tkr->mask);
+ return timekeeping_delta_to_ns(tkr, delta);
}
/**
--
2.1.4
[PATCH v8 0/8] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
Modern Intel hardware adds an Always Running Timer (ART) that allows the network and audio device clocks to precisely cross timestamp the device clock with the system clock. This allows a precise correlation of the device time and system time. This patchset adds interfaces to the timekeeping code allowing drivers to translate ART time to system time. Changelog: Changes from v7 to v8: * Fix comments in timekeeping.c * Re-use older TSC feature, removing "invariant" TSC in ART detection code Changes from v6 to v7: * Reorder several patches * Removed correlated clocksource * Fixed 32-bit compile issues * Added multiplication overflow detection to history computation * Added invariant tsc CPU feature - this is related to ART, but is a separate feature Changes from v5 to v6: * Pulled supporting code for snapshotting, correlated clocksource, and cycles to nanoseconds translation to separate patches. Added patches are marked as NEW below. There is, however, very little *actually* new code, just reorganized code * Renamed and moved clocksource change sequence to timekeeper struct (out of tk_read_base) * Renamed structs for system counter and synced device time callback to system_counterval_t and sync_device_time_cb, respectively * Changed PTP cross-timestamp callback name to getcrosststamp for consistency with the timekeeping code - corresponding function name changes in e1000e driver * Simplified PTP time calculations making use of ktime_to_* code Changes from v4 to v5: * Changes the history mechanism to interpolate system time using a single historic system time pair (monotonic raw, realtime) rather than implementing a precise history using shadow timekeeper (see v4 changes). The advantage of this approach is that the history can be arbitrarily long. This approach may also be simpler in terms of coding. The major disadvantage is that the realtime clock can be adjusted. When adjusted, the realtime clock time (when interpolating from history) is always approximate. In general, the longer the interpolation period the larger the potential error. There isn't any error interpolating the monotonic raw clock time. * This patchset also addresses objections to the previous patchsets overly complex correlated timestamp structure. This patchset splits that structure into several smaller structures. The correlated timestamp interface is renamed cross timestamp to avoid any confusion with the correlated clocksource. * The correlated clocksource is separated from the cross timestamp mechanism. * Add monotonic raw to the PTP user interface * Add e1000e driver configuration option that wraps Intel PCH specific code Changes v3 to v4: * Adds a history mechanism to accomodate slower devices. In this case the response time for timestamp reads to the Intel DSP are too slow to be accomodated by the original correlated time mechanism. The history mechanism turns shadow timekeeper into an array where the history is stored. Christopher S. Hall (8): time: Add cycles to nanoseconds translation time: Add timekeeping snapshot code capturing system time and counter time: Remove duplicated code in ktime_get_raw_and_real() time: Add driver cross timestamp interface for higher precision time synchronization time: Add history to cross timestamp interface supporting slower devices x86: tsc: Always Running Timer (ART) correlated clocksource ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping net: e1000e: Adds hardware supported cross timestamp on e1000e nic Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 2 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/tsc.c | 49 + drivers/net/ethernet/intel/Kconfig | 9 + drivers/net/ethernet/intel/e1000e/defines.h | 5 + drivers/net/ethernet/intel/e1000e/ptp.c | 85 drivers/net/ethernet/intel/e1000e/regs.h| 4 + drivers/ptp/ptp_chardev.c | 27 +++ include/linux/pps_kernel.h | 17 +- include/linux/ptp_clock_kernel.h| 8 + include/linux/timekeeper_internal.h | 2 + include/linux/timekeeping.h | 58 ++ include/uapi/linux/ptp_clock.h | 13 +- kernel/time/timekeeping.c | 290 +--- 15 files changed, 537 insertions(+), 40 deletions(-) -- 2.1.4
[PATCH v8 0/8] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
Modern Intel hardware adds an Always Running Timer (ART) that allows the network and audio device clocks to precisely cross timestamp the device clock with the system clock. This allows a precise correlation of the device time and system time. This patchset adds interfaces to the timekeeping code allowing drivers to translate ART time to system time. Changelog: Changes from v7 to v8: * Fix comments in timekeeping.c * Re-use older TSC feature, removing "invariant" TSC in ART detection code Changes from v6 to v7: * Reorder several patches * Removed correlated clocksource * Fixed 32-bit compile issues * Added multiplication overflow detection to history computation * Added invariant tsc CPU feature - this is related to ART, but is a separate feature Changes from v5 to v6: * Pulled supporting code for snapshotting, correlated clocksource, and cycles to nanoseconds translation to separate patches. Added patches are marked as NEW below. There is, however, very little *actually* new code, just reorganized code * Renamed and moved clocksource change sequence to timekeeper struct (out of tk_read_base) * Renamed structs for system counter and synced device time callback to system_counterval_t and sync_device_time_cb, respectively * Changed PTP cross-timestamp callback name to getcrosststamp for consistency with the timekeeping code - corresponding function name changes in e1000e driver * Simplified PTP time calculations making use of ktime_to_* code Changes from v4 to v5: * Changes the history mechanism to interpolate system time using a single historic system time pair (monotonic raw, realtime) rather than implementing a precise history using shadow timekeeper (see v4 changes). The advantage of this approach is that the history can be arbitrarily long. This approach may also be simpler in terms of coding. The major disadvantage is that the realtime clock can be adjusted. When adjusted, the realtime clock time (when interpolating from history) is always approximate. In general, the longer the interpolation period the larger the potential error. There isn't any error interpolating the monotonic raw clock time. * This patchset also addresses objections to the previous patchsets overly complex correlated timestamp structure. This patchset splits that structure into several smaller structures. The correlated timestamp interface is renamed cross timestamp to avoid any confusion with the correlated clocksource. * The correlated clocksource is separated from the cross timestamp mechanism. * Add monotonic raw to the PTP user interface * Add e1000e driver configuration option that wraps Intel PCH specific code Changes v3 to v4: * Adds a history mechanism to accomodate slower devices. In this case the response time for timestamp reads to the Intel DSP are too slow to be accomodated by the original correlated time mechanism. The history mechanism turns shadow timekeeper into an array where the history is stored. Christopher S. Hall (8): time: Add cycles to nanoseconds translation time: Add timekeeping snapshot code capturing system time and counter time: Remove duplicated code in ktime_get_raw_and_real() time: Add driver cross timestamp interface for higher precision time synchronization time: Add history to cross timestamp interface supporting slower devices x86: tsc: Always Running Timer (ART) correlated clocksource ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping net: e1000e: Adds hardware supported cross timestamp on e1000e nic Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 2 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/tsc.c | 49 + drivers/net/ethernet/intel/Kconfig | 9 + drivers/net/ethernet/intel/e1000e/defines.h | 5 + drivers/net/ethernet/intel/e1000e/ptp.c | 85 drivers/net/ethernet/intel/e1000e/regs.h| 4 + drivers/ptp/ptp_chardev.c | 27 +++ include/linux/pps_kernel.h | 17 +- include/linux/ptp_clock_kernel.h| 8 + include/linux/timekeeper_internal.h | 2 + include/linux/timekeeping.h | 58 ++ include/uapi/linux/ptp_clock.h | 13 +- kernel/time/timekeeping.c | 290 +--- 15 files changed, 537 insertions(+), 40 deletions(-) -- 2.1.4
[PATCH v8 3/8] time: Remove duplicated code in ktime_get_raw_and_real()
The code in ktime_get_snapshot() is a superset of the code in
ktime_get_raw_and_real() code. Further, ktime_get_raw_and_real() is
called only by the PPS code, pps_get_ts(). Consolidate the
pps_get_ts() code into a single function calling ktime_get_snapshot()
and eliminate ktime_get_raw_and_real(). A side effect of this is that
the raw and real results of pps_get_ts() correspond to exactly the
same clock cycle. Previously these values represented separate reads
of the system clock.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
include/linux/pps_kernel.h | 17 ++---
kernel/time/timekeeping.c | 40 ++--
2 files changed, 8 insertions(+), 49 deletions(-)
diff --git a/include/linux/pps_kernel.h b/include/linux/pps_kernel.h
index 54bf148..35ac903 100644
--- a/include/linux/pps_kernel.h
+++ b/include/linux/pps_kernel.h
@@ -111,22 +111,17 @@ static inline void timespec_to_pps_ktime(struct pps_ktime
*kt,
kt->nsec = ts.tv_nsec;
}
-#ifdef CONFIG_NTP_PPS
-
static inline void pps_get_ts(struct pps_event_time *ts)
{
- ktime_get_raw_and_real_ts64(>ts_raw, >ts_real);
-}
+ struct system_time_snapshot snap;
-#else /* CONFIG_NTP_PPS */
-
-static inline void pps_get_ts(struct pps_event_time *ts)
-{
- ktime_get_real_ts64(>ts_real);
+ ktime_get_snapshot();
+ ts->ts_real = ktime_to_timespec64(snap.real);
+#ifdef CONFIG_NTP_PPS
+ ts->ts_raw = ktime_to_timespec64(snap.raw);
+#endif
}
-#endif /* CONFIG_NTP_PPS */
-
/* Subtract known time delay from PPS event time(s) */
static inline void pps_sub_ts(struct pps_event_time *ts, struct timespec64
delta)
{
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 89b4695..f1a1c97 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -888,6 +888,8 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
s64 nsec_real;
cycle_t now;
+ WARN_ON(timekeeping_suspended);
+
do {
seq = read_seqcount_begin(_core.seq);
@@ -905,44 +907,6 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
}
EXPORT_SYMBOL_GPL(ktime_get_snapshot);
-#ifdef CONFIG_NTP_PPS
-
-/**
- * ktime_get_raw_and_real_ts64 - get day and raw monotonic time in timespec
format
- * @ts_raw:pointer to the timespec to be set to raw monotonic time
- * @ts_real: pointer to the timespec to be set to the time of day
- *
- * This function reads both the time of day and raw monotonic time at the
- * same time atomically and stores the resulting timestamps in timespec
- * format.
- */
-void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw, struct timespec64
*ts_real)
-{
- struct timekeeper *tk = _core.timekeeper;
- unsigned long seq;
- s64 nsecs_raw, nsecs_real;
-
- WARN_ON_ONCE(timekeeping_suspended);
-
- do {
- seq = read_seqcount_begin(_core.seq);
-
- *ts_raw = tk->raw_time;
- ts_real->tv_sec = tk->xtime_sec;
- ts_real->tv_nsec = 0;
-
- nsecs_raw = timekeeping_get_ns(>tkr_raw);
- nsecs_real = timekeeping_get_ns(>tkr_mono);
-
- } while (read_seqcount_retry(_core.seq, seq));
-
- timespec64_add_ns(ts_raw, nsecs_raw);
- timespec64_add_ns(ts_real, nsecs_real);
-}
-EXPORT_SYMBOL(ktime_get_raw_and_real_ts64);
-
-#endif /* CONFIG_NTP_PPS */
-
/**
* do_gettimeofday - Returns the time of day in a timeval
* @tv:pointer to the timeval to be set
--
2.1.4
[PATCH v8 3/8] time: Remove duplicated code in ktime_get_raw_and_real()
The code in ktime_get_snapshot() is a superset of the code in
ktime_get_raw_and_real() code. Further, ktime_get_raw_and_real() is
called only by the PPS code, pps_get_ts(). Consolidate the
pps_get_ts() code into a single function calling ktime_get_snapshot()
and eliminate ktime_get_raw_and_real(). A side effect of this is that
the raw and real results of pps_get_ts() correspond to exactly the
same clock cycle. Previously these values represented separate reads
of the system clock.
Signed-off-by: Christopher S. Hall
Signed-off-by: John Stultz
---
include/linux/pps_kernel.h | 17 ++---
kernel/time/timekeeping.c | 40 ++--
2 files changed, 8 insertions(+), 49 deletions(-)
diff --git a/include/linux/pps_kernel.h b/include/linux/pps_kernel.h
index 54bf148..35ac903 100644
--- a/include/linux/pps_kernel.h
+++ b/include/linux/pps_kernel.h
@@ -111,22 +111,17 @@ static inline void timespec_to_pps_ktime(struct pps_ktime
*kt,
kt->nsec = ts.tv_nsec;
}
-#ifdef CONFIG_NTP_PPS
-
static inline void pps_get_ts(struct pps_event_time *ts)
{
- ktime_get_raw_and_real_ts64(>ts_raw, >ts_real);
-}
+ struct system_time_snapshot snap;
-#else /* CONFIG_NTP_PPS */
-
-static inline void pps_get_ts(struct pps_event_time *ts)
-{
- ktime_get_real_ts64(>ts_real);
+ ktime_get_snapshot();
+ ts->ts_real = ktime_to_timespec64(snap.real);
+#ifdef CONFIG_NTP_PPS
+ ts->ts_raw = ktime_to_timespec64(snap.raw);
+#endif
}
-#endif /* CONFIG_NTP_PPS */
-
/* Subtract known time delay from PPS event time(s) */
static inline void pps_sub_ts(struct pps_event_time *ts, struct timespec64
delta)
{
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 89b4695..f1a1c97 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -888,6 +888,8 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
s64 nsec_real;
cycle_t now;
+ WARN_ON(timekeeping_suspended);
+
do {
seq = read_seqcount_begin(_core.seq);
@@ -905,44 +907,6 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
}
EXPORT_SYMBOL_GPL(ktime_get_snapshot);
-#ifdef CONFIG_NTP_PPS
-
-/**
- * ktime_get_raw_and_real_ts64 - get day and raw monotonic time in timespec
format
- * @ts_raw:pointer to the timespec to be set to raw monotonic time
- * @ts_real: pointer to the timespec to be set to the time of day
- *
- * This function reads both the time of day and raw monotonic time at the
- * same time atomically and stores the resulting timestamps in timespec
- * format.
- */
-void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw, struct timespec64
*ts_real)
-{
- struct timekeeper *tk = _core.timekeeper;
- unsigned long seq;
- s64 nsecs_raw, nsecs_real;
-
- WARN_ON_ONCE(timekeeping_suspended);
-
- do {
- seq = read_seqcount_begin(_core.seq);
-
- *ts_raw = tk->raw_time;
- ts_real->tv_sec = tk->xtime_sec;
- ts_real->tv_nsec = 0;
-
- nsecs_raw = timekeeping_get_ns(>tkr_raw);
- nsecs_real = timekeeping_get_ns(>tkr_mono);
-
- } while (read_seqcount_retry(_core.seq, seq));
-
- timespec64_add_ns(ts_raw, nsecs_raw);
- timespec64_add_ns(ts_real, nsecs_real);
-}
-EXPORT_SYMBOL(ktime_get_raw_and_real_ts64);
-
-#endif /* CONFIG_NTP_PPS */
-
/**
* do_gettimeofday - Returns the time of day in a timeval
* @tv:pointer to the timeval to be set
--
2.1.4
[PATCH v7 0/8] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
Modern Intel hardware adds an Always Running Timer (ART) that allows the network and audio device clocks to precisely cross timestamp the device clock with the system clock. This allows a precise correlation of the device time and system time. This patchset adds interfaces to the timekeeping code allowing drivers to translate ART time to system time. Changelog: Changes from v6 to v7: * Reorder several patches * Removed correlated clocksource * Fixed 32-bit compile issues * Added multiplication overflow detection to history computation * Added invariant tsc CPU feature - this is related to ART, but is a separate feature Changes from v5 to v6: * Pulled supporting code for snapshotting, correlated clocksource, and cycles to nanoseconds translation to separate patches. Added patches are marked as NEW below. There is, however, very little *actually* new code, just reorganized code * Renamed and moved clocksource change sequence to timekeeper struct (out of tk_read_base) * Renamed structs for system counter and synced device time callback to system_counterval_t and sync_device_time_cb, respectively * Changed PTP cross-timestamp callback name to getcrosststamp for consistency with the timekeeping code - corresponding function name changes in e1000e driver * Simplified PTP time calculations making use of ktime_to_* code Changes from v4 to v5: * Changes the history mechanism to interpolate system time using a single historic system time pair (monotonic raw, realtime) rather than implementing a precise history using shadow timekeeper (see v4 changes). The advantage of this approach is that the history can be arbitrarily long. This approach may also be simpler in terms of coding. The major disadvantage is that the realtime clock can be adjusted. When adjusted, the realtime clock time (when interpolating from history) is always approximate. In general, the longer the interpolation period the larger the potential error. There isn't any error interpolating the monotonic raw clock time. * This patchset also addresses objections to the previous patchsets overly complex correlated timestamp structure. This patchset splits that structure into several smaller structures. The correlated timestamp interface is renamed cross timestamp to avoid any confusion with the correlated clocksource. * The correlated clocksource is separated from the cross timestamp mechanism. * Add monotonic raw to the PTP user interface * Add e1000e driver configuration option that wraps Intel PCH specific code Changes v3 to v4: * Adds a history mechanism to accomodate slower devices. In this case the response time for timestamp reads to the Intel DSP are too slow to be accomodated by the original correlated time mechanism. The history mechanism turns shadow timekeeper into an array where the history is stored. Christopher S. Hall (8): time: Add cycles to nanoseconds translation time: Add timekeeping snapshot code capturing system time and counter time: Remove duplicated code in ktime_get_raw_and_real() time: Add driver cross timestamp interface for higher precision time synchronization time: Add history to cross timestamp interface supporting slower devices x86: tsc: Always Running Timer (ART) correlated clocksource ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping net: e1000e: Adds hardware supported cross timestamp on e1000e nic Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 3 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/cpu/scattered.c | 1 + arch/x86/kernel/tsc.c | 50 + drivers/net/ethernet/intel/Kconfig | 9 + drivers/net/ethernet/intel/e1000e/defines.h | 5 + drivers/net/ethernet/intel/e1000e/ptp.c | 85 drivers/net/ethernet/intel/e1000e/regs.h| 4 + drivers/ptp/ptp_chardev.c | 27 +++ include/linux/pps_kernel.h | 17 +- include/linux/ptp_clock_kernel.h| 8 + include/linux/timekeeper_internal.h | 2 + include/linux/timekeeping.h | 58 ++ include/uapi/linux/ptp_clock.h | 13 +- kernel/time/timekeeping.c | 289 +--- 16 files changed, 539 insertions(+), 40 deletions(-) -- 2.1.4
[PATCH v7 2/8] time: Add timekeeping snapshot code capturing system time and counter
In the current timekeeping code there isn't any interface to
atomically capture the current relationship between the system counter
and system time. ktime_get_snapshot() returns this triple (counter,
monotonic raw, realtime) in the system_time_snapshot struct.
Signed-off-by: Christopher S. Hall
[jstultz: Moved structure definitions around to clean things up]
Signed-off-by: John Stultz
---
include/linux/timekeeping.h | 18 ++
kernel/time/timekeeping.c | 30 ++
2 files changed, 48 insertions(+)
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index ec89d84..af220e1 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -267,6 +267,24 @@ extern void ktime_get_raw_and_real_ts64(struct timespec64
*ts_raw,
struct timespec64 *ts_real);
/*
+ * struct system_time_snapshot - simultaneous raw/real time capture with
+ * counter value
+ * @cycles:Clocksource counter value to produce the system times
+ * @real: Realtime system time
+ * @raw: Monotonic raw system time
+ */
+struct system_time_snapshot {
+ cycles_tcycles;
+ ktime_t real;
+ ktime_t raw;
+};
+
+/*
+ * Simultaneously snapshot realtime and monotonic raw clocks
+ */
+extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
+
+/*
* Persistent clock related interfaces
*/
extern int persistent_clock_is_local;
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 4243d28..89b4695 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -874,6 +874,36 @@ time64_t __ktime_get_real_seconds(void)
return tk->xtime_sec;
}
+/**
+ * ktime_get_snapshot - snapshots the realtime/monotonic raw clocks with
counter
+ * @systime_snapshot: pointer to struct receiving the system time snapshot
+ */
+void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ ktime_t base_raw;
+ ktime_t base_real;
+ s64 nsec_raw;
+ s64 nsec_real;
+ cycle_t now;
+
+ do {
+ seq = read_seqcount_begin(_core.seq);
+
+ now = tk->tkr_mono.read(tk->tkr_mono.clock);
+ base_real = ktime_add(tk->tkr_mono.base,
+ tk_core.timekeeper.offs_real);
+ base_raw = tk->tkr_raw.base;
+ nsec_real = timekeeping_cycles_to_ns(>tkr_mono, now);
+ nsec_raw = timekeeping_cycles_to_ns(>tkr_raw, now);
+ } while (read_seqcount_retry(_core.seq, seq));
+
+ systime_snapshot->cycles = now;
+ systime_snapshot->real = ktime_add_ns(base_real, nsec_real);
+ systime_snapshot->raw = ktime_add_ns(base_raw, nsec_raw);
+}
+EXPORT_SYMBOL_GPL(ktime_get_snapshot);
#ifdef CONFIG_NTP_PPS
--
2.1.4
[PATCH v7 4/8] time: Add driver cross timestamp interface for higher precision time synchronization
ACKNOWLEDGMENT: cross timestamp code was developed by Thomas Gleixner
. It has changed considerably and any mistakes are
mine.
The precision with which events on multiple networked systems can be
synchronized using, as an example, PTP (IEEE 1588, 802.1AS) is limited
by the precision of the cross timestamps between the system clock and
the device (timestamp) clock. Precision here is the degree of
simultaneity when capturing the cross timestamp.
Currently the PTP cross timestamp is captured in software using the
PTP device driver ioctl PTP_SYS_OFFSET. Reads of the device clock are
interleaved with reads of the realtime clock. At best, the precision
of this cross timestamp is on the order of several microseconds due to
software latencies. Sub-microsecond precision is required for
industrial control and some media applications. To achieve this level
of precision hardware supported cross timestamping is needed.
The function get_device_system_crosstimestamp() allows device drivers
to return a cross timestamp with system time properly scaled to
nanoseconds. The realtime value is needed to discipline that clock
using PTP and the monotonic raw value is used for applications that
don't require a "real" time, but need an unadjusted clock time. The
get_device_system_crosstimestamp() code calls back into the driver to
ensure that the system counter is within the current timekeeping
update interval.
Modern Intel hardware provides an Always Running Timer (ART) which is
exactly related to TSC through a known frequency ratio. The ART is
routed to devices on the system and is used to precisely and
simultaneously capture the device clock with the ART.
Signed-off-by: Christopher S. Hall
[jstultz: Reworked to remove extra structures and simplify calling]
Signed-off-by: John Stultz
---
include/linux/timekeeping.h | 35 +++
kernel/time/timekeeping.c | 58 +
2 files changed, 93 insertions(+)
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index af220e1..75bb836 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -280,6 +280,41 @@ struct system_time_snapshot {
};
/*
+ * struct system_device_crosststamp - system/device cross-timestamp
+ * (syncronized capture)
+ * @device:Device time
+ * @sys_realtime: Realtime simultaneous with device time
+ * @sys_monoraw: Monotonic raw simultaneous with device time
+ */
+struct system_device_crosststamp {
+ ktime_t device;
+ ktime_t sys_realtime;
+ ktime_t sys_monoraw;
+};
+
+/*
+ * struct system_counterval_t - system counter value with the pointer to the
+ * corresponding clocksource
+ * @cycles:System counter value
+ * @cs:Clocksource corresponding to system counter value. Used
by
+ * timekeeping code to verify comparibility of two cycle values
+ */
+struct system_counterval_t {
+ cycle_t cycles;
+ struct clocksource *cs;
+};
+
+/*
+ * Get cross timestamp between system clock and device clock
+ */
+extern int get_device_system_crosststamp(
+ int (*get_time_fn)(ktime_t *device_time,
+ struct system_counterval_t *system_counterval,
+ void *ctx),
+ void *ctx,
+ struct system_device_crosststamp *xtstamp);
+
+/*
* Simultaneously snapshot realtime and monotonic raw clocks
*/
extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index f1a1c97..8c53398 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -908,6 +908,64 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
EXPORT_SYMBOL_GPL(ktime_get_snapshot);
/**
+ * get_device_system_crosststamp - Synchronously capture system/device
timestamp
+ * @sync_devicetime: Callback to get simultaneous device time and
+ * system counter from the device driver
+ * @xtstamp: Receives simultaneously captured system and device time
+ *
+ * Reads a timestamp from a device and correlates it to system time
+ */
+int get_device_system_crosststamp(int (*get_time_fn)
+ (ktime_t *device_time,
+ struct system_counterval_t *sys_counterval,
+ void *ctx),
+ void *ctx,
+ struct system_device_crosststamp *xtstamp)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ struct system_counterval_t system_counterval;
+ ktime_t base_raw;
+ ktime_t base_real;
+ s64 nsec_raw;
+ s64 nsec_real;
+ int ret;
+
+ do {
+ seq = read_seqcount_begin(_core.seq);
+ /*
+* Try to synchronously capt
[PATCH v7 6/8] x86: tsc: Always Running Timer (ART) correlated clocksource
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 2]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Signed-off-by: Christopher S. Hall
[jstultz: Tweaked to fix build issue, also reworked math for
64bit division on 32bit systems]
Signed-off-by: John Stultz
---
arch/x86/include/asm/cpufeature.h | 3 ++-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/cpu/scattered.c | 1 +
arch/x86/kernel/tsc.c | 50 +++
4 files changed, 55 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 7ad8c94..111b892 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
@@ -188,6 +188,7 @@
#define X86_FEATURE_CPB( 7*32+ 2) /* AMD Core Performance
Boost */
#define X86_FEATURE_EPB( 7*32+ 3) /* IA32_ENERGY_PERF_BIAS
support */
+#define X86_FEATURE_INVARIANT_TSC (7*32+4) /* Intel Invariant TSC */
#define X86_FEATURE_HW_PSTATE ( 7*32+ 8) /* AMD HW-PState */
#define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..174c421 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 8cb57df..af0ecd7 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -35,6 +35,7 @@ void init_scattered_cpuid_features(struct cpuinfo_x86 *c)
{ X86_FEATURE_APERFMPERF, CR_ECX, 0, 0x0006, 0 },
{ X86_FEATURE_EPB, CR_ECX, 3, 0x0006, 0 },
{ X86_FEATURE_HW_PSTATE,CR_EDX, 7, 0x8007, 0 },
+ { X86_FEATURE_INVARIANT_TSC,CR_EDX, 8, 0x8007, 0 },
{ X86_FEATURE_CPB, CR_EDX, 9, 0x8007, 0 },
{ X86_FEATURE_PROC_FEEDBACK,CR_EDX,11, 0x8007, 0 },
{ 0, 0, 0, 0, 0 }
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3d743da..0ee3b62 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -43,6 +43,10 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+struct clocksource *art_related_clocksource;
+
/*
* Use a ring-buffer like data structure, where a writer advances the head by
* writing a new data entry and a reader advances the tail when it observes a
@@ -949,10 +953,35 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+/* The denominator will never be less that 2 */
+#define ART_MIN_DENOMINATOR (2)
+
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (boot_cpu_has(X86_FEATURE_INVARIANT_TSC) &&
+ art_to_tsc_denominator >= ART_MIN_DENOMINATOR)
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ det
[PATCH v7 5/8] time: Add history to cross timestamp interface supporting slower devices
Another representative use case of time sync and the correlated
clocksource (in addition to PTP noted above) is PTP synchronized
audio.
In a streaming application, as an example, samples will be sent and/or
received by multiple devices with a presentation time that is in terms
of the PTP master clock. Synchronizing the audio output on these
devices requires correlating the audio clock with the PTP master
clock. The more precise this correlation is, the better the audio
quality (i.e. out of sync audio sounds bad).
>From an application standpoint, to correlate the PTP master clock with
the audio device clock, the system clock is used as a intermediate
timebase. The transforms such an application would perform are:
System Clock <-> Audio clock
System Clock <-> Network Device Clock [<-> PTP Master Clock]
Modern Intel platforms can perform a more accurate cross timestamp in
hardware (ART,audio device clock). The audio driver requires
ART->system time transforms -- the same as required for the network
driver. These platforms offload audio processing (including
cross-timestamps) to a DSP which to ensure uninterrupted audio
processing, communicates and response to the host only once every
millsecond. As a result is takes up to a millisecond for the DSP to
receive a request, the request is processed by the DSP, the audio
output hardware is polled for completion, the result is copied into
shared memory, and the host is notified. All of these operation occur
on a millisecond cadence. This transaction requires about 2 ms, but
under heavier workloads it may take up to 4 ms.
Adding a history allows these slow devices the option of providing an
ART value outside of the current interval. In this case, the callback
provided is an accessor function for the previously obtained counter
value. If get_system_device_crosststamp() receives a counter value
previous to cycle_last, it consults the history provided as an
argument in history_ref and interpolates the realtime and monotonic
raw system time using the provided counter value. If there are any
clock discontinuities, e.g. from calling settimeofday(), the monotonic
raw time is interpolated in the usual way, but the realtime clock time
is adjusted by scaling the monotonic raw adjustment.
When an accessor function is used a history argument *must* be
provided. The history is initialized using ktime_get_snapshot() and
must be called before the counter values are read.
Signed-off-by: Christopher S. Hall
Signed-off-by: John Stultz
---
include/linux/timekeeper_internal.h | 2 +
include/linux/timekeeping.h | 5 ++
kernel/time/timekeeping.c | 172 +++-
3 files changed, 178 insertions(+), 1 deletion(-)
diff --git a/include/linux/timekeeper_internal.h
b/include/linux/timekeeper_internal.h
index 2524722..e880054 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -50,6 +50,7 @@ struct tk_read_base {
* @offs_tai: Offset clock monotonic -> clock tai
* @tai_offset:The current UTC to TAI offset in seconds
* @clock_was_set_seq: The sequence number of clock was set events
+ * @cs_was_changed_seq:The sequence number of clocksource change events
* @next_leap_ktime: CLOCK_MONOTONIC time value of a pending leap-second
* @raw_time: Monotonic raw base time in timespec64 format
* @cycle_interval:Number of clock cycles in one NTP interval
@@ -91,6 +92,7 @@ struct timekeeper {
ktime_t offs_tai;
s32 tai_offset;
unsigned intclock_was_set_seq;
+ u8 cs_was_changed_seq;
ktime_t next_leap_ktime;
struct timespec64 raw_time;
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index 75bb836..8b90d06 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -272,11 +272,15 @@ extern void ktime_get_raw_and_real_ts64(struct timespec64
*ts_raw,
* @cycles:Clocksource counter value to produce the system times
* @real: Realtime system time
* @raw: Monotonic raw system time
+ * @clock_was_set_seq: The sequence number of clock was set events
+ * @cs_was_changed_seq:The sequence number of clocksource change events
*/
struct system_time_snapshot {
cycles_tcycles;
ktime_t real;
ktime_t raw;
+ unsigned intclock_was_set_seq;
+ u8 cs_was_changed_seq;
};
/*
@@ -312,6 +316,7 @@ extern int get_device_system_crosststamp(
struct system_counterval_t *system_counterval,
void *ctx),
void *ctx,
+ struct system_time_snapshot *history,
struct system_device_crosststamp *xtstamp);
/*
diff --git a/kernel/time/
[PATCH v7 7/8] ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping
Currently, network /system cross-timestamping is performed in the
PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday() and
the gettime64() callback provided by the driver. The cross-timestamp
is best effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be
significant.
The getcrosststamp() callback and corresponding PTP_SYS_OFFSET_PRECISE
ioctl allows the driver to perform this device/system correlation when
for example cross timestamp hardware is available. Modern Intel
systems can do this for onboard Ethernet controllers using the ART
counter. There is virtually zero latency between captures of the ART
and network device clock.
The capabilities ioctl (PTP_CLOCK_GETCAPS), is augmented allowing
applications to query whether or not drivers implement the
getcrosststamp callback, providing more precise cross timestamping.
Acked-by: Richard Cochran
Signed-off-by: Christopher S. Hall
[jstultz: Commit subject tweaks]
Signed-off-by: John Stultz
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 27 +++
include/linux/ptp_clock_kernel.h | 8
include/uapi/linux/ptp_clock.h | 13 -
4 files changed, 51 insertions(+), 3 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 6c6247a..d99012f 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -277,13 +277,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..579fd65 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "ptp_private.h"
@@ -120,11 +121,13 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock_caps caps;
struct ptp_clock_request req;
struct ptp_sys_offset *sysoff = NULL;
+ struct ptp_sys_offset_precise precise_offset;
struct ptp_pin_desc pd;
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
struct timespec64 ts;
+ struct system_device_crosststamp xtstamp;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +141,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getcrosststamp != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -180,6 +184,29 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
err = ops->enable(ops, , enable);
break;
+ case PTP_SYS_OFFSET_PRECISE:
+ if (!ptp->info->getcrosststamp) {
+ err = -EOPNOTSUPP;
+ break;
+ }
+ err = ptp->info->getcrosststamp(ptp->info, );
+ if (err)
+ break;
+
+ ts = ktime_to_timespec64(xtstamp.device);
+ precise_offset.device.sec = ts.tv_sec;
+ precise_offset.device.nsec = ts.tv_nsec;
+ ts = ktime_to_timespec64(xtstamp.sys_realtime);
+ precise_offset.sys_realtime.sec = ts.tv_sec;
+ precise_offset.sys_realtime.nsec = ts.tv_nsec;
+ ts = ktime_to_timespec64(xtstamp.sys_monoraw);
+ precise_offset.sys_monoraw.sec = ts.tv_sec;
+ precise_offset.sys_monoraw.nsec = ts.tv_nsec;
+ if (copy_to_user((void __user *)arg, _offset,
+sizeof(precise_offset)))
+ err = -EFAULT;
+ break;
+
case PTP_SYS_OFFSET:
sysoff = kmalloc(s
[PATCH v7 8/8] net: e1000e: Adds hardware supported cross timestamp on e1000e nic
Modern Intel systems supports cross timestamping of the network device
clock and Always Running Timer (ART) in hardware. This allows the
device time and system time to be precisely correlated. The timestamp
pair is returned through e1000e_phc_get_syncdevicetime() used by
get_system_device_crosststamp(). The hardware cross-timestamp result
is made available to applications through the PTP_SYS_OFFSET_PRECISE
ioctl which calls e1000e_phc_getcrosststamp().
Signed-off-by: Christopher S. Hall
[jstultz: Reworked to use new interface, commit message tweaks]
Signed-off-by: John Stultz
---
drivers/net/ethernet/intel/Kconfig | 9 +++
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 85 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
4 files changed, 103 insertions(+)
diff --git a/drivers/net/ethernet/intel/Kconfig
b/drivers/net/ethernet/intel/Kconfig
index fa593dd..3772f3a 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -83,6 +83,15 @@ config E1000E
To compile this driver as a module, choose M here. The module
will be called e1000e.
+config E1000E_HWTS
+ bool "Support HW cross-timestamp on PCH devices"
+ default y
+ depends on E1000E && X86
+ ---help---
+Say Y to enable hardware supported cross-timestamping on PCH
+devices. The cross-timestamp is available through the PTP clock
+driver precise cross-timestamp ioctl (PTP_SYS_OFFSET_PRECISE).
+
config IGB
tristate "Intel(R) 82575/82576 PCI-Express Gigabit Ethernet support"
depends on PCI
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index f7c7804..0641c00 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -528,6 +528,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..e2ff3ef 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -26,6 +26,12 @@
#include "e1000.h"
+#ifdef CONFIG_E1000E_HWTS
+#include
+#include
+#include
+#endif
+
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
* @ptp: ptp clock structure
@@ -98,6 +104,78 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#ifdef CONFIG_E1000E_HWTS
+#define MAX_HW_WAIT_COUNT (3)
+
+/**
+ * e1000e_phc_get_syncdevicetime - Callback given to timekeeping code reads
system/device registers
+ * @device: current device time
+ * @system: system counter value read synchronously with device time
+ * @ctx: context provided by timekeeping code
+ *
+ * Read device and system (ART) clock simultaneously and return the corrected
+ * clock values in ns.
+ **/
+static int e1000e_phc_get_syncdevicetime(ktime_t *device,
+struct system_counterval_t *system,
+void *ctx)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)ctx;
+ struct e1000_hw *hw = >hw;
+ unsigned long flags;
+ int i;
+ u32 tsync_ctrl;
+ cycle_t dev_cycles;
+ cycle_t sys_cycles;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT)
+ return -ETIMEDOUT;
+
+ dev_cycles = er32(SYSSTMPH);
+ dev_cycles <<= 32;
+ dev_cycles |= er32(SYSSTMPL);
+ spin_lock_irqsave(>systim_lock, flags);
+ *device = ns_to_ktime(timecounter_cyc2time(>tc, dev_cycles));
+ spin_unlock_irqrestore(>systim_lock, flags);
+
+ sys_cycles = er32(PLTSTMPH);
+ sys_cycles <<= 32;
+ sys_cycles |= er32(PLTSTMPL);
+ *system = convert_art_to_tsc(sys_cycles);
+
+ return 0;
+}
+
+/**
+ * e1000e_phc_getsynctime - Reads the current system/device cross timestamp
+
[PATCH v7 1/8] time: Add cycles to nanoseconds translation
The timekeeping code does not currently provide a way to translate
externally provided clocksource cycles to system time. The cycle count
is always provided by the result clocksource read() method internal to
the timekeeping code. The added function timekeeping_cycles_to_ns()
calculated a nanosecond value from a cycle count that can be added to
tk_read_base.base value yielding the current system time. This allows
clocksource cycle values external to the timekeeping code to provide a
cycle count that can be transformed to system time.
Signed-off-by: Christopher S. Hall
Signed-off-by: John Stultz
---
kernel/time/timekeeping.c | 25 +
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 34b4ced..4243d28 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -298,17 +298,34 @@ u32 (*arch_gettimeoffset)(void) =
default_arch_gettimeoffset;
static inline u32 arch_gettimeoffset(void) { return 0; }
#endif
+static inline s64 timekeeping_delta_to_ns(struct tk_read_base *tkr,
+ cycle_t delta)
+{
+ s64 nsec;
+
+ nsec = delta * tkr->mult + tkr->xtime_nsec;
+ nsec >>= tkr->shift;
+
+ /* If arch requires, add in get_arch_timeoffset() */
+ return nsec + arch_gettimeoffset();
+}
+
static inline s64 timekeeping_get_ns(struct tk_read_base *tkr)
{
cycle_t delta;
- s64 nsec;
delta = timekeeping_get_delta(tkr);
+ return timekeeping_delta_to_ns(tkr, delta);
+}
- nsec = (delta * tkr->mult + tkr->xtime_nsec) >> tkr->shift;
+static inline s64 timekeeping_cycles_to_ns(struct tk_read_base *tkr,
+ cycle_t cycles)
+{
+ cycle_t delta;
- /* If arch requires, add in get_arch_timeoffset() */
- return nsec + arch_gettimeoffset();
+ /* calculate the delta since the last update_wall_time */
+ delta = clocksource_delta(cycles, tkr->cycle_last, tkr->mask);
+ return timekeeping_delta_to_ns(tkr, delta);
}
/**
--
2.1.4
[PATCH v7 3/8] time: Remove duplicated code in ktime_get_raw_and_real()
The code in ktime_get_snapshot() is a superset of the code in
ktime_get_raw_and_real() code. Further, ktime_get_raw_and_real() is
called only by the PPS code, pps_get_ts(). Consolidate the
pps_get_ts() code into a single function calling ktime_get_snapshot()
and eliminate ktime_get_raw_and_real(). A side effect of this is that
the raw and real results of pps_get_ts() correspond to exactly the
same clock cycle. Previously these values represented separate reads
of the system clock.
Signed-off-by: Christopher S. Hall
Signed-off-by: John Stultz
---
include/linux/pps_kernel.h | 17 ++---
kernel/time/timekeeping.c | 40 ++--
2 files changed, 8 insertions(+), 49 deletions(-)
diff --git a/include/linux/pps_kernel.h b/include/linux/pps_kernel.h
index 54bf148..35ac903 100644
--- a/include/linux/pps_kernel.h
+++ b/include/linux/pps_kernel.h
@@ -111,22 +111,17 @@ static inline void timespec_to_pps_ktime(struct pps_ktime
*kt,
kt->nsec = ts.tv_nsec;
}
-#ifdef CONFIG_NTP_PPS
-
static inline void pps_get_ts(struct pps_event_time *ts)
{
- ktime_get_raw_and_real_ts64(>ts_raw, >ts_real);
-}
+ struct system_time_snapshot snap;
-#else /* CONFIG_NTP_PPS */
-
-static inline void pps_get_ts(struct pps_event_time *ts)
-{
- ktime_get_real_ts64(>ts_real);
+ ktime_get_snapshot();
+ ts->ts_real = ktime_to_timespec64(snap.real);
+#ifdef CONFIG_NTP_PPS
+ ts->ts_raw = ktime_to_timespec64(snap.raw);
+#endif
}
-#endif /* CONFIG_NTP_PPS */
-
/* Subtract known time delay from PPS event time(s) */
static inline void pps_sub_ts(struct pps_event_time *ts, struct timespec64
delta)
{
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 89b4695..f1a1c97 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -888,6 +888,8 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
s64 nsec_real;
cycle_t now;
+ WARN_ON(timekeeping_suspended);
+
do {
seq = read_seqcount_begin(_core.seq);
@@ -905,44 +907,6 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
}
EXPORT_SYMBOL_GPL(ktime_get_snapshot);
-#ifdef CONFIG_NTP_PPS
-
-/**
- * ktime_get_raw_and_real_ts64 - get day and raw monotonic time in timespec
format
- * @ts_raw:pointer to the timespec to be set to raw monotonic time
- * @ts_real: pointer to the timespec to be set to the time of day
- *
- * This function reads both the time of day and raw monotonic time at the
- * same time atomically and stores the resulting timestamps in timespec
- * format.
- */
-void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw, struct timespec64
*ts_real)
-{
- struct timekeeper *tk = _core.timekeeper;
- unsigned long seq;
- s64 nsecs_raw, nsecs_real;
-
- WARN_ON_ONCE(timekeeping_suspended);
-
- do {
- seq = read_seqcount_begin(_core.seq);
-
- *ts_raw = tk->raw_time;
- ts_real->tv_sec = tk->xtime_sec;
- ts_real->tv_nsec = 0;
-
- nsecs_raw = timekeeping_get_ns(>tkr_raw);
- nsecs_real = timekeeping_get_ns(>tkr_mono);
-
- } while (read_seqcount_retry(_core.seq, seq));
-
- timespec64_add_ns(ts_raw, nsecs_raw);
- timespec64_add_ns(ts_real, nsecs_real);
-}
-EXPORT_SYMBOL(ktime_get_raw_and_real_ts64);
-
-#endif /* CONFIG_NTP_PPS */
-
/**
* do_gettimeofday - Returns the time of day in a timeval
* @tv:pointer to the timeval to be set
--
2.1.4
[PATCH v7 3/8] time: Remove duplicated code in ktime_get_raw_and_real()
The code in ktime_get_snapshot() is a superset of the code in
ktime_get_raw_and_real() code. Further, ktime_get_raw_and_real() is
called only by the PPS code, pps_get_ts(). Consolidate the
pps_get_ts() code into a single function calling ktime_get_snapshot()
and eliminate ktime_get_raw_and_real(). A side effect of this is that
the raw and real results of pps_get_ts() correspond to exactly the
same clock cycle. Previously these values represented separate reads
of the system clock.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
include/linux/pps_kernel.h | 17 ++---
kernel/time/timekeeping.c | 40 ++--
2 files changed, 8 insertions(+), 49 deletions(-)
diff --git a/include/linux/pps_kernel.h b/include/linux/pps_kernel.h
index 54bf148..35ac903 100644
--- a/include/linux/pps_kernel.h
+++ b/include/linux/pps_kernel.h
@@ -111,22 +111,17 @@ static inline void timespec_to_pps_ktime(struct pps_ktime
*kt,
kt->nsec = ts.tv_nsec;
}
-#ifdef CONFIG_NTP_PPS
-
static inline void pps_get_ts(struct pps_event_time *ts)
{
- ktime_get_raw_and_real_ts64(>ts_raw, >ts_real);
-}
+ struct system_time_snapshot snap;
-#else /* CONFIG_NTP_PPS */
-
-static inline void pps_get_ts(struct pps_event_time *ts)
-{
- ktime_get_real_ts64(>ts_real);
+ ktime_get_snapshot();
+ ts->ts_real = ktime_to_timespec64(snap.real);
+#ifdef CONFIG_NTP_PPS
+ ts->ts_raw = ktime_to_timespec64(snap.raw);
+#endif
}
-#endif /* CONFIG_NTP_PPS */
-
/* Subtract known time delay from PPS event time(s) */
static inline void pps_sub_ts(struct pps_event_time *ts, struct timespec64
delta)
{
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 89b4695..f1a1c97 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -888,6 +888,8 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
s64 nsec_real;
cycle_t now;
+ WARN_ON(timekeeping_suspended);
+
do {
seq = read_seqcount_begin(_core.seq);
@@ -905,44 +907,6 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
}
EXPORT_SYMBOL_GPL(ktime_get_snapshot);
-#ifdef CONFIG_NTP_PPS
-
-/**
- * ktime_get_raw_and_real_ts64 - get day and raw monotonic time in timespec
format
- * @ts_raw:pointer to the timespec to be set to raw monotonic time
- * @ts_real: pointer to the timespec to be set to the time of day
- *
- * This function reads both the time of day and raw monotonic time at the
- * same time atomically and stores the resulting timestamps in timespec
- * format.
- */
-void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw, struct timespec64
*ts_real)
-{
- struct timekeeper *tk = _core.timekeeper;
- unsigned long seq;
- s64 nsecs_raw, nsecs_real;
-
- WARN_ON_ONCE(timekeeping_suspended);
-
- do {
- seq = read_seqcount_begin(_core.seq);
-
- *ts_raw = tk->raw_time;
- ts_real->tv_sec = tk->xtime_sec;
- ts_real->tv_nsec = 0;
-
- nsecs_raw = timekeeping_get_ns(>tkr_raw);
- nsecs_real = timekeeping_get_ns(>tkr_mono);
-
- } while (read_seqcount_retry(_core.seq, seq));
-
- timespec64_add_ns(ts_raw, nsecs_raw);
- timespec64_add_ns(ts_real, nsecs_real);
-}
-EXPORT_SYMBOL(ktime_get_raw_and_real_ts64);
-
-#endif /* CONFIG_NTP_PPS */
-
/**
* do_gettimeofday - Returns the time of day in a timeval
* @tv:pointer to the timeval to be set
--
2.1.4
[PATCH v7 8/8] net: e1000e: Adds hardware supported cross timestamp on e1000e nic
Modern Intel systems supports cross timestamping of the network device
clock and Always Running Timer (ART) in hardware. This allows the
device time and system time to be precisely correlated. The timestamp
pair is returned through e1000e_phc_get_syncdevicetime() used by
get_system_device_crosststamp(). The hardware cross-timestamp result
is made available to applications through the PTP_SYS_OFFSET_PRECISE
ioctl which calls e1000e_phc_getcrosststamp().
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Reworked to use new interface, commit message tweaks]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
drivers/net/ethernet/intel/Kconfig | 9 +++
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 85 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
4 files changed, 103 insertions(+)
diff --git a/drivers/net/ethernet/intel/Kconfig
b/drivers/net/ethernet/intel/Kconfig
index fa593dd..3772f3a 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -83,6 +83,15 @@ config E1000E
To compile this driver as a module, choose M here. The module
will be called e1000e.
+config E1000E_HWTS
+ bool "Support HW cross-timestamp on PCH devices"
+ default y
+ depends on E1000E && X86
+ ---help---
+Say Y to enable hardware supported cross-timestamping on PCH
+devices. The cross-timestamp is available through the PTP clock
+driver precise cross-timestamp ioctl (PTP_SYS_OFFSET_PRECISE).
+
config IGB
tristate "Intel(R) 82575/82576 PCI-Express Gigabit Ethernet support"
depends on PCI
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index f7c7804..0641c00 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -528,6 +528,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..e2ff3ef 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -26,6 +26,12 @@
#include "e1000.h"
+#ifdef CONFIG_E1000E_HWTS
+#include
+#include
+#include
+#endif
+
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
* @ptp: ptp clock structure
@@ -98,6 +104,78 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#ifdef CONFIG_E1000E_HWTS
+#define MAX_HW_WAIT_COUNT (3)
+
+/**
+ * e1000e_phc_get_syncdevicetime - Callback given to timekeeping code reads
system/device registers
+ * @device: current device time
+ * @system: system counter value read synchronously with device time
+ * @ctx: context provided by timekeeping code
+ *
+ * Read device and system (ART) clock simultaneously and return the corrected
+ * clock values in ns.
+ **/
+static int e1000e_phc_get_syncdevicetime(ktime_t *device,
+struct system_counterval_t *system,
+void *ctx)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)ctx;
+ struct e1000_hw *hw = >hw;
+ unsigned long flags;
+ int i;
+ u32 tsync_ctrl;
+ cycle_t dev_cycles;
+ cycle_t sys_cycles;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT)
+ return -ETIMEDOUT;
+
+ dev_cycles = er32(SYSSTMPH);
+ dev_cycles <<= 32;
+ dev_cycles |= er32(SYSSTMPL);
+ spin_lock_irqsave(>systim_lock, flags);
+ *device = ns_to_ktime(timecounter_cyc2time(>tc, dev_cycles));
+ spin_unlock_irqrestore(>systim_lock, flags);
+
+ sys_cycles = er32(PLTSTMPH);
+ sys_cycles <<= 32;
+ sys_cycles |= er32(PLTSTMPL);
+ *system = convert_art_to_tsc(sys_cycles);
+
+ return 0;
+}
+
+/**
+ * e1000e_phc_
[PATCH v7 1/8] time: Add cycles to nanoseconds translation
The timekeeping code does not currently provide a way to translate
externally provided clocksource cycles to system time. The cycle count
is always provided by the result clocksource read() method internal to
the timekeeping code. The added function timekeeping_cycles_to_ns()
calculated a nanosecond value from a cycle count that can be added to
tk_read_base.base value yielding the current system time. This allows
clocksource cycle values external to the timekeeping code to provide a
cycle count that can be transformed to system time.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
kernel/time/timekeeping.c | 25 +
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 34b4ced..4243d28 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -298,17 +298,34 @@ u32 (*arch_gettimeoffset)(void) =
default_arch_gettimeoffset;
static inline u32 arch_gettimeoffset(void) { return 0; }
#endif
+static inline s64 timekeeping_delta_to_ns(struct tk_read_base *tkr,
+ cycle_t delta)
+{
+ s64 nsec;
+
+ nsec = delta * tkr->mult + tkr->xtime_nsec;
+ nsec >>= tkr->shift;
+
+ /* If arch requires, add in get_arch_timeoffset() */
+ return nsec + arch_gettimeoffset();
+}
+
static inline s64 timekeeping_get_ns(struct tk_read_base *tkr)
{
cycle_t delta;
- s64 nsec;
delta = timekeeping_get_delta(tkr);
+ return timekeeping_delta_to_ns(tkr, delta);
+}
- nsec = (delta * tkr->mult + tkr->xtime_nsec) >> tkr->shift;
+static inline s64 timekeeping_cycles_to_ns(struct tk_read_base *tkr,
+ cycle_t cycles)
+{
+ cycle_t delta;
- /* If arch requires, add in get_arch_timeoffset() */
- return nsec + arch_gettimeoffset();
+ /* calculate the delta since the last update_wall_time */
+ delta = clocksource_delta(cycles, tkr->cycle_last, tkr->mask);
+ return timekeeping_delta_to_ns(tkr, delta);
}
/**
--
2.1.4
[PATCH v7 0/8] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
Modern Intel hardware adds an Always Running Timer (ART) that allows the network and audio device clocks to precisely cross timestamp the device clock with the system clock. This allows a precise correlation of the device time and system time. This patchset adds interfaces to the timekeeping code allowing drivers to translate ART time to system time. Changelog: Changes from v6 to v7: * Reorder several patches * Removed correlated clocksource * Fixed 32-bit compile issues * Added multiplication overflow detection to history computation * Added invariant tsc CPU feature - this is related to ART, but is a separate feature Changes from v5 to v6: * Pulled supporting code for snapshotting, correlated clocksource, and cycles to nanoseconds translation to separate patches. Added patches are marked as NEW below. There is, however, very little *actually* new code, just reorganized code * Renamed and moved clocksource change sequence to timekeeper struct (out of tk_read_base) * Renamed structs for system counter and synced device time callback to system_counterval_t and sync_device_time_cb, respectively * Changed PTP cross-timestamp callback name to getcrosststamp for consistency with the timekeeping code - corresponding function name changes in e1000e driver * Simplified PTP time calculations making use of ktime_to_* code Changes from v4 to v5: * Changes the history mechanism to interpolate system time using a single historic system time pair (monotonic raw, realtime) rather than implementing a precise history using shadow timekeeper (see v4 changes). The advantage of this approach is that the history can be arbitrarily long. This approach may also be simpler in terms of coding. The major disadvantage is that the realtime clock can be adjusted. When adjusted, the realtime clock time (when interpolating from history) is always approximate. In general, the longer the interpolation period the larger the potential error. There isn't any error interpolating the monotonic raw clock time. * This patchset also addresses objections to the previous patchsets overly complex correlated timestamp structure. This patchset splits that structure into several smaller structures. The correlated timestamp interface is renamed cross timestamp to avoid any confusion with the correlated clocksource. * The correlated clocksource is separated from the cross timestamp mechanism. * Add monotonic raw to the PTP user interface * Add e1000e driver configuration option that wraps Intel PCH specific code Changes v3 to v4: * Adds a history mechanism to accomodate slower devices. In this case the response time for timestamp reads to the Intel DSP are too slow to be accomodated by the original correlated time mechanism. The history mechanism turns shadow timekeeper into an array where the history is stored. Christopher S. Hall (8): time: Add cycles to nanoseconds translation time: Add timekeeping snapshot code capturing system time and counter time: Remove duplicated code in ktime_get_raw_and_real() time: Add driver cross timestamp interface for higher precision time synchronization time: Add history to cross timestamp interface supporting slower devices x86: tsc: Always Running Timer (ART) correlated clocksource ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping net: e1000e: Adds hardware supported cross timestamp on e1000e nic Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 3 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/cpu/scattered.c | 1 + arch/x86/kernel/tsc.c | 50 + drivers/net/ethernet/intel/Kconfig | 9 + drivers/net/ethernet/intel/e1000e/defines.h | 5 + drivers/net/ethernet/intel/e1000e/ptp.c | 85 drivers/net/ethernet/intel/e1000e/regs.h| 4 + drivers/ptp/ptp_chardev.c | 27 +++ include/linux/pps_kernel.h | 17 +- include/linux/ptp_clock_kernel.h| 8 + include/linux/timekeeper_internal.h | 2 + include/linux/timekeeping.h | 58 ++ include/uapi/linux/ptp_clock.h | 13 +- kernel/time/timekeeping.c | 289 +--- 16 files changed, 539 insertions(+), 40 deletions(-) -- 2.1.4
[PATCH v7 2/8] time: Add timekeeping snapshot code capturing system time and counter
In the current timekeeping code there isn't any interface to
atomically capture the current relationship between the system counter
and system time. ktime_get_snapshot() returns this triple (counter,
monotonic raw, realtime) in the system_time_snapshot struct.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Moved structure definitions around to clean things up]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
include/linux/timekeeping.h | 18 ++
kernel/time/timekeeping.c | 30 ++
2 files changed, 48 insertions(+)
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index ec89d84..af220e1 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -267,6 +267,24 @@ extern void ktime_get_raw_and_real_ts64(struct timespec64
*ts_raw,
struct timespec64 *ts_real);
/*
+ * struct system_time_snapshot - simultaneous raw/real time capture with
+ * counter value
+ * @cycles:Clocksource counter value to produce the system times
+ * @real: Realtime system time
+ * @raw: Monotonic raw system time
+ */
+struct system_time_snapshot {
+ cycles_tcycles;
+ ktime_t real;
+ ktime_t raw;
+};
+
+/*
+ * Simultaneously snapshot realtime and monotonic raw clocks
+ */
+extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
+
+/*
* Persistent clock related interfaces
*/
extern int persistent_clock_is_local;
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 4243d28..89b4695 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -874,6 +874,36 @@ time64_t __ktime_get_real_seconds(void)
return tk->xtime_sec;
}
+/**
+ * ktime_get_snapshot - snapshots the realtime/monotonic raw clocks with
counter
+ * @systime_snapshot: pointer to struct receiving the system time snapshot
+ */
+void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ ktime_t base_raw;
+ ktime_t base_real;
+ s64 nsec_raw;
+ s64 nsec_real;
+ cycle_t now;
+
+ do {
+ seq = read_seqcount_begin(_core.seq);
+
+ now = tk->tkr_mono.read(tk->tkr_mono.clock);
+ base_real = ktime_add(tk->tkr_mono.base,
+ tk_core.timekeeper.offs_real);
+ base_raw = tk->tkr_raw.base;
+ nsec_real = timekeeping_cycles_to_ns(>tkr_mono, now);
+ nsec_raw = timekeeping_cycles_to_ns(>tkr_raw, now);
+ } while (read_seqcount_retry(_core.seq, seq));
+
+ systime_snapshot->cycles = now;
+ systime_snapshot->real = ktime_add_ns(base_real, nsec_real);
+ systime_snapshot->raw = ktime_add_ns(base_raw, nsec_raw);
+}
+EXPORT_SYMBOL_GPL(ktime_get_snapshot);
#ifdef CONFIG_NTP_PPS
--
2.1.4
[PATCH v7 4/8] time: Add driver cross timestamp interface for higher precision time synchronization
ACKNOWLEDGMENT: cross timestamp code was developed by Thomas Gleixner
<t...@linutronix.de>. It has changed considerably and any mistakes are
mine.
The precision with which events on multiple networked systems can be
synchronized using, as an example, PTP (IEEE 1588, 802.1AS) is limited
by the precision of the cross timestamps between the system clock and
the device (timestamp) clock. Precision here is the degree of
simultaneity when capturing the cross timestamp.
Currently the PTP cross timestamp is captured in software using the
PTP device driver ioctl PTP_SYS_OFFSET. Reads of the device clock are
interleaved with reads of the realtime clock. At best, the precision
of this cross timestamp is on the order of several microseconds due to
software latencies. Sub-microsecond precision is required for
industrial control and some media applications. To achieve this level
of precision hardware supported cross timestamping is needed.
The function get_device_system_crosstimestamp() allows device drivers
to return a cross timestamp with system time properly scaled to
nanoseconds. The realtime value is needed to discipline that clock
using PTP and the monotonic raw value is used for applications that
don't require a "real" time, but need an unadjusted clock time. The
get_device_system_crosstimestamp() code calls back into the driver to
ensure that the system counter is within the current timekeeping
update interval.
Modern Intel hardware provides an Always Running Timer (ART) which is
exactly related to TSC through a known frequency ratio. The ART is
routed to devices on the system and is used to precisely and
simultaneously capture the device clock with the ART.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Reworked to remove extra structures and simplify calling]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
include/linux/timekeeping.h | 35 +++
kernel/time/timekeeping.c | 58 +
2 files changed, 93 insertions(+)
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index af220e1..75bb836 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -280,6 +280,41 @@ struct system_time_snapshot {
};
/*
+ * struct system_device_crosststamp - system/device cross-timestamp
+ * (syncronized capture)
+ * @device:Device time
+ * @sys_realtime: Realtime simultaneous with device time
+ * @sys_monoraw: Monotonic raw simultaneous with device time
+ */
+struct system_device_crosststamp {
+ ktime_t device;
+ ktime_t sys_realtime;
+ ktime_t sys_monoraw;
+};
+
+/*
+ * struct system_counterval_t - system counter value with the pointer to the
+ * corresponding clocksource
+ * @cycles:System counter value
+ * @cs:Clocksource corresponding to system counter value. Used
by
+ * timekeeping code to verify comparibility of two cycle values
+ */
+struct system_counterval_t {
+ cycle_t cycles;
+ struct clocksource *cs;
+};
+
+/*
+ * Get cross timestamp between system clock and device clock
+ */
+extern int get_device_system_crosststamp(
+ int (*get_time_fn)(ktime_t *device_time,
+ struct system_counterval_t *system_counterval,
+ void *ctx),
+ void *ctx,
+ struct system_device_crosststamp *xtstamp);
+
+/*
* Simultaneously snapshot realtime and monotonic raw clocks
*/
extern void ktime_get_snapshot(struct system_time_snapshot *systime_snapshot);
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index f1a1c97..8c53398 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -908,6 +908,64 @@ void ktime_get_snapshot(struct system_time_snapshot
*systime_snapshot)
EXPORT_SYMBOL_GPL(ktime_get_snapshot);
/**
+ * get_device_system_crosststamp - Synchronously capture system/device
timestamp
+ * @sync_devicetime: Callback to get simultaneous device time and
+ * system counter from the device driver
+ * @xtstamp: Receives simultaneously captured system and device time
+ *
+ * Reads a timestamp from a device and correlates it to system time
+ */
+int get_device_system_crosststamp(int (*get_time_fn)
+ (ktime_t *device_time,
+ struct system_counterval_t *sys_counterval,
+ void *ctx),
+ void *ctx,
+ struct system_device_crosststamp *xtstamp)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ struct system_counterval_t system_counterval;
+ ktime_t base_raw;
+ ktime_t base_real;
+ s64 nsec_raw;
+ s64 nsec_real;
+ int ret;
+
+ do {
+ seq = r
[PATCH v7 6/8] x86: tsc: Always Running Timer (ART) correlated clocksource
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 2]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Tweaked to fix build issue, also reworked math for
64bit division on 32bit systems]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
arch/x86/include/asm/cpufeature.h | 3 ++-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/cpu/scattered.c | 1 +
arch/x86/kernel/tsc.c | 50 +++
4 files changed, 55 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 7ad8c94..111b892 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
@@ -188,6 +188,7 @@
#define X86_FEATURE_CPB( 7*32+ 2) /* AMD Core Performance
Boost */
#define X86_FEATURE_EPB( 7*32+ 3) /* IA32_ENERGY_PERF_BIAS
support */
+#define X86_FEATURE_INVARIANT_TSC (7*32+4) /* Intel Invariant TSC */
#define X86_FEATURE_HW_PSTATE ( 7*32+ 8) /* AMD HW-PState */
#define X86_FEATURE_PROC_FEEDBACK ( 7*32+ 9) /* AMD ProcFeedbackInterface */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..174c421 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct system_counterval_t convert_art_to_tsc(cycle_t art);
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
index 8cb57df..af0ecd7 100644
--- a/arch/x86/kernel/cpu/scattered.c
+++ b/arch/x86/kernel/cpu/scattered.c
@@ -35,6 +35,7 @@ void init_scattered_cpuid_features(struct cpuinfo_x86 *c)
{ X86_FEATURE_APERFMPERF, CR_ECX, 0, 0x0006, 0 },
{ X86_FEATURE_EPB, CR_ECX, 3, 0x0006, 0 },
{ X86_FEATURE_HW_PSTATE,CR_EDX, 7, 0x8007, 0 },
+ { X86_FEATURE_INVARIANT_TSC,CR_EDX, 8, 0x8007, 0 },
{ X86_FEATURE_CPB, CR_EDX, 9, 0x8007, 0 },
{ X86_FEATURE_PROC_FEEDBACK,CR_EDX,11, 0x8007, 0 },
{ 0, 0, 0, 0, 0 }
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3d743da..0ee3b62 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -43,6 +43,10 @@ static DEFINE_STATIC_KEY_FALSE(__use_tsc);
int tsc_clocksource_reliable;
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+struct clocksource *art_related_clocksource;
+
/*
* Use a ring-buffer like data structure, where a writer advances the head by
* writing a new data entry and a reader advances the tail when it observes a
@@ -949,10 +953,35 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+/* The denominator will never be less that 2 */
+#define ART_MIN_DENOMINATOR (2)
+
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (boot_cpu_has(X86_FEATURE_INVARIANT_TSC) &&
+ art_to_tsc_denominator >= ART_MIN_DENOMINATOR)
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+}
+
static int __init cpufreq_tsc(void)
{
[PATCH v7 7/8] ptp: Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping
Currently, network /system cross-timestamping is performed in the
PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday() and
the gettime64() callback provided by the driver. The cross-timestamp
is best effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be
significant.
The getcrosststamp() callback and corresponding PTP_SYS_OFFSET_PRECISE
ioctl allows the driver to perform this device/system correlation when
for example cross timestamp hardware is available. Modern Intel
systems can do this for onboard Ethernet controllers using the ART
counter. There is virtually zero latency between captures of the ART
and network device clock.
The capabilities ioctl (PTP_CLOCK_GETCAPS), is augmented allowing
applications to query whether or not drivers implement the
getcrosststamp callback, providing more precise cross timestamping.
Acked-by: Richard Cochran <richardcoch...@gmail.com>
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
[jstultz: Commit subject tweaks]
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 27 +++
include/linux/ptp_clock_kernel.h | 8
include/uapi/linux/ptp_clock.h | 13 -
4 files changed, 51 insertions(+), 3 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 6c6247a..d99012f 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -277,13 +277,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..579fd65 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "ptp_private.h"
@@ -120,11 +121,13 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock_caps caps;
struct ptp_clock_request req;
struct ptp_sys_offset *sysoff = NULL;
+ struct ptp_sys_offset_precise precise_offset;
struct ptp_pin_desc pd;
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
struct timespec64 ts;
+ struct system_device_crosststamp xtstamp;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +141,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getcrosststamp != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -180,6 +184,29 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
err = ops->enable(ops, , enable);
break;
+ case PTP_SYS_OFFSET_PRECISE:
+ if (!ptp->info->getcrosststamp) {
+ err = -EOPNOTSUPP;
+ break;
+ }
+ err = ptp->info->getcrosststamp(ptp->info, );
+ if (err)
+ break;
+
+ ts = ktime_to_timespec64(xtstamp.device);
+ precise_offset.device.sec = ts.tv_sec;
+ precise_offset.device.nsec = ts.tv_nsec;
+ ts = ktime_to_timespec64(xtstamp.sys_realtime);
+ precise_offset.sys_realtime.sec = ts.tv_sec;
+ precise_offset.sys_realtime.nsec = ts.tv_nsec;
+ ts = ktime_to_timespec64(xtstamp.sys_monoraw);
+ precise_offset.sys_monoraw.sec = ts.tv_sec;
+ precise_offset.sys_monoraw.nsec = ts.tv_nsec;
+ if (copy_to_user((void __user *)arg, _offset,
+sizeof(precise_offset)))
+
[PATCH v7 5/8] time: Add history to cross timestamp interface supporting slower devices
Another representative use case of time sync and the correlated
clocksource (in addition to PTP noted above) is PTP synchronized
audio.
In a streaming application, as an example, samples will be sent and/or
received by multiple devices with a presentation time that is in terms
of the PTP master clock. Synchronizing the audio output on these
devices requires correlating the audio clock with the PTP master
clock. The more precise this correlation is, the better the audio
quality (i.e. out of sync audio sounds bad).
>From an application standpoint, to correlate the PTP master clock with
the audio device clock, the system clock is used as a intermediate
timebase. The transforms such an application would perform are:
System Clock <-> Audio clock
System Clock <-> Network Device Clock [<-> PTP Master Clock]
Modern Intel platforms can perform a more accurate cross timestamp in
hardware (ART,audio device clock). The audio driver requires
ART->system time transforms -- the same as required for the network
driver. These platforms offload audio processing (including
cross-timestamps) to a DSP which to ensure uninterrupted audio
processing, communicates and response to the host only once every
millsecond. As a result is takes up to a millisecond for the DSP to
receive a request, the request is processed by the DSP, the audio
output hardware is polled for completion, the result is copied into
shared memory, and the host is notified. All of these operation occur
on a millisecond cadence. This transaction requires about 2 ms, but
under heavier workloads it may take up to 4 ms.
Adding a history allows these slow devices the option of providing an
ART value outside of the current interval. In this case, the callback
provided is an accessor function for the previously obtained counter
value. If get_system_device_crosststamp() receives a counter value
previous to cycle_last, it consults the history provided as an
argument in history_ref and interpolates the realtime and monotonic
raw system time using the provided counter value. If there are any
clock discontinuities, e.g. from calling settimeofday(), the monotonic
raw time is interpolated in the usual way, but the realtime clock time
is adjusted by scaling the monotonic raw adjustment.
When an accessor function is used a history argument *must* be
provided. The history is initialized using ktime_get_snapshot() and
must be called before the counter values are read.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
Signed-off-by: John Stultz <john.stu...@linaro.org>
---
include/linux/timekeeper_internal.h | 2 +
include/linux/timekeeping.h | 5 ++
kernel/time/timekeeping.c | 172 +++-
3 files changed, 178 insertions(+), 1 deletion(-)
diff --git a/include/linux/timekeeper_internal.h
b/include/linux/timekeeper_internal.h
index 2524722..e880054 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -50,6 +50,7 @@ struct tk_read_base {
* @offs_tai: Offset clock monotonic -> clock tai
* @tai_offset:The current UTC to TAI offset in seconds
* @clock_was_set_seq: The sequence number of clock was set events
+ * @cs_was_changed_seq:The sequence number of clocksource change events
* @next_leap_ktime: CLOCK_MONOTONIC time value of a pending leap-second
* @raw_time: Monotonic raw base time in timespec64 format
* @cycle_interval:Number of clock cycles in one NTP interval
@@ -91,6 +92,7 @@ struct timekeeper {
ktime_t offs_tai;
s32 tai_offset;
unsigned intclock_was_set_seq;
+ u8 cs_was_changed_seq;
ktime_t next_leap_ktime;
struct timespec64 raw_time;
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index 75bb836..8b90d06 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -272,11 +272,15 @@ extern void ktime_get_raw_and_real_ts64(struct timespec64
*ts_raw,
* @cycles:Clocksource counter value to produce the system times
* @real: Realtime system time
* @raw: Monotonic raw system time
+ * @clock_was_set_seq: The sequence number of clock was set events
+ * @cs_was_changed_seq:The sequence number of clocksource change events
*/
struct system_time_snapshot {
cycles_tcycles;
ktime_t real;
ktime_t raw;
+ unsigned intclock_was_set_seq;
+ u8 cs_was_changed_seq;
};
/*
@@ -312,6 +316,7 @@ extern int get_device_system_crosststamp(
struct system_counterval_t *system_counterval,
void *ctx),
void *ctx,
+ struct system_time_snapshot *history,
struct system_d
[RFC v5 4/6] Remove duplicate code from ktime_get_raw_and_real code
The code in ktime_get_snapshot() is a superset of the code in
ktime_get_raw_and_real() code. Changes the latter to call the former. A
side effect of this is that ktime_get_raw_and_real() returns two clock
times corresponding to the *exact* same clock tick. Previously, this
code read the underlying counter twice.
Signed-off-by: Christopher S. Hall
---
kernel/time/timekeeping.c | 20
1 file changed, 4 insertions(+), 16 deletions(-)
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 5a7f784..a0f096c 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -910,26 +910,14 @@ EXPORT_SYMBOL_GPL(ktime_get_snapshot);
*/
void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw, struct timespec64
*ts_real)
{
- struct timekeeper *tk = _core.timekeeper;
- unsigned long seq;
- s64 nsecs_raw, nsecs_real;
+ struct system_time_snapshot snap;
WARN_ON_ONCE(timekeeping_suspended);
- do {
- seq = read_seqcount_begin(_core.seq);
-
- *ts_raw = tk->raw_time;
- ts_real->tv_sec = tk->xtime_sec;
- ts_real->tv_nsec = 0;
-
- nsecs_raw = timekeeping_get_ns(>tkr_raw);
- nsecs_real = timekeeping_get_ns(>tkr_mono);
-
- } while (read_seqcount_retry(_core.seq, seq));
+ ktime_get_snapshot();
- timespec64_add_ns(ts_raw, nsecs_raw);
- timespec64_add_ns(ts_real, nsecs_real);
+ *ts_raw = ktime_to_timespec64(snap.raw);
+ *ts_real = ktime_to_timespec64(snap.real);
}
EXPORT_SYMBOL(ktime_get_raw_and_real_ts64);
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[RFC v5 5/6] Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping
Currently, network /system cross-timestamping is performed in the
PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday() and
the gettime64() callback provided by the driver. The cross-timestamp
is best effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be
significant.
The getsynctime() callback and corresponding PTP_SYS_OFFSET_PRECISE
ioctl allows the driver to perform this device/system correlation when
for example cross timestamp hardware is available. Modern Intel
systems can do this for onboard Ethernet controllers using the ART
counter. There is virtually zero latency between captures of the ART
and network device clock.
The capabilities ioctl (PTP_CLOCK_GETCAPS), is augmented allowing
applications to query whether or not drivers implement the getsynctime
callback, providing more precise cross timestamping.
Signed-off-by: Christopher S. Hall
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 31 +++
include/linux/ptp_clock_kernel.h | 8
include/uapi/linux/ptp_clock.h | 13 -
4 files changed, 55 insertions(+), 3 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 6c6247a..d99012f 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -277,13 +277,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..f39ee73 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "ptp_private.h"
@@ -120,11 +121,14 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock_caps caps;
struct ptp_clock_request req;
struct ptp_sys_offset *sysoff = NULL;
+ struct ptp_sys_offset_precise precise_offset;
struct ptp_pin_desc pd;
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
struct timespec64 ts;
+ struct system_device_crosststamp xtstamp;
+ u32 rem;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +142,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getsynctime != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -180,6 +185,32 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
err = ops->enable(ops, , enable);
break;
+ case PTP_SYS_OFFSET_PRECISE:
+ if (!ptp->info->getsynctime) {
+ err = -EINVAL;
+ break;
+ }
+ err = ptp->info->getsynctime(ptp->info, );
+ if (err)
+ break;
+
+ precise_offset.sys_real.sec =
+ div_u64_rem(ktime_to_ns(xtstamp.sys_realtime),
+ NSEC_PER_SEC, );
+ precise_offset.sys_real.nsec = rem;
+ precise_offset.sys_raw.sec =
+ div_u64_rem(ktime_to_ns(xtstamp.sys_monoraw),
+ NSEC_PER_SEC, );
+ precise_offset.sys_raw.nsec = rem;
+ precise_offset.dev.sec =
+ div_u64_rem(ktime_to_ns(xtstamp.device), NSEC_PER_SEC,
+ );
+ precise_offset.dev.nsec = rem;
+ if (copy_to_user((void __user *)arg, _offset,
+sizeof(precise_offset)))
+ err = -EFAULT;
+ break;
+
case PTP_SYS_OFFSET:
[RFC v5 3/6] Add history to cross timestamp interface supporting slower devices
Another representative use case of time sync and the correlated
clocksource (in addition to PTP noted above) is PTP synchronized
audio.
In a streaming application, as an example, samples will be sent and/or
received by multiple devices with a presentation time that is in terms
of the PTP master clock. Synchronizing the audio output on these
devices requires correlating the audio clock with the PTP master
clock. The more precise this correlation is, the better the audio
quality (i.e. out of sync audio sounds bad).
>From an application standpoint, to correlate the PTP master clock with
the audio device clock, the system clock is used as a intermediate
timebase. The transforms such an application would perform are:
System Clock <-> Audio clock
System Clock <-> Network Device Clock [<-> PTP Master Clock]
Such audio applications make use of some existing ALSA library calls
that provide audio/system cross-timestamps (e.g.
snd_pcm_status_get_htstamp()). Previous driver implementations capture
these cross timestamps by reading the system clock (raw/mono/real) and
the device clock with greatest degree of simultaneity possible in
software.
Modern Intel platforms can perform a more accurate cross timestamp in
hardware (ART,audio device clock). The audio driver requires
ART->system time transforms -- the same as required for the network
driver. These platforms offload audio processing (including
cross-timestamps) to a DSP which to ensure uninterrupted audio
processing, communicates and response to the host only once every
millsecond. As a result is takes up to a millisecond for the DSP to
receive a request, the request is processed by the DSP, the audio
output hardware is polled for completion, the result is copied into
shared memory, and the host is notified. All of these operation occur
on a millisecond cadence. This transaction requires about 2 ms, but
under heavier workloads it may take up to 4 ms.
If update_wall_time() is called while waiting for a response within
get_device_system_crosststamp() (from previous patch), a retry is
attempted. This will occur if the cycle_interval (determined by
CONFIG_HZ and mult/shift values) cycles elapse.
Adding a history allows these slow devices the option of providing an
ART value outside of the retry loop. In this case, the callback
provided is an accessor function for the previously obtained counter
value. If get_system_device_crosststamp() receives a counter value
previous to cycle_last, it consults the history provided as an
argument in history_ref and interpolates the realtime and monotonic
raw system time using the provided counter value. If there are any
clock discontinuities, e.g. from calling settimeofday(), the monotonic
raw time is interpolated in the usual way, but the realtime clock time
is adjusted by scaling the monotonic raw adjustment.
When an accessor function is used a history argument *must* be
provided. The history is initialized using ktime_get_snapshot() and
must be called before the counter values are read.
When the history is used to interpolate timestamp values, the realtime
clock time may be inaccurate to some degree. In general, the
longer the length of history the larger the interpolation error. If
there are discontinuities (large step changes) to the time, the error
can be very large.
Signed-off-by: Christopher S. Hall
---
include/linux/clocksource.h | 16
include/linux/timekeeper_internal.h | 4 +
include/linux/timekeeping.h | 9 +-
kernel/time/timekeeping.c | 162 ++--
4 files changed, 185 insertions(+), 6 deletions(-)
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 4b7973d..f413157 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -282,4 +282,20 @@ struct raw_system_counterval {
struct clocksource *cs;
};
+/*
+ * struct system_time_snapshot - simultaneous raw/real time capture with
+ * counter value
+ * @cycles:Clocksource counter value to produce the system times
+ * @real: Realtime system time
+ * @raw: Monotonic raw system time
+ * @cs_seq:Sequence number associated with changed clocksource
+ */
+struct system_time_snapshot {
+ cycles_tcycles;
+ ktime_t real;
+ ktime_t raw;
+ u8 cs_seq;
+ unsigned intclock_set_seq;
+};
+
#endif /* _LINUX_CLOCKSOURCE_H */
diff --git a/include/linux/timekeeper_internal.h
b/include/linux/timekeeper_internal.h
index 2524722..156b51d 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -13,6 +13,9 @@
/**
* struct tk_read_base - base structure for timekeeping readout
* @clock: Current clocksource used for timekeeping.
+ * @cs_seq:Clocksource sequence is incremented per clocksource change.
+ * It's used to determine whether past system time can be related to
+ * curre
[RFC v5 2/6] Always Running Timer (ART) correlated clocksource
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 2]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Signed-off-by: Christopher S. Hall
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 46 +++
3 files changed, 49 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index e4f8010..58c799e 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..9474c9c 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct correlated_cs art_timestamper;
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index c7c4d9c..26dcf63 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -949,10 +949,36 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+/* The denominator will never be less that 2 */
+#define ART_MIN_DENOMINATOR (2)
+
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (art_to_tsc_denominator >= ART_MIN_DENOMINATOR)
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ detect_art();
+
if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
return 0;
cpufreq_register_notifier(_cpufreq_notifier_block,
@@ -1071,6 +1097,24 @@ int unsynchronized_tsc(void)
return 0;
}
+/*
+ * Convert ART to TSC given numerator/denominator found in detect_art()
+ */
+static u64 convert_art_to_tsc(struct correlated_cs *cs, u64 cycles)
+{
+ u64 tmp, res;
+
+ res = (cycles / art_to_tsc_denominator) * art_to_tsc_numerator;
+ tmp = (cycles % art_to_tsc_denominator) * art_to_tsc_numerator;
+ res += tmp / art_to_tsc_denominator;
+
+ return res;
+}
+
+struct correlated_cs art_timestamper = {
+ .convert= convert_art_to_tsc,
+};
+EXPORT_SYMBOL(art_timestamper);
static void tsc_refine_calibration_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(tsc_irqwork, tsc_refine_calibration_work);
@@ -1142,6 +1186,8 @@ static void tsc_refine_calibration_work(struct
work_struct *work)
(unsigned long)tsc_khz % 1000);
out:
+ if (boot_cpu_has(X86_FEATURE_ART))
+ art_timestamper.related_cs = _tsc;
clocksource_register_khz(_tsc, tsc_khz);
}
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[RFC v5 6/6] Adds hardware supported cross timestamp
Modern Intel systems supports cross timestamping of the network device
clock and Always Running Timer (ART) in hardware. This allows the
device time and system time to be precisely correlated. The timestamp
pair is returned through e1000e_phc_get_sync_devicetime() used by
get_device_system_crosststamp(). The hardware cross-timestamp result
is made available to applications through the PTP_SYS_OFFSET_PRECISE
ioctl which call e1000e_phc_getsynctime().
Signed-off-by: Christopher S. Hall
---
drivers/net/ethernet/intel/Kconfig | 9 +++
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 92 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
4 files changed, 110 insertions(+)
diff --git a/drivers/net/ethernet/intel/Kconfig
b/drivers/net/ethernet/intel/Kconfig
index 4163b16..62bbddc 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -83,6 +83,15 @@ config E1000E
To compile this driver as a module, choose M here. The module
will be called e1000e.
+config E1000E_HWTS
+ bool "Support HW cross-timestamp on PCH devices"
+ default y
+ depends on E1000E && X86
+ ---help---
+Say Y to enable hardware supported cross-timestamping on PCH
+devices. The cross-timestamp is available through the PTP clock
+driver precise cross-timestamp ioctl (PTP_SYS_OFFSET_PRECISE).
+
config IGB
tristate "Intel(R) 82575/82576 PCI-Express Gigabit Ethernet support"
depends on PCI
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index 133d407..13cff75 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -527,6 +527,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..44d2caa 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -26,6 +26,11 @@
#include "e1000.h"
+#ifdef CONFIG_E1000E_HWTS
+#include
+#include
+#endif
+
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
* @ptp: ptp clock structure
@@ -98,6 +103,87 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#ifdef CONFIG_E1000E_HWTS
+#define MAX_HW_WAIT_COUNT (3)
+
+/**
+ * e1000e_phc_get_sync_devicetime - Callback given to timekeeping code reads
system/device registers
+ * @device: current device time
+ * @system: raw system counter value read synchronously with device time
+ * @ctx: context provided by timekeeping code
+ *
+ * Read device and system (ART) clock simultaneously and return the corrected
+ * clock values in ns.
+ **/
+static int e1000e_phc_get_sync_devicetime(ktime_t *device,
+ struct raw_system_counterval *system,
+ void *ctx)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)ctx;
+ struct e1000_hw *hw = >hw;
+ unsigned long flags;
+ int i;
+ u32 tsync_ctrl;
+ cycle_t dev_cycles;
+ int ret;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT) {
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ system->cycles = er32(PLTSTMPH);
+ system->cycles <<= 32;
+ system->cycles |= er32(PLTSTMPL);
+ system->cs = art_timestamper.related_cs;
+ system->cycles = art_timestamper.convert(_timestamper,
+system->cycles);
+
+ dev_cycles = er32(SYSSTMPH);
+ dev_cycles <<= 32;
+ dev_cycles |= er32(SYSSTMPL);
+ spin_lock_irqsave(>systim_lock, flags);
+ device->tv64 = timecounter_cyc2time(>tc, dev_cycles);
[RFC v5 0/6] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
Modern Intel hardware adds an Always Running Timer (ART) that allows the network and audio device clocks to precisely cross timestamp the device clock with the system clock. This allows a precise correlation of the device time and system time. This patchset adds interfaces to the timekeeping code allowing drivers to translate ART time to system time. Changelog: Changes from v4 to v5: * Changes the history mechanism to interpolate system time using a single historic system time pair (monotonic raw, realtime) rather than implementing a precise history using shadow timekeeper (see v4 changes). The advantage of this approach is that the history can be arbitrarily long. This approach may also be simpler in terms of coding. The major disadvantage is that the realtime clock can be adjusted. When adjusted, the realtime clock time (when interpolating from history) is always approximate. In general, the longer the interpolation period the larger the potential error. There isn't any error interpolating the monotonic raw clock time. * This patchset also addresses objections to the previous patchsets overly complex correlated timestamp structure. This patchset splits that structure into several smaller structures. The correlated timestamp interface is renamed cross timestamp to avoid any confusion with the correlated clocksource. * The correlated clocksource is separated from the cross timestamp mechanism. * Add monotonic raw to the PTP user interface * Add e1000e driver configuration option that wraps Intel PCH specific code Changes v3 to v4: * Adds a history mechanism to accomodate slower devices. In this case the response time for timestamp reads to the Intel DSP are too slow to be accomodated by the original correlated time mechanism. The history mechanism turns shadow timekeeper into an array where the history is stored. Christopher S. Hall (6): Timekeeping cross timestamp interface for device drivers Always Running Timer (ART) correlated clocksource Add history to cross timestamp interface supporting slower devices Remove duplicate code from ktime_get_raw_and_real code Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping Adds hardware supported cross timestamp Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 2 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/tsc.c | 46 ++ drivers/net/ethernet/intel/Kconfig | 9 ++ drivers/net/ethernet/intel/e1000e/defines.h | 5 + drivers/net/ethernet/intel/e1000e/ptp.c | 92 + drivers/net/ethernet/intel/e1000e/regs.h| 4 + drivers/ptp/ptp_chardev.c | 31 include/linux/clocksource.h | 43 + include/linux/ptp_clock_kernel.h| 8 + include/linux/timekeeper_internal.h | 4 + include/linux/timekeeping.h | 42 + include/uapi/linux/ptp_clock.h | 13 +- kernel/time/timekeeping.c | 243 ++-- 15 files changed, 521 insertions(+), 21 deletions(-) -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[RFC v5 1/6] Timekeeping cross timestamp interface for device drivers
ACKNOWLEDGMENT: The original correlated clock source and cross
timestamp code was developed by Thomas Gleixner
. It has changed considerably and any mistakes are
mine.
The precision with which events on multiple networked systems can be
synchronized using, as an example, PTP (IEEE 1588, 802.1AS) is limited
by the precision of the cross timestamps between the system clock and
the device (timestamp) clock. Precision here is the degree of
simultaneity when capturing the cross timestamp.
Currently the PTP cross timestamp is captured in software using the
PTP device driver ioctl PTP_SYS_OFFSET. Reads of the device clock are
interleaved with reads of the realtime clock. At best, the precision
of this cross timestamp is on the order of several microseconds due to
software latencies. Sub-microsecond precision is required for
industrial control and some media applications. To achieve this level
of precision hardware supported cross timestamping is needed.
Hardware cross timestamps are derived from simultaneously capturing
the device clock and the system clock. Applications use timestamps in
nanoseconds rather than clock ticks. The device driver can scale
device clock ticks to device time in nanoseconds, but cannot transform
system clock ticks. Only the kernel timekeeping code can do this. The
function get_device_system_crosststamp() allows device drivers to
return a cross timestamp with system time properly scaled to
nanoseconds.
The cross timestamps contain the realtime and monotonic raw clock
times. The realtime value is needed to discipline that clock using PTP
and the monotonic raw value is used for applications that don't
require a "real" time, but need an unadjusted clock time.
The get_device_system_crosststamp() code calls back into the driver
to ensure that the system counter is within the current timekeeping
update interval. Because of possible NTP/PTP frequency adjustments,
extrapolating a realtime clock time outside the current interval with
a potentially different scaling factor can result in a small amount of
error.
Modern Intel hardware provides an Always Running Timer (ART) which is
exactly related to TSC through a known frequency ratio. The ART is
routed to devices on the system and is used to precisely and
simultaneously capture the device clock with the ART. The kernel
timekeeping code requires a system clock value from the current clock
source to calculate the system time. The correlated clocksource adds a
means to transform a clock (in this case ART) exactly to the system
clock (TSC clocksource) that is used to calculate system time.
Signed-off-by: Christopher S. Hall
---
include/linux/clocksource.h | 27
include/linux/timekeeping.h | 35 +
kernel/time/timekeeping.c | 77 ++---
3 files changed, 135 insertions(+), 4 deletions(-)
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 7784b59..4b7973d 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -255,4 +255,31 @@ static inline void clocksource_probe(void) {}
#define CLOCKSOURCE_ACPI_DECLARE(name, table_id, fn) \
ACPI_DECLARE_PROBE_ENTRY(clksrc, name, table_id, 0, NULL, 0, fn)
+/*
+ * struct correlated_cs - Descriptor for a clocksource correlated to another
+ * clocksource
+ * @related_cs:Pointer to the related timekeeping clocksource
+ * @convert: Conversion function to convert a timestamp from
+ * the correlated clocksource to cycles of the related
+ * timekeeping clocksource
+ */
+struct correlated_cs {
+ struct clocksource *related_cs;
+ cycle_t (*convert)(struct correlated_cs *cs,
+ cycle_t cycles);
+};
+
+/*
+ * struct raw_system_counterval - system counter value captured in device
+ * driver used to produce system/device cross-timestamp
+ * @system:System counter value
+ * @cs:Clocksource related to system counter value. This is
used by
+ * timekeeping code to verify validity of counter for system time
+ * conversion
+ */
+struct raw_system_counterval {
+ cycle_t cycles;
+ struct clocksource *cs;
+};
+
#endif /* _LINUX_CLOCKSOURCE_H */
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index ec89d84..2209943 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -266,6 +266,41 @@ extern void timekeeping_inject_sleeptime64(struct
timespec64 *delta);
extern void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw,
struct timespec64 *ts_real);
+
+/*
+ * struct system_device_crosststamp - system/device cross-timestamp
+ * (syncronized capture)
+ * @device:Device time
+ * @realtime: Realtime simultaneous with device time
+ * @monoraw: Monotonic raw si
[RFC v5 6/6] Adds hardware supported cross timestamp
Modern Intel systems supports cross timestamping of the network device
clock and Always Running Timer (ART) in hardware. This allows the
device time and system time to be precisely correlated. The timestamp
pair is returned through e1000e_phc_get_sync_devicetime() used by
get_device_system_crosststamp(). The hardware cross-timestamp result
is made available to applications through the PTP_SYS_OFFSET_PRECISE
ioctl which call e1000e_phc_getsynctime().
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
drivers/net/ethernet/intel/Kconfig | 9 +++
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 92 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
4 files changed, 110 insertions(+)
diff --git a/drivers/net/ethernet/intel/Kconfig
b/drivers/net/ethernet/intel/Kconfig
index 4163b16..62bbddc 100644
--- a/drivers/net/ethernet/intel/Kconfig
+++ b/drivers/net/ethernet/intel/Kconfig
@@ -83,6 +83,15 @@ config E1000E
To compile this driver as a module, choose M here. The module
will be called e1000e.
+config E1000E_HWTS
+ bool "Support HW cross-timestamp on PCH devices"
+ default y
+ depends on E1000E && X86
+ ---help---
+Say Y to enable hardware supported cross-timestamping on PCH
+devices. The cross-timestamp is available through the PTP clock
+driver precise cross-timestamp ioctl (PTP_SYS_OFFSET_PRECISE).
+
config IGB
tristate "Intel(R) 82575/82576 PCI-Express Gigabit Ethernet support"
depends on PCI
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index 133d407..13cff75 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -527,6 +527,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..44d2caa 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -26,6 +26,11 @@
#include "e1000.h"
+#ifdef CONFIG_E1000E_HWTS
+#include
+#include
+#endif
+
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
* @ptp: ptp clock structure
@@ -98,6 +103,87 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#ifdef CONFIG_E1000E_HWTS
+#define MAX_HW_WAIT_COUNT (3)
+
+/**
+ * e1000e_phc_get_sync_devicetime - Callback given to timekeeping code reads
system/device registers
+ * @device: current device time
+ * @system: raw system counter value read synchronously with device time
+ * @ctx: context provided by timekeeping code
+ *
+ * Read device and system (ART) clock simultaneously and return the corrected
+ * clock values in ns.
+ **/
+static int e1000e_phc_get_sync_devicetime(ktime_t *device,
+ struct raw_system_counterval *system,
+ void *ctx)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)ctx;
+ struct e1000_hw *hw = >hw;
+ unsigned long flags;
+ int i;
+ u32 tsync_ctrl;
+ cycle_t dev_cycles;
+ int ret;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT) {
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ system->cycles = er32(PLTSTMPH);
+ system->cycles <<= 32;
+ system->cycles |= er32(PLTSTMPL);
+ system->cs = art_timestamper.related_cs;
+ system->cycles = art_timestamper.convert(_timestamper,
+system->cycles);
+
+ dev_cycles = er32(SYSSTMPH);
+ dev_cycles <<= 32;
+ dev_cycles |= er32(SYSSTMPL);
+ spin_lock_irqsave(>systim_lock, flags);
+ device->tv64 = timecou
[RFC v5 0/6] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
Modern Intel hardware adds an Always Running Timer (ART) that allows the network and audio device clocks to precisely cross timestamp the device clock with the system clock. This allows a precise correlation of the device time and system time. This patchset adds interfaces to the timekeeping code allowing drivers to translate ART time to system time. Changelog: Changes from v4 to v5: * Changes the history mechanism to interpolate system time using a single historic system time pair (monotonic raw, realtime) rather than implementing a precise history using shadow timekeeper (see v4 changes). The advantage of this approach is that the history can be arbitrarily long. This approach may also be simpler in terms of coding. The major disadvantage is that the realtime clock can be adjusted. When adjusted, the realtime clock time (when interpolating from history) is always approximate. In general, the longer the interpolation period the larger the potential error. There isn't any error interpolating the monotonic raw clock time. * This patchset also addresses objections to the previous patchsets overly complex correlated timestamp structure. This patchset splits that structure into several smaller structures. The correlated timestamp interface is renamed cross timestamp to avoid any confusion with the correlated clocksource. * The correlated clocksource is separated from the cross timestamp mechanism. * Add monotonic raw to the PTP user interface * Add e1000e driver configuration option that wraps Intel PCH specific code Changes v3 to v4: * Adds a history mechanism to accomodate slower devices. In this case the response time for timestamp reads to the Intel DSP are too slow to be accomodated by the original correlated time mechanism. The history mechanism turns shadow timekeeper into an array where the history is stored. Christopher S. Hall (6): Timekeeping cross timestamp interface for device drivers Always Running Timer (ART) correlated clocksource Add history to cross timestamp interface supporting slower devices Remove duplicate code from ktime_get_raw_and_real code Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping Adds hardware supported cross timestamp Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 2 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/tsc.c | 46 ++ drivers/net/ethernet/intel/Kconfig | 9 ++ drivers/net/ethernet/intel/e1000e/defines.h | 5 + drivers/net/ethernet/intel/e1000e/ptp.c | 92 + drivers/net/ethernet/intel/e1000e/regs.h| 4 + drivers/ptp/ptp_chardev.c | 31 include/linux/clocksource.h | 43 + include/linux/ptp_clock_kernel.h| 8 + include/linux/timekeeper_internal.h | 4 + include/linux/timekeeping.h | 42 + include/uapi/linux/ptp_clock.h | 13 +- kernel/time/timekeeping.c | 243 ++-- 15 files changed, 521 insertions(+), 21 deletions(-) -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[RFC v5 1/6] Timekeeping cross timestamp interface for device drivers
ACKNOWLEDGMENT: The original correlated clock source and cross
timestamp code was developed by Thomas Gleixner
<t...@linutronix.de>. It has changed considerably and any mistakes are
mine.
The precision with which events on multiple networked systems can be
synchronized using, as an example, PTP (IEEE 1588, 802.1AS) is limited
by the precision of the cross timestamps between the system clock and
the device (timestamp) clock. Precision here is the degree of
simultaneity when capturing the cross timestamp.
Currently the PTP cross timestamp is captured in software using the
PTP device driver ioctl PTP_SYS_OFFSET. Reads of the device clock are
interleaved with reads of the realtime clock. At best, the precision
of this cross timestamp is on the order of several microseconds due to
software latencies. Sub-microsecond precision is required for
industrial control and some media applications. To achieve this level
of precision hardware supported cross timestamping is needed.
Hardware cross timestamps are derived from simultaneously capturing
the device clock and the system clock. Applications use timestamps in
nanoseconds rather than clock ticks. The device driver can scale
device clock ticks to device time in nanoseconds, but cannot transform
system clock ticks. Only the kernel timekeeping code can do this. The
function get_device_system_crosststamp() allows device drivers to
return a cross timestamp with system time properly scaled to
nanoseconds.
The cross timestamps contain the realtime and monotonic raw clock
times. The realtime value is needed to discipline that clock using PTP
and the monotonic raw value is used for applications that don't
require a "real" time, but need an unadjusted clock time.
The get_device_system_crosststamp() code calls back into the driver
to ensure that the system counter is within the current timekeeping
update interval. Because of possible NTP/PTP frequency adjustments,
extrapolating a realtime clock time outside the current interval with
a potentially different scaling factor can result in a small amount of
error.
Modern Intel hardware provides an Always Running Timer (ART) which is
exactly related to TSC through a known frequency ratio. The ART is
routed to devices on the system and is used to precisely and
simultaneously capture the device clock with the ART. The kernel
timekeeping code requires a system clock value from the current clock
source to calculate the system time. The correlated clocksource adds a
means to transform a clock (in this case ART) exactly to the system
clock (TSC clocksource) that is used to calculate system time.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
include/linux/clocksource.h | 27
include/linux/timekeeping.h | 35 +
kernel/time/timekeeping.c | 77 ++---
3 files changed, 135 insertions(+), 4 deletions(-)
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 7784b59..4b7973d 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -255,4 +255,31 @@ static inline void clocksource_probe(void) {}
#define CLOCKSOURCE_ACPI_DECLARE(name, table_id, fn) \
ACPI_DECLARE_PROBE_ENTRY(clksrc, name, table_id, 0, NULL, 0, fn)
+/*
+ * struct correlated_cs - Descriptor for a clocksource correlated to another
+ * clocksource
+ * @related_cs:Pointer to the related timekeeping clocksource
+ * @convert: Conversion function to convert a timestamp from
+ * the correlated clocksource to cycles of the related
+ * timekeeping clocksource
+ */
+struct correlated_cs {
+ struct clocksource *related_cs;
+ cycle_t (*convert)(struct correlated_cs *cs,
+ cycle_t cycles);
+};
+
+/*
+ * struct raw_system_counterval - system counter value captured in device
+ * driver used to produce system/device cross-timestamp
+ * @system:System counter value
+ * @cs:Clocksource related to system counter value. This is
used by
+ * timekeeping code to verify validity of counter for system time
+ * conversion
+ */
+struct raw_system_counterval {
+ cycle_t cycles;
+ struct clocksource *cs;
+};
+
#endif /* _LINUX_CLOCKSOURCE_H */
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index ec89d84..2209943 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -266,6 +266,41 @@ extern void timekeeping_inject_sleeptime64(struct
timespec64 *delta);
extern void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw,
struct timespec64 *ts_real);
+
+/*
+ * struct system_device_crosststamp - system/device cross-timestamp
+ * (syncronized capture)
+ * @device:Device time
+ * @realtime: Realtime si
[RFC v5 2/6] Always Running Timer (ART) correlated clocksource
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 2]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 46 +++
3 files changed, 49 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index e4f8010..58c799e 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..9474c9c 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct correlated_cs art_timestamper;
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index c7c4d9c..26dcf63 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -949,10 +949,36 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+/* The denominator will never be less that 2 */
+#define ART_MIN_DENOMINATOR (2)
+
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (art_to_tsc_denominator >= ART_MIN_DENOMINATOR)
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ detect_art();
+
if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
return 0;
cpufreq_register_notifier(_cpufreq_notifier_block,
@@ -1071,6 +1097,24 @@ int unsynchronized_tsc(void)
return 0;
}
+/*
+ * Convert ART to TSC given numerator/denominator found in detect_art()
+ */
+static u64 convert_art_to_tsc(struct correlated_cs *cs, u64 cycles)
+{
+ u64 tmp, res;
+
+ res = (cycles / art_to_tsc_denominator) * art_to_tsc_numerator;
+ tmp = (cycles % art_to_tsc_denominator) * art_to_tsc_numerator;
+ res += tmp / art_to_tsc_denominator;
+
+ return res;
+}
+
+struct correlated_cs art_timestamper = {
+ .convert= convert_art_to_tsc,
+};
+EXPORT_SYMBOL(art_timestamper);
static void tsc_refine_calibration_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(tsc_irqwork, tsc_refine_calibration_work);
@@ -1142,6 +1186,8 @@ static void tsc_refine_calibration_work(struct
work_struct *work)
(unsigned long)tsc_khz % 1000);
out:
+ if (boot_cpu_has(X86_FEATURE_ART))
+ art_timestamper.related_cs = _tsc;
clocksource_register_khz(_tsc, tsc_khz);
}
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[RFC v5 5/6] Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping
Currently, network /system cross-timestamping is performed in the
PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday() and
the gettime64() callback provided by the driver. The cross-timestamp
is best effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be
significant.
The getsynctime() callback and corresponding PTP_SYS_OFFSET_PRECISE
ioctl allows the driver to perform this device/system correlation when
for example cross timestamp hardware is available. Modern Intel
systems can do this for onboard Ethernet controllers using the ART
counter. There is virtually zero latency between captures of the ART
and network device clock.
The capabilities ioctl (PTP_CLOCK_GETCAPS), is augmented allowing
applications to query whether or not drivers implement the getsynctime
callback, providing more precise cross timestamping.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 31 +++
include/linux/ptp_clock_kernel.h | 8
include/uapi/linux/ptp_clock.h | 13 -
4 files changed, 55 insertions(+), 3 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 6c6247a..d99012f 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -277,13 +277,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..f39ee73 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -22,6 +22,7 @@
#include
#include
#include
+#include
#include "ptp_private.h"
@@ -120,11 +121,14 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock_caps caps;
struct ptp_clock_request req;
struct ptp_sys_offset *sysoff = NULL;
+ struct ptp_sys_offset_precise precise_offset;
struct ptp_pin_desc pd;
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
struct timespec64 ts;
+ struct system_device_crosststamp xtstamp;
+ u32 rem;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +142,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getsynctime != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -180,6 +185,32 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
err = ops->enable(ops, , enable);
break;
+ case PTP_SYS_OFFSET_PRECISE:
+ if (!ptp->info->getsynctime) {
+ err = -EINVAL;
+ break;
+ }
+ err = ptp->info->getsynctime(ptp->info, );
+ if (err)
+ break;
+
+ precise_offset.sys_real.sec =
+ div_u64_rem(ktime_to_ns(xtstamp.sys_realtime),
+ NSEC_PER_SEC, );
+ precise_offset.sys_real.nsec = rem;
+ precise_offset.sys_raw.sec =
+ div_u64_rem(ktime_to_ns(xtstamp.sys_monoraw),
+ NSEC_PER_SEC, );
+ precise_offset.sys_raw.nsec = rem;
+ precise_offset.dev.sec =
+ div_u64_rem(ktime_to_ns(xtstamp.device), NSEC_PER_SEC,
+ );
+ precise_offset.dev.nsec = rem;
+ if (copy_to_user((void __user *)arg, _offset,
+sizeof(precise_offset)))
+ err = -EFAULT;
+ break;
+
[RFC v5 4/6] Remove duplicate code from ktime_get_raw_and_real code
The code in ktime_get_snapshot() is a superset of the code in
ktime_get_raw_and_real() code. Changes the latter to call the former. A
side effect of this is that ktime_get_raw_and_real() returns two clock
times corresponding to the *exact* same clock tick. Previously, this
code read the underlying counter twice.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
kernel/time/timekeeping.c | 20
1 file changed, 4 insertions(+), 16 deletions(-)
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index 5a7f784..a0f096c 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -910,26 +910,14 @@ EXPORT_SYMBOL_GPL(ktime_get_snapshot);
*/
void ktime_get_raw_and_real_ts64(struct timespec64 *ts_raw, struct timespec64
*ts_real)
{
- struct timekeeper *tk = _core.timekeeper;
- unsigned long seq;
- s64 nsecs_raw, nsecs_real;
+ struct system_time_snapshot snap;
WARN_ON_ONCE(timekeeping_suspended);
- do {
- seq = read_seqcount_begin(_core.seq);
-
- *ts_raw = tk->raw_time;
- ts_real->tv_sec = tk->xtime_sec;
- ts_real->tv_nsec = 0;
-
- nsecs_raw = timekeeping_get_ns(>tkr_raw);
- nsecs_real = timekeeping_get_ns(>tkr_mono);
-
- } while (read_seqcount_retry(_core.seq, seq));
+ ktime_get_snapshot();
- timespec64_add_ns(ts_raw, nsecs_raw);
- timespec64_add_ns(ts_real, nsecs_real);
+ *ts_raw = ktime_to_timespec64(snap.raw);
+ *ts_real = ktime_to_timespec64(snap.real);
}
EXPORT_SYMBOL(ktime_get_raw_and_real_ts64);
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
[RFC v5 3/6] Add history to cross timestamp interface supporting slower devices
Another representative use case of time sync and the correlated
clocksource (in addition to PTP noted above) is PTP synchronized
audio.
In a streaming application, as an example, samples will be sent and/or
received by multiple devices with a presentation time that is in terms
of the PTP master clock. Synchronizing the audio output on these
devices requires correlating the audio clock with the PTP master
clock. The more precise this correlation is, the better the audio
quality (i.e. out of sync audio sounds bad).
>From an application standpoint, to correlate the PTP master clock with
the audio device clock, the system clock is used as a intermediate
timebase. The transforms such an application would perform are:
System Clock <-> Audio clock
System Clock <-> Network Device Clock [<-> PTP Master Clock]
Such audio applications make use of some existing ALSA library calls
that provide audio/system cross-timestamps (e.g.
snd_pcm_status_get_htstamp()). Previous driver implementations capture
these cross timestamps by reading the system clock (raw/mono/real) and
the device clock with greatest degree of simultaneity possible in
software.
Modern Intel platforms can perform a more accurate cross timestamp in
hardware (ART,audio device clock). The audio driver requires
ART->system time transforms -- the same as required for the network
driver. These platforms offload audio processing (including
cross-timestamps) to a DSP which to ensure uninterrupted audio
processing, communicates and response to the host only once every
millsecond. As a result is takes up to a millisecond for the DSP to
receive a request, the request is processed by the DSP, the audio
output hardware is polled for completion, the result is copied into
shared memory, and the host is notified. All of these operation occur
on a millisecond cadence. This transaction requires about 2 ms, but
under heavier workloads it may take up to 4 ms.
If update_wall_time() is called while waiting for a response within
get_device_system_crosststamp() (from previous patch), a retry is
attempted. This will occur if the cycle_interval (determined by
CONFIG_HZ and mult/shift values) cycles elapse.
Adding a history allows these slow devices the option of providing an
ART value outside of the retry loop. In this case, the callback
provided is an accessor function for the previously obtained counter
value. If get_system_device_crosststamp() receives a counter value
previous to cycle_last, it consults the history provided as an
argument in history_ref and interpolates the realtime and monotonic
raw system time using the provided counter value. If there are any
clock discontinuities, e.g. from calling settimeofday(), the monotonic
raw time is interpolated in the usual way, but the realtime clock time
is adjusted by scaling the monotonic raw adjustment.
When an accessor function is used a history argument *must* be
provided. The history is initialized using ktime_get_snapshot() and
must be called before the counter values are read.
When the history is used to interpolate timestamp values, the realtime
clock time may be inaccurate to some degree. In general, the
longer the length of history the larger the interpolation error. If
there are discontinuities (large step changes) to the time, the error
can be very large.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
include/linux/clocksource.h | 16
include/linux/timekeeper_internal.h | 4 +
include/linux/timekeeping.h | 9 +-
kernel/time/timekeeping.c | 162 ++--
4 files changed, 185 insertions(+), 6 deletions(-)
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 4b7973d..f413157 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -282,4 +282,20 @@ struct raw_system_counterval {
struct clocksource *cs;
};
+/*
+ * struct system_time_snapshot - simultaneous raw/real time capture with
+ * counter value
+ * @cycles:Clocksource counter value to produce the system times
+ * @real: Realtime system time
+ * @raw: Monotonic raw system time
+ * @cs_seq:Sequence number associated with changed clocksource
+ */
+struct system_time_snapshot {
+ cycles_tcycles;
+ ktime_t real;
+ ktime_t raw;
+ u8 cs_seq;
+ unsigned intclock_set_seq;
+};
+
#endif /* _LINUX_CLOCKSOURCE_H */
diff --git a/include/linux/timekeeper_internal.h
b/include/linux/timekeeper_internal.h
index 2524722..156b51d 100644
--- a/include/linux/timekeeper_internal.h
+++ b/include/linux/timekeeper_internal.h
@@ -13,6 +13,9 @@
/**
* struct tk_read_base - base structure for timekeeping readout
* @clock: Current clocksource used for timekeeping.
+ * @cs_seq:Clocksource sequence is incremented per clocksource change.
+ * It's used to determine whether p
[PATCH v4 1/4] Produce system time from correlated clocksource
From: Thomas Gleixner
Modern Intel hardware provides the so called Always Running Timer
(ART). The TSC which is usually used for timekeeping is derived from
ART and runs with a fixed frequency ratio to it. ART is routed to
devices and allows to take atomic timestamp samples from the device
clock and the ART. One use case is PTP timestamps on network cards. We
want to utilize this feature as it allows us to better correlate the
PTP timestamp to the system time.
In order to gather precise timestamps we need to make sure that the
conversion from ART to TSC and the following conversion from TSC to
clock realtime happens synchronized with the ongoing timekeeping
updates. Otherwise we might convert an ART timestamp from point A in
time with the conversion factors of point B in time. These conversion
factors can differ due to NTP/PTP frequency adjustments and therefor
the resulting clock realtime timestamp would be slightly off, which is
contrary to the whole purpose of synchronized hardware timestamps.
Provide data structures which describe the correlation between two
clocksources and a function to gather correlated and convert
timestamps from a device. The function is as any other timekeeping
function protected against current timekeeper updates via the
timekeeper sequence lock. It calls the device function to gather the
hardware timestamps and converts them to clock real time and clock
monotonic raw.
Signed-off-by: Thomas Gleixner
Another representative use case of time sync and the correlated
clocksource (in addition to PTP noted above) is PTP synchronized
audio.
In a streaming application, as an example, samples will be sent
and/or received by multiple devices with a presentation time that is
in terms of the PTP master clock. Synchronizing the audio output on
these devices requires correlating the audio clock with the PTP
master clock. The more precise this correlation is, the better the
audio quality (i.e. out of sync audio sounds bad).
>From an application standpoint, to correlate the PTP master clock
with the audio device clock, the system clock is used as a
intermediate timebase. The transforms such an application would
perform are:
System Clock <-> Audio clock
System Clock <-> Network Device Clock [<-> PTP Master Clock]
Such audio applications make use of some existing ALSA library
calls that provide audio/system cross-timestamps (e.g.
snd_pcm_status_get_htstamp()). Previous driver implementations
capture these cross by reading the system clock (raw/mono/real)
and the device clock atomically in software.
Modern Intel platforms can perform a more accurate cross-
timestamp in hardware (ART,audio device clock). The audio driver
requires ART->system time transforms -- the same as required for
the network driver. These platforms offload audio processing
(including cross-timestamps) to a DSP which to ensure
uninterrupted audio processing, communicates and response to the
host only once every millsecond. As a result is takes up to a
millisecond for the DSP to receive a request, the request is
processed by the DSP, the audio output hardware is polled for
completion, the result is copied into shared memory, and the
host is notified. All of these operation occur on a millisecond
cadence. This transaction requires about 2 ms, but under
heavier workloads it may take up to 4 ms.
If update_wall_time() is called while waiting for a
response within get_correlated_ts() (from original patch), a retry
is attempted. This will occur if the cycle_interval(determined by
CONFIG_HZ and mult/shift values) cycles elapse.
The modification to the original patch accomodates these
slow devices by adding the option of providing an ART value outside
of the retry loop and adding a history which can consulted in the
case of an out of date counter value. The history is kept by
making the shadow_timekeeper an array. Each write to the
timekeeper rotates through the array, preserving a
history of updates.
With these changes, if get_correlated_timestamp() detects a counter
value previous to cycle_now, it consults the history in
shadow_timekeeper and translates the timestamp to the system time
value. If the timestamp value is too old, an error is returned
Signed-off-by: Christopher S. Hall
---
include/linux/clocksource.h | 33 +++
include/linux/timekeeping.h | 4 +
kernel/time/timekeeping.c | 203 ++--
3 files changed, 235 insertions(+), 5 deletions(-)
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 278dd27..4bedadb 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -258,4 +258,37 @@ void acpi_generic_timer_init(void);
static inline void acpi_generic_timer_init(void) { }
#endif
+/*
+ * struct correlated_cs - Descriptor for a clocksource correlated to another
+ * clocksource
+ * @related_cs:Pointer to the related timekeeping clocksource
+ * @convert: Conver
[PATCH v4 2/4] Always running timer correlated clocksource
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). In addition, ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 2]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Signed-off-by: Christopher S. Hall
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 48 ++-
3 files changed, 50 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index e6cf2ad..90868a6 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..9474c9c 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct correlated_cs art_timestamper;
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index c3f7602..c3f098c 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -820,7 +820,7 @@ int recalibrate_cpu_khz(void)
#ifndef CONFIG_SMP
unsigned long cpu_khz_old = cpu_khz;
- if (cpu_has_tsc) {
+ if (boot_cpu_has(X86_FEATURE_ART)) {
tsc_khz = x86_platform.calibrate_tsc();
cpu_khz = tsc_khz;
cpu_data(0).loops_per_jiffy =
@@ -940,10 +940,36 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+/* The denominator will never be less that 2 */
+#define ART_MIN_DENOMINATOR (2)
+
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (art_to_tsc_denominator >= ART_MIN_DENOMINATOR)
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ detect_art();
+
if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
return 0;
cpufreq_register_notifier(_cpufreq_notifier_block,
@@ -1062,6 +1088,24 @@ int unsynchronized_tsc(void)
return 0;
}
+/*
+ * Convert ART to TSC given numerator/denominator found in detect_art()
+ */
+static u64 convert_art_to_tsc(struct correlated_cs *cs, u64 cycles)
+{
+ u64 tmp, res;
+
+ res = (cycles / art_to_tsc_denominator) * art_to_tsc_numerator;
+ tmp = (cycles % art_to_tsc_denominator) * art_to_tsc_numerator;
+ res += tmp / art_to_tsc_denominator;
+
+ return res;
+}
+
+struct correlated_cs art_timestamper = {
+ .convert= convert_art_to_tsc,
+};
+EXPORT_SYMBOL(art_timestamper);
static void tsc_refine_calibration_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(tsc_irqwork, tsc_refine_calibration_work);
@@ -1133,6 +1177,8 @@ static void tsc_refine_calibration_work(struct
work_struct *work)
(unsigned long)tsc_khz % 1000);
out:
+ if (boot_cpu_has(X86_FEATURE_ART))
+ art_timestamper.related_cs = _tsc;
clocksource_register_khz(_tsc, tsc_khz);
}
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo
[PATCH v4 3/4] Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping
Currently, network /system cross-timestamping is performed in the
PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday() and the
gettime64() callback provided by the driver. The cross-timestamp is best
effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be significant.
The getsynctime() callback and corresponding PTP_SYS_OFFSET_PRECISE ioctl
allows the driver to perform this device/system correlation when for example
cross timestamp hardware is available. Modern Intel systems can do this for
onboard Ethernet controllers using the ART counter. There is virtually zero
latency between captures of the ART and network device clock.
The capabilities ioctl (PTP_CLOCK_GETCAPS), is augmented allowing
applications to query whether or not drivers implement the
getsynctime callback, providing more precise cross timestamping.
Signed-off-by: Christopher S. Hall
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 26 ++
include/linux/ptp_clock_kernel.h | 6 ++
include/uapi/linux/ptp_clock.h | 12 +++-
4 files changed, 47 insertions(+), 3 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 2bc8abc..8004efd 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -276,13 +276,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..7db6f02 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -120,11 +120,14 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock_caps caps;
struct ptp_clock_request req;
struct ptp_sys_offset *sysoff = NULL;
+ struct ptp_sys_offset_precise precise_offset;
struct ptp_pin_desc pd;
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
struct timespec64 ts;
+ u64 cross_dev, cross_sys;
+ u32 rem;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +141,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getsynctime != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -180,6 +184,28 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
err = ops->enable(ops, , enable);
break;
+ case PTP_SYS_OFFSET_PRECISE:
+ if (!ptp->info->getsynctime || ptp->info->getsynctime(
+ ptp->info, _dev, _sys)) {
+ err = -ENOSYS;
+ break;
+ }
+ if (copy_from_user(_offset, (void __user *)arg,
+ sizeof(precise_offset))) {
+ err = -EFAULT;
+ break;
+ }
+ precise_offset.dev.sec =
+ div_u64_rem(cross_dev, NSEC_PER_SEC, );
+ precise_offset.dev.nsec = rem;
+ precise_offset.sys.sec =
+ div_u64_rem(cross_sys, NSEC_PER_SEC, );
+ precise_offset.sys.nsec = rem;
+ if (copy_to_user((void __user *)arg, _offset,
+sizeof(precise_offset)))
+ err = -EFAULT;
+ break;
+
case PTP_SYS_OFFSET:
sysoff = kmalloc(sizeof(*sysoff), GFP_KERNEL);
if (!sysoff) {
diff --git a/include/linux/ptp_clock_kernel.h b/include/linux/ptp_clock_kernel.h
index b8b7306..eb6fe50 100644
--- a/include/linux/ptp_clock_kernel.h
+++ b/include/linux/ptp_clock_kernel.h
@@ -
[PATCH v4 4/4] Adds hardware supported cross timestamp
Modern Intel systems supports cross timestamping of the network device
clock and Always Running Timer (ART) in hardware. This allows the device
time and system time to be precisely correlated. The timestamp pair is
exposed through the *_get_ts callback used by get_correlated_timestamp().
The hardware cross-timestamp result is made available to applications
through the PTP_SYS_OFFSET_PRECISE ioctl.
Signed-off-by: Christopher S. Hall
---
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 77 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index 133d407..13cff75 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -527,6 +527,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..25e2641 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -25,6 +25,8 @@
*/
#include "e1000.h"
+#include
+#include
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
@@ -98,6 +100,77 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#define MAX_HW_WAIT_COUNT (3)
+
+static int e1000e_phc_get_ts(struct correlated_ts *cts)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)cts->private;
+ struct e1000_hw *hw = >hw;
+ int i;
+ u32 tsync_ctrl;
+ int ret;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT) {
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ cts->system_ts = er32(PLTSTMPH);
+ cts->system_ts <<= 32;
+ cts->system_ts |= er32(PLTSTMPL);
+ cts->device_ts = er32(SYSSTMPH);
+ cts->device_ts <<= 32;
+ cts->device_ts |= er32(SYSSTMPL);
+ }
+
+ return ret;
+}
+
+/**
+ * e1000e_phc_getsynctime - Reads the current time from the hardware clock and
+ * correlated system time
+ * @ptp: ptp clock structure
+ * @devts: timespec structure to hold the current device time value
+ * @systs: timespec structure to hold the current system time value
+ *
+ * Read device and system (ART) clock simultaneously and return the correct
+ * clock values in ns after converting into a struct timespec.
+ **/
+static int e1000e_phc_getsynctime(struct ptp_clock_info *ptp, u64 *dev,
+ u64 *sys )
+{
+ struct e1000_adapter *adapter = container_of(ptp, struct e1000_adapter,
+ptp_clock_info);
+ unsigned long flags;
+ struct correlated_ts art_correlated_ts;
+ int ret;
+
+ art_correlated_ts.get_ts = e1000e_phc_get_ts;
+ art_correlated_ts.private = adapter;
+ ret = get_correlated_timestamp(_correlated_ts,
+ _timestamper);
+ if (ret != 0)
+ return ret;
+
+ *sys = art_correlated_ts.system_real.tv64;
+
+ spin_lock_irqsave(>systim_lock, flags);
+ *dev = timecounter_cyc2time(>tc, art_correlated_ts.device_ts);
+ spin_unlock_irqrestore(>systim_lock, flags);
+
+ return 0;
+}
+
/**
* e1000e_phc_gettime - Reads the current time from the hardware clock
* @ptp: ptp clock structure
@@ -236,6 +309,10 @@ void e1000e_ptp_init(struct e1000_adapter *adapter)
break;
}
+ /* CPU must have ART and GBe must be from Sunrise Point or greater */
+ if (hw->mac.type >= e1000_pch_spt && boot_cpu_has(X86_FEATURE_ART))
+ adapter->ptp_clock_info.getsynctime = e1000e_phc_getsynctime;
+
INIT_DELAYED_WORK(>systim_overflow_work,
e1000e_systim_overflow_work
[PATCH v4 0/4] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
Modern Intel hardware adds an Always Running Timer (ART) that allows the network and audio device clocks to precisely cross timestamp the device clock with the system clock. This allows a precise correlation of the device time and system time. v4 adds a history which enables the audio DSP (a "slow" device) to perform cross-timestamping. Christopher S. Hall (4): Produce system time from correlated clocksource Always running timer correlated clocksource Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping Adds hardware supported cross timestamp Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 2 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/tsc.c | 48 ++- drivers/net/ethernet/intel/e1000e/defines.h | 5 + drivers/net/ethernet/intel/e1000e/ptp.c | 77 +++ drivers/net/ethernet/intel/e1000e/regs.h| 4 + drivers/ptp/ptp_chardev.c | 26 include/linux/clocksource.h | 33 + include/linux/ptp_clock_kernel.h| 6 + include/linux/timekeeping.h | 4 + include/uapi/linux/ptp_clock.h | 12 +- kernel/time/timekeeping.c | 203 +++- 13 files changed, 418 insertions(+), 10 deletions(-) -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v4 0/4] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
Modern Intel hardware adds an Always Running Timer (ART) that allows the network and audio device clocks to precisely cross timestamp the device clock with the system clock. This allows a precise correlation of the device time and system time. v4 adds a history which enables the audio DSP (a "slow" device) to perform cross-timestamping. Christopher S. Hall (4): Produce system time from correlated clocksource Always running timer correlated clocksource Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping Adds hardware supported cross timestamp Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 2 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/tsc.c | 48 ++- drivers/net/ethernet/intel/e1000e/defines.h | 5 + drivers/net/ethernet/intel/e1000e/ptp.c | 77 +++ drivers/net/ethernet/intel/e1000e/regs.h| 4 + drivers/ptp/ptp_chardev.c | 26 include/linux/clocksource.h | 33 + include/linux/ptp_clock_kernel.h| 6 + include/linux/timekeeping.h | 4 + include/uapi/linux/ptp_clock.h | 12 +- kernel/time/timekeeping.c | 203 +++- 13 files changed, 418 insertions(+), 10 deletions(-) -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v4 2/4] Always running timer correlated clocksource
On modern Intel systems TSC is derived from the new Always Running Timer
(ART). In addition, ART can be captured simultaneous to the capture of
audio and network device clocks, allowing a correlation between timebases
to be constructed. Upon capture, the driver converts the captured ART
value to the appropriate system clock using the correlated clocksource
mechanism.
On systems that support ART a new CPUID leaf (0x15) returns parameters
“m” and “n” such that:
TSC_value = (ART_value * m) / n + k [n >= 2]
[k is an offset that can adjusted by a privileged agent. The
IA32_TSC_ADJUST MSR is an example of an interface to adjust k.
See 17.14.4 of the Intel SDM for more details]
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
arch/x86/include/asm/cpufeature.h | 2 +-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 48 ++-
3 files changed, 50 insertions(+), 2 deletions(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index e6cf2ad..90868a6 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 6d7c547..9474c9c 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -29,6 +29,8 @@ static inline cycles_t get_cycles(void)
return rdtsc();
}
+extern struct correlated_cs art_timestamper;
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index c3f7602..c3f098c 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -820,7 +820,7 @@ int recalibrate_cpu_khz(void)
#ifndef CONFIG_SMP
unsigned long cpu_khz_old = cpu_khz;
- if (cpu_has_tsc) {
+ if (boot_cpu_has(X86_FEATURE_ART)) {
tsc_khz = x86_platform.calibrate_tsc();
cpu_khz = tsc_khz;
cpu_data(0).loops_per_jiffy =
@@ -940,10 +940,36 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+/* The denominator will never be less that 2 */
+#define ART_MIN_DENOMINATOR (2)
+
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+
+/*
+ * If ART is present detect the numerator:denominator to convert to TSC
+ */
+static void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (art_to_tsc_denominator >= ART_MIN_DENOMINATOR)
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ detect_art();
+
if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
return 0;
cpufreq_register_notifier(_cpufreq_notifier_block,
@@ -1062,6 +1088,24 @@ int unsynchronized_tsc(void)
return 0;
}
+/*
+ * Convert ART to TSC given numerator/denominator found in detect_art()
+ */
+static u64 convert_art_to_tsc(struct correlated_cs *cs, u64 cycles)
+{
+ u64 tmp, res;
+
+ res = (cycles / art_to_tsc_denominator) * art_to_tsc_numerator;
+ tmp = (cycles % art_to_tsc_denominator) * art_to_tsc_numerator;
+ res += tmp / art_to_tsc_denominator;
+
+ return res;
+}
+
+struct correlated_cs art_timestamper = {
+ .convert= convert_art_to_tsc,
+};
+EXPORT_SYMBOL(art_timestamper);
static void tsc_refine_calibration_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(tsc_irqwork, tsc_refine_calibration_work);
@@ -1133,6 +1177,8 @@ static void tsc_refine_calibration_work(struct
work_struct *work)
(unsigned long)tsc_khz % 1000);
out:
+ if (boot_cpu_has(X86_FEATURE_ART))
+ art_timestamper.related_cs = _tsc;
clocksource_register_khz(_tsc, tsc_khz);
}
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http
[PATCH v4 3/4] Add PTP_SYS_OFFSET_PRECISE for driver crosstimestamping
Currently, network /system cross-timestamping is performed in the
PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday() and the
gettime64() callback provided by the driver. The cross-timestamp is best
effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be significant.
The getsynctime() callback and corresponding PTP_SYS_OFFSET_PRECISE ioctl
allows the driver to perform this device/system correlation when for example
cross timestamp hardware is available. Modern Intel systems can do this for
onboard Ethernet controllers using the ART counter. There is virtually zero
latency between captures of the ART and network device clock.
The capabilities ioctl (PTP_CLOCK_GETCAPS), is augmented allowing
applications to query whether or not drivers implement the
getsynctime callback, providing more precise cross timestamping.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 26 ++
include/linux/ptp_clock_kernel.h | 6 ++
include/uapi/linux/ptp_clock.h | 12 +++-
4 files changed, 47 insertions(+), 3 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 2bc8abc..8004efd 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -276,13 +276,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..7db6f02 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -120,11 +120,14 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock_caps caps;
struct ptp_clock_request req;
struct ptp_sys_offset *sysoff = NULL;
+ struct ptp_sys_offset_precise precise_offset;
struct ptp_pin_desc pd;
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
struct timespec64 ts;
+ u64 cross_dev, cross_sys;
+ u32 rem;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +141,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getsynctime != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -180,6 +184,28 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
err = ops->enable(ops, , enable);
break;
+ case PTP_SYS_OFFSET_PRECISE:
+ if (!ptp->info->getsynctime || ptp->info->getsynctime(
+ ptp->info, _dev, _sys)) {
+ err = -ENOSYS;
+ break;
+ }
+ if (copy_from_user(_offset, (void __user *)arg,
+ sizeof(precise_offset))) {
+ err = -EFAULT;
+ break;
+ }
+ precise_offset.dev.sec =
+ div_u64_rem(cross_dev, NSEC_PER_SEC, );
+ precise_offset.dev.nsec = rem;
+ precise_offset.sys.sec =
+ div_u64_rem(cross_sys, NSEC_PER_SEC, );
+ precise_offset.sys.nsec = rem;
+ if (copy_to_user((void __user *)arg, _offset,
+sizeof(precise_offset)))
+ err = -EFAULT;
+ break;
+
case PTP_SYS_OFFSET:
sysoff = kmalloc(sizeof(*sysoff), GFP_KERNEL);
if (!sysoff) {
diff --git a/include/linux/ptp_clock_kernel.h b/include/linux/ptp_clock_kernel.h
index b8b7306..eb6fe50 100644
--- a/include/linux/ptp_clock_kernel.h
+++ b/include/linux/ptp_clock_k
[PATCH v4 4/4] Adds hardware supported cross timestamp
Modern Intel systems supports cross timestamping of the network device
clock and Always Running Timer (ART) in hardware. This allows the device
time and system time to be precisely correlated. The timestamp pair is
exposed through the *_get_ts callback used by get_correlated_timestamp().
The hardware cross-timestamp result is made available to applications
through the PTP_SYS_OFFSET_PRECISE ioctl.
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 77 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
3 files changed, 86 insertions(+)
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index 133d407..13cff75 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -527,6 +527,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..25e2641 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -25,6 +25,8 @@
*/
#include "e1000.h"
+#include
+#include
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
@@ -98,6 +100,77 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#define MAX_HW_WAIT_COUNT (3)
+
+static int e1000e_phc_get_ts(struct correlated_ts *cts)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)cts->private;
+ struct e1000_hw *hw = >hw;
+ int i;
+ u32 tsync_ctrl;
+ int ret;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT) {
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ cts->system_ts = er32(PLTSTMPH);
+ cts->system_ts <<= 32;
+ cts->system_ts |= er32(PLTSTMPL);
+ cts->device_ts = er32(SYSSTMPH);
+ cts->device_ts <<= 32;
+ cts->device_ts |= er32(SYSSTMPL);
+ }
+
+ return ret;
+}
+
+/**
+ * e1000e_phc_getsynctime - Reads the current time from the hardware clock and
+ * correlated system time
+ * @ptp: ptp clock structure
+ * @devts: timespec structure to hold the current device time value
+ * @systs: timespec structure to hold the current system time value
+ *
+ * Read device and system (ART) clock simultaneously and return the correct
+ * clock values in ns after converting into a struct timespec.
+ **/
+static int e1000e_phc_getsynctime(struct ptp_clock_info *ptp, u64 *dev,
+ u64 *sys )
+{
+ struct e1000_adapter *adapter = container_of(ptp, struct e1000_adapter,
+ptp_clock_info);
+ unsigned long flags;
+ struct correlated_ts art_correlated_ts;
+ int ret;
+
+ art_correlated_ts.get_ts = e1000e_phc_get_ts;
+ art_correlated_ts.private = adapter;
+ ret = get_correlated_timestamp(_correlated_ts,
+ _timestamper);
+ if (ret != 0)
+ return ret;
+
+ *sys = art_correlated_ts.system_real.tv64;
+
+ spin_lock_irqsave(>systim_lock, flags);
+ *dev = timecounter_cyc2time(>tc, art_correlated_ts.device_ts);
+ spin_unlock_irqrestore(>systim_lock, flags);
+
+ return 0;
+}
+
/**
* e1000e_phc_gettime - Reads the current time from the hardware clock
* @ptp: ptp clock structure
@@ -236,6 +309,10 @@ void e1000e_ptp_init(struct e1000_adapter *adapter)
break;
}
+ /* CPU must have ART and GBe must be from Sunrise Point or greater */
+ if (hw->mac.type >= e1000_pch_spt && boot_cpu_has(X86_FEATURE_ART))
+ adapter->ptp_clock_info.getsynctime = e1000e_phc_getsynctime;
+
INIT_DELAYED_WORK(>systim_overflow_work,
[PATCH v4 1/4] Produce system time from correlated clocksource
From: Thomas Gleixner <t...@linutronix.de>
Modern Intel hardware provides the so called Always Running Timer
(ART). The TSC which is usually used for timekeeping is derived from
ART and runs with a fixed frequency ratio to it. ART is routed to
devices and allows to take atomic timestamp samples from the device
clock and the ART. One use case is PTP timestamps on network cards. We
want to utilize this feature as it allows us to better correlate the
PTP timestamp to the system time.
In order to gather precise timestamps we need to make sure that the
conversion from ART to TSC and the following conversion from TSC to
clock realtime happens synchronized with the ongoing timekeeping
updates. Otherwise we might convert an ART timestamp from point A in
time with the conversion factors of point B in time. These conversion
factors can differ due to NTP/PTP frequency adjustments and therefor
the resulting clock realtime timestamp would be slightly off, which is
contrary to the whole purpose of synchronized hardware timestamps.
Provide data structures which describe the correlation between two
clocksources and a function to gather correlated and convert
timestamps from a device. The function is as any other timekeeping
function protected against current timekeeper updates via the
timekeeper sequence lock. It calls the device function to gather the
hardware timestamps and converts them to clock real time and clock
monotonic raw.
Signed-off-by: Thomas Gleixner <t...@linutronix.de>
Another representative use case of time sync and the correlated
clocksource (in addition to PTP noted above) is PTP synchronized
audio.
In a streaming application, as an example, samples will be sent
and/or received by multiple devices with a presentation time that is
in terms of the PTP master clock. Synchronizing the audio output on
these devices requires correlating the audio clock with the PTP
master clock. The more precise this correlation is, the better the
audio quality (i.e. out of sync audio sounds bad).
>From an application standpoint, to correlate the PTP master clock
with the audio device clock, the system clock is used as a
intermediate timebase. The transforms such an application would
perform are:
System Clock <-> Audio clock
System Clock <-> Network Device Clock [<-> PTP Master Clock]
Such audio applications make use of some existing ALSA library
calls that provide audio/system cross-timestamps (e.g.
snd_pcm_status_get_htstamp()). Previous driver implementations
capture these cross by reading the system clock (raw/mono/real)
and the device clock atomically in software.
Modern Intel platforms can perform a more accurate cross-
timestamp in hardware (ART,audio device clock). The audio driver
requires ART->system time transforms -- the same as required for
the network driver. These platforms offload audio processing
(including cross-timestamps) to a DSP which to ensure
uninterrupted audio processing, communicates and response to the
host only once every millsecond. As a result is takes up to a
millisecond for the DSP to receive a request, the request is
processed by the DSP, the audio output hardware is polled for
completion, the result is copied into shared memory, and the
host is notified. All of these operation occur on a millisecond
cadence. This transaction requires about 2 ms, but under
heavier workloads it may take up to 4 ms.
If update_wall_time() is called while waiting for a
response within get_correlated_ts() (from original patch), a retry
is attempted. This will occur if the cycle_interval(determined by
CONFIG_HZ and mult/shift values) cycles elapse.
The modification to the original patch accomodates these
slow devices by adding the option of providing an ART value outside
of the retry loop and adding a history which can consulted in the
case of an out of date counter value. The history is kept by
making the shadow_timekeeper an array. Each write to the
timekeeper rotates through the array, preserving a
history of updates.
With these changes, if get_correlated_timestamp() detects a counter
value previous to cycle_now, it consults the history in
shadow_timekeeper and translates the timestamp to the system time
value. If the timestamp value is too old, an error is returned
Signed-off-by: Christopher S. Hall <christopher.s.h...@intel.com>
---
include/linux/clocksource.h | 33 +++
include/linux/timekeeping.h | 4 +
kernel/time/timekeeping.c | 203 ++--
3 files changed, 235 insertions(+), 5 deletions(-)
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 278dd27..4bedadb 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -258,4 +258,37 @@ void acpi_generic_timer_init(void);
static inline void acpi_generic_timer_init(void) { }
#endif
+/*
+ * struct correlated_cs - Descriptor for a clocksource correlated to another
+ * clocksource
+ * @related_cs:
[PATCH v3 3/4] Add support for driver cross-timestamp to PTP_SYS_OFFSET ioctl
From: Christopher Hall
This patch allows system and device time ("cross-timestamp") to be
performed by the driver. Currently, the cross-timestamping is performed
in the PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday()
and the gettime64() callback provided by the driver. The cross-timestamp
is best effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be
significant.
This patch adds an additional callback getsynctime64(). Which will be
called when the driver is able to perform a more accurate, implementation
specific cross-timestamping. For example, future network devices that
implement PCIE PTM will be able to precisely correlate the device clock
with the system clock with virtually zero latency between captures.
This added callback can be used by the driver to expose this functionality.
The callback, getsynctime64(), will only be called when defined and
n_samples == 1 because the driver returns only 1 cross-timestamp where
multiple samples cannot be chained together.
This patch also adds to the capabilities ioctl (PTP_CLOCK_GETCAPS),
allowing applications to query whether or not drivers implement the
getsynctime callback, providing more precise cross timestamping.
Commit Details:
Added additional callback to ptp_clock_info:
* getsynctime64()
This takes 2 arguments referring to system and device time
With this callback drivers may provide both system time and device time
to ensure precise correlation
Modified PTP_SYS_OFFSET ioctl in PTP clock driver to use the above
callback if it's available
Added capability (PTP_CLOCK_GETCAPS) for checking whether driver supports
cross timestamping
Added check for cross timestamping flag to testptp.c
Signed-off-by: Christopher S. Hall
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 29 +
include/linux/ptp_clock_kernel.h | 7 +++
include/uapi/linux/ptp_clock.h | 4 +++-
4 files changed, 35 insertions(+), 11 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 2bc8abc..8004efd 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -276,13 +276,15 @@ int main(int argc, char *argv[])
" %d external time stamp channels\n"
" %d programmable periodic signals\n"
" %d pulse per second\n"
- " %d programmable pins\n",
+ " %d programmable pins\n"
+ " %d cross timestamping\n",
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..392ccfa 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -124,7 +124,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp->info;
struct ptp_clock_time *pct;
- struct timespec64 ts;
+ struct timespec64 ts, systs;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +138,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp->info->n_per_out;
caps.pps = ptp->info->pps;
caps.n_pins = ptp->info->n_pins;
+ caps.cross_timestamping = ptp->info->getsynctime64 != NULL;
if (copy_to_user((void __user *)arg, , sizeof(caps)))
err = -EFAULT;
break;
@@ -196,19 +197,31 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
break;
}
pct = >ts[0];
- for (i = 0; i < sysoff->n_samples; i++) {
- getnstimeofday64();
+ if (ptp->info->getsynctime64 && sysoff->n_samples == 1 &&
+ ptp->info->getsynctime64(ptp->info, , ) == 0) {
+ pct->sec = systs.tv_sec;
+ pct->nsec = systs.tv_nsec;
+ pct++;
pct->sec = ts.tv_sec;
pct->nsec = ts.tv_nsec;
pct++;
- ptp->info->gettime64(ptp->info, );
+
[PATCH v3 4/4] Enabling hardware supported PTP system/device crosstimestamping
From: Christopher Hall
Add getsynctime() PTP device callback to cross timestamp system device
clock using ART translation depends on platform being >= SPT
and having ART
getsynctime() reads ART (TSC-derived)/device cross timestamp and
converts to realtime/device time reporting cross timestamp to
PTP driver
Signed-off-by: Christopher S. Hall
---
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 88 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
3 files changed, 97 insertions(+)
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index 133d407..13cff75 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -527,6 +527,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..228f3f3 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -25,6 +25,8 @@
*/
#include "e1000.h"
+#include
+#include
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
@@ -98,6 +100,87 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#define MAX_HW_WAIT_COUNT (3)
+
+static int e1000e_phc_get_ts(struct correlated_ts *cts)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)cts->private;
+ struct e1000_hw *hw = >hw;
+ int i;
+ u32 tsync_ctrl;
+ int ret;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i < MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl & E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT) {
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ cts->system_ts = er32(PLTSTMPH);
+ cts->system_ts <<= 32;
+ cts->system_ts |= er32(PLTSTMPL);
+ cts->device_ts = er32(SYSSTMPH);
+ cts->device_ts <<= 32;
+ cts->device_ts |= er32(SYSSTMPL);
+ }
+
+ return ret;
+}
+
+/**
+ * e1000e_phc_getsynctime - Reads the current time from the hardware clock and
+ * correlated system time
+ * @ptp: ptp clock structure
+ * @devts: timespec structure to hold the current device time value
+ * @systs: timespec structure to hold the current system time value
+ *
+ * Read device and system (ART) clock simultaneously and return the correct
+ * clock values in ns after converting into a struct timespec.
+ **/
+static int e1000e_phc_getsynctime(struct ptp_clock_info *ptp,
+ struct timespec64 *devts,
+ struct timespec64 *systs)
+{
+ struct e1000_adapter *adapter = container_of(ptp, struct e1000_adapter,
+ptp_clock_info);
+ unsigned long flags;
+ u32 remainder;
+ struct correlated_ts art_correlated_ts;
+ u64 device_time;
+ int ret;
+
+ art_correlated_ts.get_ts = e1000e_phc_get_ts;
+ art_correlated_ts.private = adapter;
+ ret = get_correlated_timestamp(_correlated_ts,
+ _timestamper);
+ if (ret != 0)
+ goto bail;
+
+ systs->tv_sec =
+ div_u64_rem(art_correlated_ts.system_real.tv64,
+ NSEC_PER_SEC, );
+ systs->tv_nsec = remainder;
+ spin_lock_irqsave(>systim_lock, flags);
+ device_time = timecounter_cyc2time(>tc,
+ art_correlated_ts.device_ts);
+ spin_unlock_irqrestore(>systim_lock, flags);
+ devts->tv_sec =
+ div_u64_rem(device_time, NSEC_PER_SEC, );
+ devts->tv_nsec = remainder;
+
+bail:
+ return ret;
+}
+
/**
* e1000e_phc_gettime - Reads the current time from the hardware clock
* @ptp: ptp clock structure
@@ -190,6 +273,7 @@ static const struct ptp_clock_info e1000e_ptp_clock_info = {
.adjf
[PATCH v3 1/4] Add correlated clocksource deriving system time from an auxiliary clocksource
Add struct correlated_cs with pointer to original clocksource and
function pointer to convert correlated clocksource to the original
Add get_correlated_timestamp() function which given specific correlated_cs
and correlated_ts convert correlated counter value to system time
Signed-off-by: Christopher S. Hall
---
include/linux/clocksource.h | 33 +++
include/linux/timekeeping.h | 4 +++
kernel/time/timekeeping.c | 65 +
3 files changed, 102 insertions(+)
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 278dd27..4bedadb 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -258,4 +258,37 @@ void acpi_generic_timer_init(void);
static inline void acpi_generic_timer_init(void) { }
#endif
+/*
+ * struct correlated_cs - Descriptor for a clocksource correlated to another
+ * clocksource
+ * @related_cs:Pointer to the related timekeeping clocksource
+ * @convert: Conversion function to convert a timestamp from
+ * the correlated clocksource to cycles of the related
+ * timekeeping clocksource
+ */
+struct correlated_cs {
+ struct clocksource *related_cs;
+ u64 (*convert)(struct correlated_cs *cs,
+ u64 cycles);
+};
+
+struct correlated_ts;
+
+/**
+ * struct correlated_ts - Descriptor for taking a correlated time stamp
+ * @get_ts:Function to read out a synced system and device
+ * timestamp
+ * @system_ts: The raw system clock timestamp
+ * @device_ts: The raw device timestamp
+ * @system_real: @system_ts converted to CLOCK_REALTIME
+ * @system_raw:@system_ts converted to CLOCK_MONOTONIC_RAW
+ */
+struct correlated_ts {
+ int (*get_ts)(struct correlated_ts *ts);
+ u64 system_ts;
+ u64 device_ts;
+ ktime_t system_real;
+ ktime_t system_raw;
+ void*private;
+};
#endif /* _LINUX_CLOCKSOURCE_H */
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index 6e191e4..a9e1a2d 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -258,6 +258,10 @@ extern void timekeeping_inject_sleeptime64(struct
timespec64 *delta);
*/
extern void getnstime_raw_and_real(struct timespec *ts_raw,
struct timespec *ts_real);
+struct correlated_ts;
+struct correlated_cs;
+extern int get_correlated_timestamp(struct correlated_ts *crt,
+ struct correlated_cs *crs);
/*
* Persistent clock related interfaces
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index bca3667..90a7c6f 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -312,6 +312,19 @@ static inline s64 timekeeping_get_ns(struct tk_read_base
*tkr)
return nsec + arch_gettimeoffset();
}
+static inline s64 timekeeping_convert_to_ns(struct tk_read_base *tkr,
+ cycle_t cycles)
+{
+ cycle_t delta;
+ s64 nsec;
+
+ /* calculate the delta since the last update_wall_time */
+ delta = clocksource_delta(cycles, tkr->cycle_last, tkr->mask);
+
+ nsec = delta * tkr->mult + tkr->xtime_nsec;
+ return nsec >> tkr->shift;
+}
+
/**
* update_fast_timekeeper - Update the fast and NMI safe monotonic timekeeper.
* @tkr: Timekeeping readout base from which we take the update
@@ -885,6 +898,58 @@ EXPORT_SYMBOL(getnstime_raw_and_real);
#endif /* CONFIG_NTP_PPS */
/**
+ * get_correlated_timestamp - Get a correlated timestamp
+ *
+ * Reads a timestamp from a device and correlates it to system time
+ */
+int get_correlated_timestamp(struct correlated_ts *crt,
+struct correlated_cs *crs)
+{
+ struct timekeeper *tk = _core.timekeeper;
+ unsigned long seq;
+ cycles_t cycles;
+ ktime_t base;
+ s64 nsecs;
+ int ret;
+
+ do {
+ seq = read_seqcount_begin(_core.seq);
+ /*
+* Verify that the correlated clocksoure is related to
+* the currently installed timekeeper clocksoure
+*/
+ if (tk->tkr_mono.clock != crs->related_cs)
+ return -ENODEV;
+
+ /*
+* Try to get a timestamp from the device.
+*/
+ ret = crt->get_ts(crt);
+ if (ret)
+ return ret;
+
+ /*
+* Convert the timestamp to timekeeper clock cycles
+*/
+ cycles = crs->convert(crs, crt->system_ts);
+
+ /* Convert to clock realtime */
[PATCH v3 2/4] Added ART correlated clocksource and ART CPU feature
Add detect_art() call to early TSC initialization which reads ART->TSC
numerator/denominator and sets CPU feature if present
Add convert_art_to_tsc() function performing conversion ART to TSC
Add art_timestamp referencing art_to_tsc() and clocksource_tsc enabling
driver conversion of ART to TSC
Signed-off-by: Christopher S. Hall
---
arch/x86/include/asm/cpufeature.h | 3 ++-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 54 +++
3 files changed, 58 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 3d6606f..a9322e5 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* "" P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * "" FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
@@ -352,6 +352,7 @@ extern const char * const x86_bug_flags[NBUGINTS*32];
#define cpu_has_de boot_cpu_has(X86_FEATURE_DE)
#define cpu_has_pseboot_cpu_has(X86_FEATURE_PSE)
#define cpu_has_tscboot_cpu_has(X86_FEATURE_TSC)
+#define cpu_has_artboot_cpu_has(X86_FEATURE_ART)
#define cpu_has_pgeboot_cpu_has(X86_FEATURE_PGE)
#define cpu_has_apic boot_cpu_has(X86_FEATURE_APIC)
#define cpu_has_sepboot_cpu_has(X86_FEATURE_SEP)
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 94605c0..8d52d91 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -45,6 +45,8 @@ static __always_inline cycles_t vget_cycles(void)
return (cycles_t)__native_read_tsc();
}
+extern struct correlated_cs art_timestamper;
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 7437b41..13f12e0 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -939,10 +939,36 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+#define ART_MIN_DENOMINATOR (2)
+
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+
+/*
+ * If ART is present detect the numberator:denominator to convert to TSC
+ */
+void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level >= ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, _to_tsc_denominator,
+ _to_tsc_numerator, unused, unused+1);
+
+ if (art_to_tsc_denominator >= ART_MIN_DENOMINATOR) {
+ set_cpu_cap(_cpu_data, X86_FEATURE_ART);
+ }
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ detect_art();
+
if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
return 0;
cpufreq_register_notifier(_cpufreq_notifier_block,
@@ -1059,6 +1085,32 @@ int unsynchronized_tsc(void)
return 0;
}
+/*
+ * Convert ART to TSC given numerator/denominator found in detect_art()
+ */
+static u64 convert_art_to_tsc(struct correlated_cs *cs, u64 cycles)
+{
+ u64 tmp, res;
+
+ switch (art_to_tsc_denominator) {
+ default:
+ res = (cycles / art_to_tsc_denominator) * art_to_tsc_numerator;
+ tmp = (cycles % art_to_tsc_denominator) * art_to_tsc_numerator;
+ res += tmp / art_to_tsc_denominator;
+ break;
+ case 2:
+ res = (cycles >> 1) * art_to_tsc_numerator;
+ tmp = (cycles & 0x1) * art_to_tsc_numerator;
+ res += tmp >> 1;
+ break;
+ }
+ return res;
+}
+
+struct correlated_cs art_timestamper = {
+ .convert= convert_art_to_tsc,
+};
+EXPORT_SYMBOL(art_timestamper);
static void tsc_refine_calibration_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(tsc_irqwork, tsc_refine_calibration_work);
@@ -1130,6 +1182,8 @@ static void tsc_refine_calibration_work(struct
work_struct *work)
(unsigned long)tsc_khz % 1000);
out:
+ if (cpu_has_art)
+ art_timestamper.related_cs = _tsc;
clocksource_register_khz(_tsc, tsc_khz);
}
--
2.1.4
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
[PATCH v3 0/4] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
6th generation Intel platforms will have an Always Running Timer (ART) that always runs when the system is powered and is available to both the CPU and various on-board devices. Initially, those devices include audio and network. The ART will give these devices the capability of precisely cross timestamping their local device clock with the system clock. The ART is precisely related to the TSC by a ratio read from CPUID leaf 0x15. A device (such as the network controller) produces cross timestamps in terms of the ART and the local device clock. The ART value on its own isn't useful. The first two patches enable translation of ART to system time. The first patch adds the correlated clocksource concept which is an auxiliary clock directly relate-able to a clock registered as a clocksource. The second patch adds the Intel specific ART correlated clocksource. The last two patches modify the PTP character driver to call a cross timestamp function (getsynctime()) in the driver when available and perform the cross timestamp in the e1000e driver. The patches taken together enable sub-microsecond cross timestamps between the system clock and network device clock Changelog since v2: Split out x86 architecture specific code from common timekeeping code additions Split ART initialization between early TSC initialization and TSC frequency refinement. Now, cpu_has_art can be used in driver initialization code Added e1000e PTP init code that detects presence of ART/spt disabling cross timestamp if they're not available Added additional commenting in TSC/ART init code, minor renaming of functions and variables for greater clarity Fixed a few formatting problems in e1000e driver patch Christopher Hall (2): Add support for driver cross-timestamp to PTP_SYS_OFFSET ioctl Enabling hardware supported PTP system/device crosstimestamping Christopher S. Hall (2): Add correlated clocksource deriving system time from an auxiliary clocksource Added ART correlated clocksource and ART CPU feature Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 3 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/tsc.c | 54 ++ drivers/net/ethernet/intel/e1000e/defines.h | 5 ++ drivers/net/ethernet/intel/e1000e/ptp.c | 88 + drivers/net/ethernet/intel/e1000e/regs.h| 4 ++ drivers/ptp/ptp_chardev.c | 29 +++--- include/linux/clocksource.h | 33 +++ include/linux/ptp_clock_kernel.h| 7 +++ include/linux/timekeeping.h | 4 ++ include/uapi/linux/ptp_clock.h | 4 +- kernel/time/timekeeping.c | 65 + 13 files changed, 292 insertions(+), 12 deletions(-) -- 2.1.4 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH v3 2/4] Added ART correlated clocksource and ART CPU feature
Add detect_art() call to early TSC initialization which reads ART-TSC
numerator/denominator and sets CPU feature if present
Add convert_art_to_tsc() function performing conversion ART to TSC
Add art_timestamp referencing art_to_tsc() and clocksource_tsc enabling
driver conversion of ART to TSC
Signed-off-by: Christopher S. Hall christopher.s.h...@intel.com
---
arch/x86/include/asm/cpufeature.h | 3 ++-
arch/x86/include/asm/tsc.h| 2 ++
arch/x86/kernel/tsc.c | 54 +++
3 files changed, 58 insertions(+), 1 deletion(-)
diff --git a/arch/x86/include/asm/cpufeature.h
b/arch/x86/include/asm/cpufeature.h
index 3d6606f..a9322e5 100644
--- a/arch/x86/include/asm/cpufeature.h
+++ b/arch/x86/include/asm/cpufeature.h
@@ -85,7 +85,7 @@
#define X86_FEATURE_P4 ( 3*32+ 7) /* P4 */
#define X86_FEATURE_CONSTANT_TSC ( 3*32+ 8) /* TSC ticks at a constant rate */
#define X86_FEATURE_UP ( 3*32+ 9) /* smp kernel running on up */
-/* free, was #define X86_FEATURE_FXSAVE_LEAK ( 3*32+10) * FXSAVE leaks
FOP/FIP/FOP */
+#define X86_FEATURE_ART(3*32+10) /* Platform has always
running timer (ART) */
#define X86_FEATURE_ARCH_PERFMON ( 3*32+11) /* Intel Architectural PerfMon */
#define X86_FEATURE_PEBS ( 3*32+12) /* Precise-Event Based Sampling */
#define X86_FEATURE_BTS( 3*32+13) /* Branch Trace Store */
@@ -352,6 +352,7 @@ extern const char * const x86_bug_flags[NBUGINTS*32];
#define cpu_has_de boot_cpu_has(X86_FEATURE_DE)
#define cpu_has_pseboot_cpu_has(X86_FEATURE_PSE)
#define cpu_has_tscboot_cpu_has(X86_FEATURE_TSC)
+#define cpu_has_artboot_cpu_has(X86_FEATURE_ART)
#define cpu_has_pgeboot_cpu_has(X86_FEATURE_PGE)
#define cpu_has_apic boot_cpu_has(X86_FEATURE_APIC)
#define cpu_has_sepboot_cpu_has(X86_FEATURE_SEP)
diff --git a/arch/x86/include/asm/tsc.h b/arch/x86/include/asm/tsc.h
index 94605c0..8d52d91 100644
--- a/arch/x86/include/asm/tsc.h
+++ b/arch/x86/include/asm/tsc.h
@@ -45,6 +45,8 @@ static __always_inline cycles_t vget_cycles(void)
return (cycles_t)__native_read_tsc();
}
+extern struct correlated_cs art_timestamper;
+
extern void tsc_init(void);
extern void mark_tsc_unstable(char *reason);
extern int unsynchronized_tsc(void);
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 7437b41..13f12e0 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -939,10 +939,36 @@ static struct notifier_block time_cpufreq_notifier_block
= {
.notifier_call = time_cpufreq_notifier
};
+#define ART_CPUID_LEAF (0x15)
+#define ART_MIN_DENOMINATOR (2)
+
+static u32 art_to_tsc_numerator;
+static u32 art_to_tsc_denominator;
+
+/*
+ * If ART is present detect the numberator:denominator to convert to TSC
+ */
+void detect_art(void)
+{
+ unsigned int unused[2];
+
+ if (boot_cpu_data.cpuid_level = ART_CPUID_LEAF) {
+ cpuid(ART_CPUID_LEAF, art_to_tsc_denominator,
+ art_to_tsc_numerator, unused, unused+1);
+
+ if (art_to_tsc_denominator = ART_MIN_DENOMINATOR) {
+ set_cpu_cap(boot_cpu_data, X86_FEATURE_ART);
+ }
+ }
+}
+
static int __init cpufreq_tsc(void)
{
if (!cpu_has_tsc)
return 0;
+
+ detect_art();
+
if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC))
return 0;
cpufreq_register_notifier(time_cpufreq_notifier_block,
@@ -1059,6 +1085,32 @@ int unsynchronized_tsc(void)
return 0;
}
+/*
+ * Convert ART to TSC given numerator/denominator found in detect_art()
+ */
+static u64 convert_art_to_tsc(struct correlated_cs *cs, u64 cycles)
+{
+ u64 tmp, res;
+
+ switch (art_to_tsc_denominator) {
+ default:
+ res = (cycles / art_to_tsc_denominator) * art_to_tsc_numerator;
+ tmp = (cycles % art_to_tsc_denominator) * art_to_tsc_numerator;
+ res += tmp / art_to_tsc_denominator;
+ break;
+ case 2:
+ res = (cycles 1) * art_to_tsc_numerator;
+ tmp = (cycles 0x1) * art_to_tsc_numerator;
+ res += tmp 1;
+ break;
+ }
+ return res;
+}
+
+struct correlated_cs art_timestamper = {
+ .convert= convert_art_to_tsc,
+};
+EXPORT_SYMBOL(art_timestamper);
static void tsc_refine_calibration_work(struct work_struct *work);
static DECLARE_DELAYED_WORK(tsc_irqwork, tsc_refine_calibration_work);
@@ -1130,6 +1182,8 @@ static void tsc_refine_calibration_work(struct
work_struct *work)
(unsigned long)tsc_khz % 1000);
out:
+ if (cpu_has_art)
+ art_timestamper.related_cs = clocksource_tsc;
clocksource_register_khz(clocksource_tsc, tsc_khz);
}
--
2.1.4
--
To unsubscribe from this list: send the line unsubscribe linux-kernel in
the body
[PATCH v3 3/4] Add support for driver cross-timestamp to PTP_SYS_OFFSET ioctl
From: Christopher Hall christopher.s.h...@intel.com
This patch allows system and device time (cross-timestamp) to be
performed by the driver. Currently, the cross-timestamping is performed
in the PTP_SYS_OFFSET ioctl. The PTP clock driver reads gettimeofday()
and the gettime64() callback provided by the driver. The cross-timestamp
is best effort where the latency between the capture of system time
(getnstimeofday()) and the device time (driver callback) may be
significant.
This patch adds an additional callback getsynctime64(). Which will be
called when the driver is able to perform a more accurate, implementation
specific cross-timestamping. For example, future network devices that
implement PCIE PTM will be able to precisely correlate the device clock
with the system clock with virtually zero latency between captures.
This added callback can be used by the driver to expose this functionality.
The callback, getsynctime64(), will only be called when defined and
n_samples == 1 because the driver returns only 1 cross-timestamp where
multiple samples cannot be chained together.
This patch also adds to the capabilities ioctl (PTP_CLOCK_GETCAPS),
allowing applications to query whether or not drivers implement the
getsynctime callback, providing more precise cross timestamping.
Commit Details:
Added additional callback to ptp_clock_info:
* getsynctime64()
This takes 2 arguments referring to system and device time
With this callback drivers may provide both system time and device time
to ensure precise correlation
Modified PTP_SYS_OFFSET ioctl in PTP clock driver to use the above
callback if it's available
Added capability (PTP_CLOCK_GETCAPS) for checking whether driver supports
cross timestamping
Added check for cross timestamping flag to testptp.c
Signed-off-by: Christopher S. Hall christopher.s.h...@intel.com
---
Documentation/ptp/testptp.c | 6 --
drivers/ptp/ptp_chardev.c| 29 +
include/linux/ptp_clock_kernel.h | 7 +++
include/uapi/linux/ptp_clock.h | 4 +++-
4 files changed, 35 insertions(+), 11 deletions(-)
diff --git a/Documentation/ptp/testptp.c b/Documentation/ptp/testptp.c
index 2bc8abc..8004efd 100644
--- a/Documentation/ptp/testptp.c
+++ b/Documentation/ptp/testptp.c
@@ -276,13 +276,15 @@ int main(int argc, char *argv[])
%d external time stamp channels\n
%d programmable periodic signals\n
%d pulse per second\n
-%d programmable pins\n,
+%d programmable pins\n
+%d cross timestamping\n,
caps.max_adj,
caps.n_alarm,
caps.n_ext_ts,
caps.n_per_out,
caps.pps,
- caps.n_pins);
+ caps.n_pins,
+ caps.cross_timestamping);
}
}
diff --git a/drivers/ptp/ptp_chardev.c b/drivers/ptp/ptp_chardev.c
index da7bae9..392ccfa 100644
--- a/drivers/ptp/ptp_chardev.c
+++ b/drivers/ptp/ptp_chardev.c
@@ -124,7 +124,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
struct ptp_clock *ptp = container_of(pc, struct ptp_clock, clock);
struct ptp_clock_info *ops = ptp-info;
struct ptp_clock_time *pct;
- struct timespec64 ts;
+ struct timespec64 ts, systs;
int enable, err = 0;
unsigned int i, pin_index;
@@ -138,6 +138,7 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
caps.n_per_out = ptp-info-n_per_out;
caps.pps = ptp-info-pps;
caps.n_pins = ptp-info-n_pins;
+ caps.cross_timestamping = ptp-info-getsynctime64 != NULL;
if (copy_to_user((void __user *)arg, caps, sizeof(caps)))
err = -EFAULT;
break;
@@ -196,19 +197,31 @@ long ptp_ioctl(struct posix_clock *pc, unsigned int cmd,
unsigned long arg)
break;
}
pct = sysoff-ts[0];
- for (i = 0; i sysoff-n_samples; i++) {
- getnstimeofday64(ts);
+ if (ptp-info-getsynctime64 sysoff-n_samples == 1
+ ptp-info-getsynctime64(ptp-info, ts, systs) == 0) {
+ pct-sec = systs.tv_sec;
+ pct-nsec = systs.tv_nsec;
+ pct++;
pct-sec = ts.tv_sec;
pct-nsec = ts.tv_nsec;
pct++;
- ptp-info-gettime64(ptp-info, ts);
+ pct-sec = systs.tv_sec;
+ pct-nsec = systs.tv_nsec;
+ } else {
+ for (i = 0; i
[PATCH v3 4/4] Enabling hardware supported PTP system/device crosstimestamping
From: Christopher Hall christopher.s.h...@intel.com
Add getsynctime() PTP device callback to cross timestamp system device
clock using ART translation depends on platform being = SPT
and having ART
getsynctime() reads ART (TSC-derived)/device cross timestamp and
converts to realtime/device time reporting cross timestamp to
PTP driver
Signed-off-by: Christopher S. Hall christopher.s.h...@intel.com
---
drivers/net/ethernet/intel/e1000e/defines.h | 5 ++
drivers/net/ethernet/intel/e1000e/ptp.c | 88 +
drivers/net/ethernet/intel/e1000e/regs.h| 4 ++
3 files changed, 97 insertions(+)
diff --git a/drivers/net/ethernet/intel/e1000e/defines.h
b/drivers/net/ethernet/intel/e1000e/defines.h
index 133d407..13cff75 100644
--- a/drivers/net/ethernet/intel/e1000e/defines.h
+++ b/drivers/net/ethernet/intel/e1000e/defines.h
@@ -527,6 +527,11 @@
#define E1000_RXCW_C 0x2000/* Receive config */
#define E1000_RXCW_SYNCH 0x4000/* Receive config synch */
+/* HH Time Sync */
+#define E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK 0xF000 /* max delay */
+#define E1000_TSYNCTXCTL_SYNC_COMP 0x4000 /* sync complete */
+#define E1000_TSYNCTXCTL_START_SYNC0x8000 /* initiate sync */
+
#define E1000_TSYNCTXCTL_VALID 0x0001 /* Tx timestamp valid */
#define E1000_TSYNCTXCTL_ENABLED 0x0010 /* enable Tx timestamping */
diff --git a/drivers/net/ethernet/intel/e1000e/ptp.c
b/drivers/net/ethernet/intel/e1000e/ptp.c
index 25a0ad5..228f3f3 100644
--- a/drivers/net/ethernet/intel/e1000e/ptp.c
+++ b/drivers/net/ethernet/intel/e1000e/ptp.c
@@ -25,6 +25,8 @@
*/
#include e1000.h
+#include asm/tsc.h
+#include linux/timekeeping.h
/**
* e1000e_phc_adjfreq - adjust the frequency of the hardware clock
@@ -98,6 +100,87 @@ static int e1000e_phc_adjtime(struct ptp_clock_info *ptp,
s64 delta)
return 0;
}
+#define MAX_HW_WAIT_COUNT (3)
+
+static int e1000e_phc_get_ts(struct correlated_ts *cts)
+{
+ struct e1000_adapter *adapter = (struct e1000_adapter *)cts-private;
+ struct e1000_hw *hw = adapter-hw;
+ int i;
+ u32 tsync_ctrl;
+ int ret;
+
+ tsync_ctrl = er32(TSYNCTXCTL);
+ tsync_ctrl |= E1000_TSYNCTXCTL_START_SYNC |
+ E1000_TSYNCTXCTL_MAX_ALLOWED_DLY_MASK;
+ ew32(TSYNCTXCTL, tsync_ctrl);
+ for (i = 0; i MAX_HW_WAIT_COUNT; ++i) {
+ udelay(1);
+ tsync_ctrl = er32(TSYNCTXCTL);
+ if (tsync_ctrl E1000_TSYNCTXCTL_SYNC_COMP)
+ break;
+ }
+
+ if (i == MAX_HW_WAIT_COUNT) {
+ ret = -ETIMEDOUT;
+ } else {
+ ret = 0;
+ cts-system_ts = er32(PLTSTMPH);
+ cts-system_ts = 32;
+ cts-system_ts |= er32(PLTSTMPL);
+ cts-device_ts = er32(SYSSTMPH);
+ cts-device_ts = 32;
+ cts-device_ts |= er32(SYSSTMPL);
+ }
+
+ return ret;
+}
+
+/**
+ * e1000e_phc_getsynctime - Reads the current time from the hardware clock and
+ * correlated system time
+ * @ptp: ptp clock structure
+ * @devts: timespec structure to hold the current device time value
+ * @systs: timespec structure to hold the current system time value
+ *
+ * Read device and system (ART) clock simultaneously and return the correct
+ * clock values in ns after converting into a struct timespec.
+ **/
+static int e1000e_phc_getsynctime(struct ptp_clock_info *ptp,
+ struct timespec64 *devts,
+ struct timespec64 *systs)
+{
+ struct e1000_adapter *adapter = container_of(ptp, struct e1000_adapter,
+ptp_clock_info);
+ unsigned long flags;
+ u32 remainder;
+ struct correlated_ts art_correlated_ts;
+ u64 device_time;
+ int ret;
+
+ art_correlated_ts.get_ts = e1000e_phc_get_ts;
+ art_correlated_ts.private = adapter;
+ ret = get_correlated_timestamp(art_correlated_ts,
+ art_timestamper);
+ if (ret != 0)
+ goto bail;
+
+ systs-tv_sec =
+ div_u64_rem(art_correlated_ts.system_real.tv64,
+ NSEC_PER_SEC, remainder);
+ systs-tv_nsec = remainder;
+ spin_lock_irqsave(adapter-systim_lock, flags);
+ device_time = timecounter_cyc2time(adapter-tc,
+ art_correlated_ts.device_ts);
+ spin_unlock_irqrestore(adapter-systim_lock, flags);
+ devts-tv_sec =
+ div_u64_rem(device_time, NSEC_PER_SEC, remainder);
+ devts-tv_nsec = remainder;
+
+bail:
+ return ret;
+}
+
/**
* e1000e_phc_gettime - Reads the current time from the hardware clock
* @ptp: ptp clock structure
@@ -190,6 +273,7 @@ static const struct ptp_clock_info
[PATCH v3 1/4] Add correlated clocksource deriving system time from an auxiliary clocksource
Add struct correlated_cs with pointer to original clocksource and
function pointer to convert correlated clocksource to the original
Add get_correlated_timestamp() function which given specific correlated_cs
and correlated_ts convert correlated counter value to system time
Signed-off-by: Christopher S. Hall christopher.s.h...@intel.com
---
include/linux/clocksource.h | 33 +++
include/linux/timekeeping.h | 4 +++
kernel/time/timekeeping.c | 65 +
3 files changed, 102 insertions(+)
diff --git a/include/linux/clocksource.h b/include/linux/clocksource.h
index 278dd27..4bedadb 100644
--- a/include/linux/clocksource.h
+++ b/include/linux/clocksource.h
@@ -258,4 +258,37 @@ void acpi_generic_timer_init(void);
static inline void acpi_generic_timer_init(void) { }
#endif
+/*
+ * struct correlated_cs - Descriptor for a clocksource correlated to another
+ * clocksource
+ * @related_cs:Pointer to the related timekeeping clocksource
+ * @convert: Conversion function to convert a timestamp from
+ * the correlated clocksource to cycles of the related
+ * timekeeping clocksource
+ */
+struct correlated_cs {
+ struct clocksource *related_cs;
+ u64 (*convert)(struct correlated_cs *cs,
+ u64 cycles);
+};
+
+struct correlated_ts;
+
+/**
+ * struct correlated_ts - Descriptor for taking a correlated time stamp
+ * @get_ts:Function to read out a synced system and device
+ * timestamp
+ * @system_ts: The raw system clock timestamp
+ * @device_ts: The raw device timestamp
+ * @system_real: @system_ts converted to CLOCK_REALTIME
+ * @system_raw:@system_ts converted to CLOCK_MONOTONIC_RAW
+ */
+struct correlated_ts {
+ int (*get_ts)(struct correlated_ts *ts);
+ u64 system_ts;
+ u64 device_ts;
+ ktime_t system_real;
+ ktime_t system_raw;
+ void*private;
+};
#endif /* _LINUX_CLOCKSOURCE_H */
diff --git a/include/linux/timekeeping.h b/include/linux/timekeeping.h
index 6e191e4..a9e1a2d 100644
--- a/include/linux/timekeeping.h
+++ b/include/linux/timekeeping.h
@@ -258,6 +258,10 @@ extern void timekeeping_inject_sleeptime64(struct
timespec64 *delta);
*/
extern void getnstime_raw_and_real(struct timespec *ts_raw,
struct timespec *ts_real);
+struct correlated_ts;
+struct correlated_cs;
+extern int get_correlated_timestamp(struct correlated_ts *crt,
+ struct correlated_cs *crs);
/*
* Persistent clock related interfaces
diff --git a/kernel/time/timekeeping.c b/kernel/time/timekeeping.c
index bca3667..90a7c6f 100644
--- a/kernel/time/timekeeping.c
+++ b/kernel/time/timekeeping.c
@@ -312,6 +312,19 @@ static inline s64 timekeeping_get_ns(struct tk_read_base
*tkr)
return nsec + arch_gettimeoffset();
}
+static inline s64 timekeeping_convert_to_ns(struct tk_read_base *tkr,
+ cycle_t cycles)
+{
+ cycle_t delta;
+ s64 nsec;
+
+ /* calculate the delta since the last update_wall_time */
+ delta = clocksource_delta(cycles, tkr-cycle_last, tkr-mask);
+
+ nsec = delta * tkr-mult + tkr-xtime_nsec;
+ return nsec tkr-shift;
+}
+
/**
* update_fast_timekeeper - Update the fast and NMI safe monotonic timekeeper.
* @tkr: Timekeeping readout base from which we take the update
@@ -885,6 +898,58 @@ EXPORT_SYMBOL(getnstime_raw_and_real);
#endif /* CONFIG_NTP_PPS */
/**
+ * get_correlated_timestamp - Get a correlated timestamp
+ *
+ * Reads a timestamp from a device and correlates it to system time
+ */
+int get_correlated_timestamp(struct correlated_ts *crt,
+struct correlated_cs *crs)
+{
+ struct timekeeper *tk = tk_core.timekeeper;
+ unsigned long seq;
+ cycles_t cycles;
+ ktime_t base;
+ s64 nsecs;
+ int ret;
+
+ do {
+ seq = read_seqcount_begin(tk_core.seq);
+ /*
+* Verify that the correlated clocksoure is related to
+* the currently installed timekeeper clocksoure
+*/
+ if (tk-tkr_mono.clock != crs-related_cs)
+ return -ENODEV;
+
+ /*
+* Try to get a timestamp from the device.
+*/
+ ret = crt-get_ts(crt);
+ if (ret)
+ return ret;
+
+ /*
+* Convert the timestamp to timekeeper clock cycles
+*/
+ cycles = crs-convert(crs, crt-system_ts);
+
+ /* Convert to clock realtime */
+ base
[PATCH v3 0/4] Patchset enabling hardware based cross-timestamps for next gen Intel platforms
6th generation Intel platforms will have an Always Running Timer (ART) that always runs when the system is powered and is available to both the CPU and various on-board devices. Initially, those devices include audio and network. The ART will give these devices the capability of precisely cross timestamping their local device clock with the system clock. The ART is precisely related to the TSC by a ratio read from CPUID leaf 0x15. A device (such as the network controller) produces cross timestamps in terms of the ART and the local device clock. The ART value on its own isn't useful. The first two patches enable translation of ART to system time. The first patch adds the correlated clocksource concept which is an auxiliary clock directly relate-able to a clock registered as a clocksource. The second patch adds the Intel specific ART correlated clocksource. The last two patches modify the PTP character driver to call a cross timestamp function (getsynctime()) in the driver when available and perform the cross timestamp in the e1000e driver. The patches taken together enable sub-microsecond cross timestamps between the system clock and network device clock Changelog since v2: Split out x86 architecture specific code from common timekeeping code additions Split ART initialization between early TSC initialization and TSC frequency refinement. Now, cpu_has_art can be used in driver initialization code Added e1000e PTP init code that detects presence of ART/spt disabling cross timestamp if they're not available Added additional commenting in TSC/ART init code, minor renaming of functions and variables for greater clarity Fixed a few formatting problems in e1000e driver patch Christopher Hall (2): Add support for driver cross-timestamp to PTP_SYS_OFFSET ioctl Enabling hardware supported PTP system/device crosstimestamping Christopher S. Hall (2): Add correlated clocksource deriving system time from an auxiliary clocksource Added ART correlated clocksource and ART CPU feature Documentation/ptp/testptp.c | 6 +- arch/x86/include/asm/cpufeature.h | 3 +- arch/x86/include/asm/tsc.h | 2 + arch/x86/kernel/tsc.c | 54 ++ drivers/net/ethernet/intel/e1000e/defines.h | 5 ++ drivers/net/ethernet/intel/e1000e/ptp.c | 88 + drivers/net/ethernet/intel/e1000e/regs.h| 4 ++ drivers/ptp/ptp_chardev.c | 29 +++--- include/linux/clocksource.h | 33 +++ include/linux/ptp_clock_kernel.h| 7 +++ include/linux/timekeeping.h | 4 ++ include/uapi/linux/ptp_clock.h | 4 +- kernel/time/timekeeping.c | 65 + 13 files changed, 292 insertions(+), 12 deletions(-) -- 2.1.4 -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
