Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Miroslav Lichvar]

On Thu, Jun 15, 2017 at 08:58:40AM -0400, Chris Perl wrote:
> On Thu, Jun 15, 2017 at 8:48 AM, Miroslav Lichvar  wrote:
> > It works even with <7.4 kernels if there is only one interface which
> > supports HW timestamping as SOF_TIMESTAMPING_OPT_CMSG is not required
> > in such case.
> 
> Oh, exciting, thanks!
> 
> Does that mean exactly one up interface that supports HWTSTAMP_FILTER_ALL?
> 
> Meaning, would I be ok if the other interfaces on the machine
> supported things like HWTSTAMP_FILTER_PTP_V1_L4_SYNC,
> HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ, etc, but did not support
> HWTSTAMP_FILTER_ALL?

No, that won't work. The filter applies to received packets. If an
interface doesn't have HWTSTAMP_FILTER_ALL, it can still generate
transmit timestamps and chronyd won't know from which interface it
came.

Please note that the OPT_CMSG option is needed only for IPv4. If you
could use NTP over IPv6, that should work even with multiple
interfaces.

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Chris Perl]

On Thu, Jun 15, 2017 at 8:48 AM, Miroslav Lichvar  wrote:
> I doubt they will be added in 6, but 7.4 should have the OPT_CMSG
> option. Some minor changes in the chrony code will still be necessary
> in order to enable HW timestamping.

Yea, I wasn't expecting 6 to ever support it.

> Here is a copr repo for Fedora and CentOS 7 which has a patch to
> enable HW timestamping.
> https://copr.fedorainfracloud.org/coprs/mlichvar/chrony/
>
> It works even with <7.4 kernels if there is only one interface which
> supports HW timestamping as SOF_TIMESTAMPING_OPT_CMSG is not required
> in such case.

Oh, exciting, thanks!

Does that mean exactly one up interface that supports HWTSTAMP_FILTER_ALL?

Meaning, would I be ok if the other interfaces on the machine
supported things like HWTSTAMP_FILTER_PTP_V1_L4_SYNC,
HWTSTAMP_FILTER_PTP_V1_L4_DELAY_REQ, etc, but did not support
HWTSTAMP_FILTER_ALL?

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Miroslav Lichvar]

On Thu, Jun 15, 2017 at 07:50:07AM -0400, Chris Perl wrote:
> I have played around with chrony 3.1 on both CentOS 6.8 and CentOS 7.3
> (built myself) and both were missing kernel features necessary to allow the
> hardware timestamping to work (I believe in 6 it was lack of
> `SO_SELECT_ERR_QUEUE' and `SOF_TIMESTAMPING_OPT_CMSG', but in 7 it was just
> the latter).

I doubt they will be added in 6, but 7.4 should have the OPT_CMSG
option. Some minor changes in the chrony code will still be necessary
in order to enable HW timestamping.

Here is a copr repo for Fedora and CentOS 7 which has a patch to
enable HW timestamping.
https://copr.fedorainfracloud.org/coprs/mlichvar/chrony/

It works even with <7.4 kernels if there is only one interface which
supports HW timestamping as SOF_TIMESTAMPING_OPT_CMSG is not required
in such case.

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Chris Perl]

On Fri, Jun 9, 2017 at 3:29 AM, Miroslav Lichvar 
wrote:

> Note that in 7.4 will be chrony-3.1, which supports HW timestamping.
> If you don't have switches with PTP support and the NIC on the machine
> can timestamp all packets (not just PTP packets), NTP might work
> better for you.
>

Sorry for taking a slight aside here.  Does that also mean that 7.4 is
going to ship with the necessary kernel changes backported to allow
hardware timestamping to work?

I have played around with chrony 3.1 on both CentOS 6.8 and CentOS 7.3
(built myself) and both were missing kernel features necessary to allow the
hardware timestamping to work (I believe in 6 it was lack of
`SO_SELECT_ERR_QUEUE' and `SOF_TIMESTAMPING_OPT_CMSG', but in 7 it was just
the latter).

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Denny Page]

At the moment, I would unfortunately have to say no. The vendor has committed 
to fixing the issue, but it’s been outstanding for 7-8 months now.

I can certainly recommend the LeoNTP units. They are incredibly stable: my 
dedicated monitoring system (i210) is current tracking the Leo units at an 
average standard deviation of ~33ns for locally attached (100Mb), and ~36ns for 
across a switch (1Gb). Even my server (i211) holds ~42ns across the switch. 
Good gear.

Denny



> On Jun 13, 2017, at 06:01, Miroslav Lichvar  wrote:
> 
> On Fri, Jun 09, 2017 at 07:31:24AM -0700, Denny Page wrote:
>> A great deal depends upon the server side ntp implementation which is why I 
>> asked which device he was using. I have several ntp devices under test, 
>> including one that also supports ptp. While quite accurate in the service of 
>> ptp, it's service of ntp has errors of up to 3ms. I have been working with 
>> the vendor on the issue. 
> 
> Beside the LeoNTP unit, are there any NTP devices using their current
> firmware versions that you would recommend?


--
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Miroslav Lichvar]

On Fri, Jun 09, 2017 at 07:31:24AM -0700, Denny Page wrote:
> A great deal depends upon the server side ntp implementation which is why I 
> asked which device he was using. I have several ntp devices under test, 
> including one that also supports ptp. While quite accurate in the service of 
> ptp, it's service of ntp has errors of up to 3ms. I have been working with 
> the vendor on the issue. 

Beside the LeoNTP unit, are there any NTP devices using their current
firmware versions that you would recommend?

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Bill Unruh]

t has a problem in that while one is being servicec, the other is being held
off destroying its timing if you are not careful.

William G. Unruh __| Canadian Institute for| Tel: +1(604)822-3273
Physics&Astronomy _|___ Advanced Research _| Fax: +1(604)822-5324
UBC, Vancouver,BC _|_ Program in Cosmology | un...@physics.ubc.ca
Canada V6T 1Z1 | and Gravity __|_ www.theory.physics.ubc.ca/

On Sat, 10 Jun 2017, Denny Page wrote:


Mike,
I haven’t tested that one, so I don’t know I can’t speak to its accuracy. Best 
path forward I would see is to set up a single
client for both PTP and NTP and see how closely they follow.
Best,
Denny


  On Jun 9, 2017, at 15:40, Michael Schwager  wrote:

Sonoma N12

On Thu, Jun 8, 2017 at 7:15 PM, Denny Page  wrote:
  Which one?

  Denny

  > On Jun 8, 2017, at 15:09, Michael Schwager  wrote:
  >
  > I have a local GPS NTP/PTP time appliance

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Denny Page]

Mike,

I haven’t tested that one, so I don’t know I can’t speak to its accuracy. Best 
path forward I would see is to set up a single client for both PTP and NTP and 
see how closely they follow.

Best,
Denny


> On Jun 9, 2017, at 15:40, Michael Schwager  wrote:
> 
> Sonoma N12
> 
> On Thu, Jun 8, 2017 at 7:15 PM, Denny Page  > wrote:
> Which one?
> 
> Denny
> 
> > On Jun 8, 2017, at 15:09, Michael Schwager  > > wrote:
> >
> > I have a local GPS NTP/PTP time appliance

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Michael Schwager]

Sonoma N12

On Thu, Jun 8, 2017 at 7:15 PM, Denny Page  wrote:

> Which one?
>
> Denny
>
> > On Jun 8, 2017, at 15:09, Michael Schwager  wrote:
> >
> > I have a local GPS NTP/PTP time appliance
>
> --
> To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org
> with "unsubscribe" in the subject.
> For help email chrony-users-requ...@chrony.tuxfamily.org
> with "help" in the subject.
> Trouble?  Email listmas...@chrony.tuxfamily.org.
>
>


-- 
-Mike Schwager

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Denny Page]

A great deal depends upon the server side ntp implementation which is why I 
asked which device he was using. I have several ntp devices under test, 
including one that also supports ptp. While quite accurate in the service of 
ptp, it's service of ntp has errors of up to 3ms. I have been working with the 
vendor on the issue. 

Denny
--
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Miroslav Lichvar]

On Thu, Jun 08, 2017 at 05:09:49PM -0500, Michael Schwager wrote:
> Hello,
> Is it possible to get clock accuracy in the microseconds using chrony and a
> local time server?

Yes, that should be possible.

> I have tried with ntpd and the best I have seen is 5 ms accuracy (according
> to ntpstat). I have tweaked polling intervals but still the best is 5 ms.

What polling interval have you tried? ntpstat reports the NTP root
distance, which includes delay and dispersion accumulated from the
stratum 0 time source down to the client. The problem may be with the
server. If it is reporting a large root delay or dispersion, it
doesn't matter how stable and symmetric is the network delay or what
the client does, the distance reported by the client will be always
large.

You can check the root delay and dispersion of the server with
"ntpq -c as -c 'rv &1'" or "chronyc ntpdata".

The distance gives you the maximum estimated error. If it's 5
milliseconds, it doesn't mean the actual error of the clock is 5
milliseconds. It's more likely just few tens or hundreds of
microseconds. With a decent server and a sufficiently short polling
interval it should be closer to the former.

BTW, PTP doesn't have anything like that and generally PTP slaves
cannot estimate their maximum error.

> In short: The business wants microsecond accuracy today and PTP will have
> to wait on CentOS 7 which will be a couple of weeks down the road due to
> our development priorities.

Note that in 7.4 will be chrony-3.1, which supports HW timestamping.
If you don't have switches with PTP support and the NIC on the machine
can timestamp all packets (not just PTP packets), NTP might work
better for you.

-- 
Miroslav Lichvar

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Bill Unruh]

On Thu, 8 Jun 2017, Michael Schwager wrote:


Hello,Is it possible to get clock accuracy in the microseconds using chrony
and a local time server?


With its averaging over the past N polls, chrony can get to a precision that
is significantly less than 1usec. However that is not the same as accuracy.
The problems are twofold. It takes time to service an interrupt, and it takes
time for that interrupt process to read the system clock. Those service
intervals are around 1usec and can vary (eg if other interrups are being
serviced, and interrupt handling is switched off by them, that adds a few usec
) This will give an accuracy of around 1usec. (usec=1/100 of a second) 
Note that this is ONLY if you use PPS. If you do not use PPS from a gps say,

then you are down to the 10s of usec range. If you use GPS with NMEA, not PPS,
then you are down to the many msec range. 


I have tried with ntpd and the best I have seen is 5 ms accuracy (according
to ntpstat). I have tweaked polling intervals but still the best is 5 ms.


I have no idea what you mean here by ms. I assume you mean milliseconds.

ntpd will give you tens of usec, not msec. even from a local stratum 1
source, not GPS. So it is not clear what you are doing, or what kind of system
you are runnint ntpd on.






I have a local GPS NTP/PTP time appliance, and on the same switch, a few
servers. One of the servers has a Solarflare PTP-enabled network card and we
are running sfptpd. On that host, our delay to our appliance varies
generally by less that 50 ns, with some jitter into the 150 ns range. In



How are you measuring that?



short: little jitter in our network, and only one (local) hop between time
server and client.. 

I have another server that I need to discipline using NTP. This is because
it's CentOS 6.3, which lacks the facilities to support
ptp.(see http://linuxptp.sourceforge.net/, "In July of 2011, the PTP
Hardware Clock (PHC) subsystem was merged into Linux version 3.0...").
Upgrading to CentOS 7 is not on option right now, due to work priorities for
our development crew.


shrony will run on it. Chrony will,
 give a lot lot better than ms accuracy.



In short: The business wants microsecond accuracy today and PTP will have to
wait on CentOS 7 which will be a couple of weeks down the road due to our
development priorities.

Thanks.
--
-Mike Schwager

Re: [chrony-users] Possible to get sub-millisecond accuracy?

[Denny Page]

Which one?

Denny

> On Jun 8, 2017, at 15:09, Michael Schwager  wrote:
> 
> I have a local GPS NTP/PTP time appliance

-- 
To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org 
with "unsubscribe" in the subject.
For help email chrony-users-requ...@chrony.tuxfamily.org 
with "help" in the subject.
Trouble?  Email listmas...@chrony.tuxfamily.org.