Re: [chrony-users] Massive difference between NTP clocks and PHC
On Fri, Oct 14, 2016 at 12:26:05PM +0100, Denys Rtveliashvili wrote: > I see that "linuxptp" has configured chrony to connect to PTP0 (shared > memory segment) rather that PHC0 (direct connection to a PHC clock). I > wonder if that makes any real difference. One difference is that the SHM refclock is always in UTC, while the PHC can be in TAI or UTC, so you don't have to deal with the offset between TAI and UTC. Another difference is that the SHM refclock will stop when the PHC is not synchronized by PTP. > The only unknown is whether I should add some kind of static offset as while > PTP measures the time at the card quite precisely, there is still some delay > for getting time from that clock on the card over PCIE express and a shared > memory segment and then to RTC clock. I guess it would be on a scale of > microseconds. No, you shouldn't need to specify any offset for that unless you have measured the amount of asymmetry in the delay. The delay is assumed to be symmetric and the measurements are automatically corrected by half of the delay. -- Miroslav Lichvar -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Massive difference between NTP clocks and PHC
Thank you very much everyone. I have almost figured it out. The first problem was that there was a typo in the network address of the PTP port. PTP packets still arrived but it somehow made the system unhappy. The second thing was that the configuration of ptp4l had some kind of a problem. After I started using linuxptp to configure both chrony and ptp4l thought it - it became much better. I see that "linuxptp" has configured chrony to connect to PTP0 (shared memory segment) rather that PHC0 (direct connection to a PHC clock). I wonder if that makes any real difference. So now it looks clean. The only unknown is whether I should add some kind of static offset as while PTP measures the time at the card quite precisely, there is still some delay for getting time from that clock on the card over PCIE express and a shared memory segment and then to RTC clock. I guess it would be on a scale of microseconds. With kind regards, Denys Rtveliashvili On Fri, Oct 14, 2016 at 10:38:00AM +0100, Denys Rtveliashvili wrote: PHC0 is a clock on an Intel card running under igb driver. The PTP port of the card is connected to a PTP-enabled Cisco Nexus switch, which is connected to a PTP grandmaster. The same grandmaster in fact, which is also a NTP source ntp-clock-1. I would understand 36 secons of a difference (TIA vs GMT) or some microseconds of a difference (some network and PCIE jitter). But a third of a second is very strange. Do you have any ideas? Most likely the PHC is not synchronized. Is chronyd using the right PHC? Maybe you have more than one. By the way, for synchronization of on-the-NIC PTP clock I tried using ptp4l in both software and hardware mode and that did not make any difference. It needs to be in the hardware mode. Was ptp4l working correctly? Was it reporting it's in the slave state? It could be a firewall issue. In any case, I'd not recommend using the PHC refclock directly when it's keeping time in TAI as you will need to reconfigure chronyd on each leap second. A better way is to use ptp4l and phc2sys -E SHM to provide a SHM refclock in UTC for chrony. With the timemaster program from linuxptp it can be easily configured. It can prepare configuration files for ptp4l, phc2sys and chronyd and start them with the right options as needed. Here is an article showing some examples with timemaster if you don't mind reading some RHEL-specific stuff. It's the last third of the post. http://rhelblog.redhat.com/2016/07/20/combining-ptp-with-ntp-to-get-the-best-of-both-worlds/ -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Massive difference between NTP clocks and PHC
On Fri, Oct 14, 2016 at 02:57:23AM -0700, Bill Unruh wrote: > On Fri, 14 Oct 2016, Denys Rtveliashvili wrote: > > I have tried adding PHC to the configuration of chrony and see things like > > What is PHC? Could it be trailing-vs-leading-edge-of-a-pulse problem? It's a clock on a network card, which can precisely timestamp packets. Typically they are used with PTP and have been historically called PTP hardware clocks (PHC), but most of them can work also with NTP or any other protocol. Support for HW timestamping is one of the things I'm planning for the next version of chrony. -- Miroslav Lichvar -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Massive difference between NTP clocks and PHC
On Fri, Oct 14, 2016 at 10:38:00AM +0100, Denys Rtveliashvili wrote: > PHC0 is a clock on an Intel card running under igb driver. > The PTP port of the card is connected to a PTP-enabled Cisco Nexus switch, > which is connected to a PTP grandmaster. The same grandmaster in fact, which > is also a NTP source ntp-clock-1. > > I would understand 36 secons of a difference (TIA vs GMT) or some > microseconds of a difference (some network and PCIE jitter). But a third of > a second is very strange. > Do you have any ideas? Most likely the PHC is not synchronized. Is chronyd using the right PHC? Maybe you have more than one. > By the way, for synchronization of on-the-NIC PTP clock I tried using ptp4l > in both software and hardware mode and that did not make any difference. It needs to be in the hardware mode. Was ptp4l working correctly? Was it reporting it's in the slave state? It could be a firewall issue. In any case, I'd not recommend using the PHC refclock directly when it's keeping time in TAI as you will need to reconfigure chronyd on each leap second. A better way is to use ptp4l and phc2sys -E SHM to provide a SHM refclock in UTC for chrony. With the timemaster program from linuxptp it can be easily configured. It can prepare configuration files for ptp4l, phc2sys and chronyd and start them with the right options as needed. Here is an article showing some examples with timemaster if you don't mind reading some RHEL-specific stuff. It's the last third of the post. http://rhelblog.redhat.com/2016/07/20/combining-ptp-with-ntp-to-get-the-best-of-both-worlds/ -- Miroslav Lichvar -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
Re: [chrony-users] Massive difference between NTP clocks and PHC
William G. Unruh __| Canadian Institute for| Tel: +1(604)822-3273 Physics&Astronomy _|___ Advanced Research _| Fax: +1(604)822-5324 UBC, Vancouver,BC _|_ Program in Cosmology | un...@physics.ubc.ca Canada V6T 1Z1 | and Gravity __|_ www.theory.physics.ubc.ca/ On Fri, 14 Oct 2016, Denys Rtveliashvili wrote: Hello, I have tried adding PHC to the configuration of chrony and see things like What is PHC? Could it be trailing-vs-leading-edge-of-a-pulse problem? this: MS Name/IP address Stratum Poll Reach LastRx Last sample === #x PHC0 0 3 37711 -322ms[ -322ms] +/-2ns ^* ntp-clock-1 1 4 37711 -2579ns[-4000ns] +/- 441us ^- ntp-clock-2 1 4 37711 -2000ns[-2000ns] +/- 31ms 322ms is a huge offset and I wonder if anybody has an idea of what could be the root source. ntp-clock-1 and ntp-clock-2 are two proper grandmasters. ntp-clock-2 is geographically far. ntp-clock-1 is in the same location. PHC0 is a clock on an Intel card running under igb driver. The PTP port of the card is connected to a PTP-enabled Cisco Nexus switch, which is connected to a PTP grandmaster. The same grandmaster in fact, which is also a NTP source ntp-clock-1. I would understand 36 secons of a difference (TIA vs GMT) or some microseconds of a difference (some network and PCIE jitter). But a third of a second is very strange. Do you have any ideas? By the way, for synchronization of on-the-NIC PTP clock I tried using ptp4l in both software and hardware mode and that did not make any difference. With kind regards, Denys Rtveliashvili -- -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
[chrony-users] Massive difference between NTP clocks and PHC
Hello, I have tried adding PHC to the configuration of chrony and see things like this: MS Name/IP address Stratum Poll Reach LastRx Last sample === #x PHC0 0 3 37711 -322ms[ -322ms] +/-2ns ^* ntp-clock-1 1 4 37711 -2579ns[-4000ns] +/- 441us ^- ntp-clock-2 1 4 37711 -2000ns[-2000ns] +/- 31ms 322ms is a huge offset and I wonder if anybody has an idea of what could be the root source. ntp-clock-1 and ntp-clock-2 are two proper grandmasters. ntp-clock-2 is geographically far. ntp-clock-1 is in the same location. PHC0 is a clock on an Intel card running under igb driver. The PTP port of the card is connected to a PTP-enabled Cisco Nexus switch, which is connected to a PTP grandmaster. The same grandmaster in fact, which is also a NTP source ntp-clock-1. I would understand 36 secons of a difference (TIA vs GMT) or some microseconds of a difference (some network and PCIE jitter). But a third of a second is very strange. Do you have any ideas? By the way, for synchronization of on-the-NIC PTP clock I tried using ptp4l in both software and hardware mode and that did not make any difference. With kind regards, Denys Rtveliashvili -- To unsubscribe email chrony-users-requ...@chrony.tuxfamily.org with "unsubscribe" in the subject. For help email chrony-users-requ...@chrony.tuxfamily.org with "help" in the subject. Trouble? Email listmas...@chrony.tuxfamily.org.
