Jake, OK using the definition of master/slave with respect to the PTP-network:
1. The PTP-network is a master when receiving time from the system clock (NTP feeds the system clock, and 'phc2sy' 'feeds the PTP master hardware clock which then feeds the PTP network), 2. And the PTP-network is the slave when setting system clock (the 'ptp4'l writes the hardware clock, then 'phc2sys' writes the system clock tuned to the hardware clock). When the PTP-network is the master the system clock is being sit by some source (NTP, GPS, the system crystal, etc.). Topic of interest is when the PTP-network is a master. Timemaster is slave-only. That is, PTP slave. The timemaster configures such that it 'ptp4l' node will *never* become master clock. Therefore 'timemaster' is totally useless when one wants any node to become PTP master should the master fail. Example: 6 PTP node network: 1. ptp4l -i eth0 (so it can become master if the master fails); phcsys -a -r -r (phcsys uses system clock to set hardware clock if it should become master) 2. ptp4l -i eth0 (so it can become master if the master fails); phcsys -a -r -r (phcsys uses system clock to set hardware clock if it should become master) 3. ptp4l -i eth0 (so it can become master if the master fails); phcsys -a -r -r (phcsys uses system clock to set hardware clock if it should become master) 4. ptp4l -i eth0 (so it can become master if the master fails); phcsys -a -r -r (phcsys uses system clock to set hardware clock if it should become master) 5. ptp4l -i eth0 (so it can become master if the master fails); phcsys -a -r -r (phcsys uses system clock to set hardware clock if it should become master) 6. ptp4l -i eth0 (so it can become master if the master fails); phcsys -a -r -r (phcsys uses system clock to set hardware clock if it should become master) 7. In a Linux system all PTP nodes are created at boot, 8. By auto-negotiation one of the nodes becomes PTP master and the remaining 5 nodes become PTP slaves, 9. A background application written by me CheckMasterStatus is started on each of the system nodes at boot monitoring PTP status using PMC. . if PTP master the ntpd deamon is started to feed the system clock . if PTP slave a check is made for ntdp running on the system and if it exist it is killed (this would happen as devices are plugged and unplugged into the network and ptp4l considers the new device to be a more accurate source). Was hoping to use 'timemaster' but since it will only start ptp4l in slave only mode (i.e., ptp4l -i eth0 -s) it won't work when automatic recovery of the network is needed. Automatic system recovery is something everyone would want built into the clock architecture and therefore manual configuration has to be done at system bring-up/boot using 'linuxptp', ptp4l, phcsys and pmc (no use for 'timemaster'). Harold -----Original Message----- From: Keller, Jacob E [mailto:jacob.e.kel...@intel.com] Sent: Tuesday, August 18, 2015 4:04 PM To: mlich...@redhat.com; Harold Lapprich <hlappr...@pixel-velocity.com> Cc: linuxptp-users@lists.sourceforge.net Subject: Re: [Linuxptp-users] Grandmaster Auto-Negotiation and Reconfiguration of phc2sys to ptp4l Synchronization Hi, On Tue, 2015-08-18 at 14:54 +0000, Harold Lapprich wrote: > Jake, > > Thanks for the detailed reply. My apologies but I need to iterate on > this one more time, want to keep master/slave with respect to the > system-network and NOT the PTP-network (for example the PTP-network is > a slave to the system-network master). > It doesn't matter what you want. The terminology is what it means. You changing it is confusing the discussion. The PTP network derives it's time from somewhere, be that GPS, NTP, it's own crystal on the grandmaster, whatever. This is not relevant to PTP. However, how you transition this time in is, sure. > Want to drop the 'ptp4l' scenario of system-network master from the > discussion to simplify things, another words PTP-network is always a > slave to the system-network or to NTP/ntpd master: > PTP always derives it's time from some source. > 1. NTP --> system clock -> phc2sys -> ptp4l (master) -> network > out to other PTP slaves --> ptp4l (slave) --> phc2sys --> system clock > (No NTP here) > When PTP is a master node, it gets its time from some source. In this case, NTP feeds the system clock, and phc2sys feeds the PTP master hardware clock which then feeds the PTP network. > > > Timemaster is slave-only. That is, PTP slave. The timemaster > configures such that it's ptp4l node will *never* become master clock. > The reason for this is because phc2sys only supports ntpshm segment > clock going from slave mode, and would have to switch servos which no > one has written code for. > Time master is slaveOnly, that is, PTP will always be slave, so it will always be creating a SHM segment clock that may (or may not) feed PTP. > For example: > ptp4l -i eth0 > phc2sys -a -r -r > chronyd 'local stratum 1' 'allow' > > The example configuration file shows 'ptp4l' being able to assume > GrandMaster but NOT system-network master due to a clock update > The example configuration file for PTP allows it perfectly to become GrandMaster and feed the system time. The issue you have is that when this happens you STILL have NTP configured to set the host clock. Your whole issue is that NTP is not aware of this and tries to reset system clock instead of serving that time. You need to fix NTP to assume that the system clock is absolute. > conflict between 'phc2sys' and the NTP/ntpd deamon (along with the SHM > or shared memory code issue you mentioned). Therefore 'phc2sys' > must always be a slave passing updates to 'ptp4l' which is GrandMaster > to the PTP-network? > This only happens when ptp4l is in "slave" mode. phc2sys will only write the system clock (when configured in -a -r mode) when ptp4l is *receiving* time from the network. Ie; it is deriving its time source from some other GrandMaster clock. > If 'ptp4l' can NOT become GrandMaster there would be no way of > synchronizing system-network clock with PTP-network clock. > Sure there is. Not every node has to be configured the same way. You configure one node manually with high values for its clock source and then it is elected GrandMaster clock. Then the other nodes are slaveOnly so they will never send announce messages, thus will never be selected. The issue is that the default NTP keeps trying to reset the clock after phc2sys configures it. If ptp4l is in "master" (or even the GrandMaster of a larger network) you can configure NTP to write the system clock, then phc2sys to tune the hardware clock running ptp4l to the system clock time source. Thus, ptp4l will serve (or as above, feed) the PTP network with this time source. If ptp4l is in "slave" mode it receives PTP time from the PTP network and tunes its internal clock. Then phc2sys feeds the system time from the internal clock. However, by default, ntpd is also trying to feed the system clock via its own mechanism. This is the conflict. Two sources controlling the clock. Thus, timemaster gets around this by not having phc2sys feed the system clock, but instead feed an SHM segmnent which ntpd/chronyd read to use as a possible input, along with NTP, and feed the system time from whichever is best. The issue is that phc2sys can't switch between these two modes, because phc2sys would have to change servos which is not supported. Regards, Jake > Thanks, > Harold > > > > > -----Original Message----- > From: Keller, Jacob E [mailto:jacob.e.kel...@intel.com] > Sent: Monday, August 17, 2015 6:14 PM > To: mlich...@redhat.com; Harold Lapprich < > hlappr...@pixel-velocity.com> > Cc: linuxptp-users@lists.sourceforge.net > Subject: Re: [Linuxptp-users] Grandmaster Auto-Negotiation and > Reconfiguration of phc2sys to ptp4l Synchronization > > Hello again, > > I think you're still missing something. > > Timemaster is slave-only. That is, PTP slave. The timemaster > configures such that it's ptp4l node will *never* become master clock. > The reason for this is because phc2sys only supports ntpshm segment > clock going from slave mode, and would have to switch servos which no > one has written code for. > > Thus, timemaster will only ever setup PTP slaves. Ie: these nodes can > never be elected as the master clock in the PTP network at all. > > I'm not sure what you mean by "entireSystem-network". > > Hopefully we are clear here, and "master/slave" designation only > refers to PTP network. > > You can decide what the SOURCE of the time on the ptp master is, > either GPS, NTP, etc. That's not of concern to PTP network. > > timemaster lets you configure a ptp4l *slave* to feed a time clock to > chronyd such that chronyd will either use NTP or PTP time whichever > one is considered more accurate. > > I don't believe timemaster configures chronyd to actually *serve* the > PTP time onto the NTP protocol, but Miroslav can correct me if I am > wrong. > > If you have a PTP master node, then it needs to get time from > somewhere. Usually this would be from say, NTP which writes the master > clock then the phc2sys writes the hardware clock onboard the ethernet > device using the system clock as a reference. > > In reverse, the ptp4l writes the hardware clock, then phc2sys writes > the system clock tuned to the hardware clock. But by default ntpd and > chronyd are configured to write the system clock and thus we have two > processes writing to the system clock, which results in the > clock_check errors you saw at the beginning of this thread. > > Those are the two flows in regards to ptp4l clock, system clock and > phc2sys and chronyd's roles. > > phc2sys can also provide a "SHM" reference to chronyd WITHOUT writing > the system clock, and chronyd will use this as a reference to tune the > system clock. This however can't work when phc2sys must write the > hardware clock, as chronyd and ntpd don't know of the hardware clock. > Thus, phc2sys can't swap between these modes, since it is not designed > to change servos from the PI servo to the NTPSHM servo mid run, thus > it can't automatically switch if the node is selected as master. This > is why timemaster configures ptp4l as slave only. > > Regards, > Jake > > On Mon, 2015-08-17 at 21:56 +0000, Harold Lapprich wrote: > Jake, > > Thanks for clarifying the function of the 'timemaster', a program > purely for configuring other programs. Going to go through your > response 1 line item at a time to simplify my response and clarify my > understanding. Getting specific terms in place for discussing this > matter has been helpful. > > So “master” means serving time out over the entireSystem-network NOT > the PTP-network (intentionally separating entireSystem-network and > PTP-network). When 'master' is used it is with regards to the > entireSystem-network and GrandMaster is used with regards to the PTP > -network. > > > > Miroslav mentioned that 'timemaster' is PTP slave only and starts > ptp4l with the slaveOnly option, so it can't become a master and that > didn't add up for me. I believe master and GrandMaster were getting > mixed up here (the slaveOnly options is for GrandMaster and not > master). > > "No, timemaster is currently PTP slave only. It starts ptp4l with the > slaveOnly option, so it can't become a master. With current phc2sys it > wouldn't work anyway. It would need to be modified to switch between > two servos, NTP SHM when the PTP clock is synchronized and a real > servo when the system clock is the source." > > > "For example: > > ptp4l -i eth0 > > phc2sys -a -r -r > > chronyd 'local stratum 1' 'allow'" > > This example provided 'ptp4l -i eth0' shows ptp4l starting withOUT the > slaveOnly option '-s' so in this case 'ptp4l' can assume GrandMaster > of the PTP-network (again it appears to be a mixing of the terms > master/entireSystem-network and Grandmaster/PTP-network)? > > > > Thanks for the flow diagram in your response is most helpful, I was > thinking 'timemaster' did more than just configure programs: > > > 1. NTP --> system clock -> phc2sys -> ptp4l (master) -> network > out to other PTP slaves --> ptp4l (slave) --> phc2sys --> system clock > (No NTP here) > > As you mentioned ptp4l(master) is the GrandMaster of the PTP > -network but NOT the 'master' serving time out over the entireSystem > -network. And ptp4l(master) is a slave to the NTP daemon via > 'phc2sys'. > > So having 'phc2sys' use the system clock to update 'ptp4l' isn't an > issue since the system clock is only being updated by one program the > ntpd deamon. > > > > > > OK we are now discussing the following flow configuration (if ptp4l > was to be the GrandMaster/PTP-network and 'master' of the > entireSystem-network): > > > 1. NTP< --system clock<- - phc2sys<-- ptp4l (master) -> network > out to other PTP slaves --> ptp4l (slave) --> phc2sys --> system clock > (No NTP here) > > To perform the NTP syncs PTP master setup, you need to do something > like: > > > 1. Configure NTP to control the system clock > > 2. Configure phc2sys to drive the PTP hardware clock on your > device. Your problem is that this behavior does not allow automatic > configuration, because phc2sys can’t switch servos while running. > Since it will attempt to drive the system time 100% when tuning system > time to ptp4l you get the result where NTP attempts to interfere. > > a. If you do this manually, however, then it won’t automatically > switch directions for you > > In this case the system running the NTP server (ntpd deamon) would be > taking system time and broadcasting it across the entireSystem > -network or the 'master' using the Network Timing Protocol (NTP). > Also running on this same system would be the GrandMaster/PTP > -network. There would be NO conflict changing the system clock since > 'phc2sys' would be the only program and the ntpd daemon would be > broadcasting it across the entireSystem-network, correct? > > But there is an issue should the GrandMaster fail (removed or powered > down) in that both the NTP server and GrandMaster would be loss. In > this case the remaining slave/PTP-network devices would auto > -negotiate another GrandMaster but no NTP server would co-exist. So a > program would need to be running in the background on all systems > checking for a switch from slave to a GrandMaster/PTP-network device > and bring a NTP server up upon detecting a switch from slave to > GrandMaster, correct? > > > > > > It is possible your device is good enough to actually be the grand > -master of time, in which case it could be used to set the system time > as well. But that would really only occur if your PTP node is a > dedicated clock, and I can’t say I have experience setting one of > those up. > > This is a good point and I don't have an answer right now. If just a > local PTP-network existed then one of the system could be the NTP > server but way when you have the entire local network covered by PTP. > > > Thanks, > Harold > > > > > > From: Keller, Jacob E [mailto:jacob.e.kel...@intel.com] > Sent: Monday, August 17, 2015 2:09 PM > To: Harold Lapprich <hlappr...@pixel-velocity.com>; Miroslav Lichvar > <mlich...@redhat.com> > Cc: linuxptp-users@lists.sourceforge.net > Subject: RE: [Linuxptp-users] Grandmaster Auto-Negotiation and > Reconfiguration of phc2sys to ptp4l Synchronization > > Hi, > > Timemaster is a program to help simplify the configuration of the > other programs, in that sense, it doesn’t “do” any of the things on > its own. > > The flow diagram should be: > > > 1. NTP --> system clock -> phc2sys -> ptp4l (master) -> network > out to other PTP slaves --> ptp4l (slave) --> phc2sys --> system clock > (No NTP here) > > If you’re ptp4l is “master” that means it is serving time out the > network and thus is the only time when NTP should be configured it. > In this case, ptp4l would function as master on the PTP network, but > would be the slave of the NTP daemon via phc2sys. > > It is possible your device is good enough to actually be the grand > -master of time, in which case it could be used to set the system time > as well. But that would really only occur if your PTP node is a > dedicated clock, and I can’t say I have experience setting one of > those up. > > To perform the NTP syncs PTP master setup, you need to do something > like: > > > 1. Configure NTP to control the system clock > > 2. Configure phc2sys to drive the PTP hardware clock on your > device. Your problem is that this behavior does not allow automatic > configuration, because phc2sys can’t switch servos while running. > Since it will attempt to drive the system time 100% when tuning system > time to ptp4l you get the result where NTP attempts to interfere. > > a. If you do this manually, however, then it won’t automatically > switch directions for you > > I believe that is the crux if your issue here. In addition, I think > you have the notion of slave/master backwards. Slave means that it is > receiving time from the network and thus is the “master” time source > in the system. Master means it is serving time out the network and is > thus the “slave” to the system time. This dual usage of terminology > can get confusing. > > Regards, > Jake > > From: Harold Lapprich [mailto:hlappr...@pixel-velocity.com] > Sent: Monday, August 17, 2015 10:16 AM > To: Miroslav Lichvar > Cc: Keller, Jacob E; linuxptp-users@lists.sourceforge.net<mailto: > linuxptp-users@lists.sourceforge.net> > Subject: RE: [Linuxptp-users] Grandmaster Auto-Negotiation and > Reconfiguration of phc2sys to ptp4l Synchronization > > Miroslav, > > Thanks again for the quick response, trying to simplify the discussion > and therefore minimize any mis-understanding by providing the > following simply flow diagrams: > > Need to support the following 2 scenarios using the 'linuxptp' > applications but NOT both in the same network configuration (only 1 of > the 2 will exist) > > 1. NTP <---- timemaster <---- system clock <---- phc2sys <--- > - 'ptp4l (master)' ----//---->ptp4l (slaves)---->phc2sys---->system > clock > > 2. NTP ----> timemaster ----> system clock ----> phc2sys --- > -> 'ptp4l (master)' ----//----ptp4l (slaves)---->phc2sys---->system > clock > > Please let me know if the proceeding flow diagrams are NOT correct? > > > > > I. Do you need NTP as a time source? Or just serve time over NTP? The > former conflicts with your requirement to allow ptp4l to be a master > as phc2sys would need to be the process that synchronizes the clock > and not chronyd/ntpd. > > Want the ability to have NTP as a time source or serve time over NTP > but not both at the same time (therefore need the capability to do > both, 1: when local network configuration is standalone then a network > server will be locked to PTP time via PTP-->NTP, and 2: when the local > network is a subset of a larger network and therefore NTP - > -> PTP). The configuration will be a known entity and therefore the > 'linuxptp' application configuration files created appropriately this > is NOT something that needs to happen automatically. > > > > > II. The former conflicts with your requirement to allow ptp4l to be a > master as phc2sys would need to be the process that synchronizes the > clock and not chronyd/ntpd. > > It is my understanding that 1 of the PTP clients has to be a master > (ptp4l master) but the master can be locked to the system clock by > phc2sys (this is what I am currently doing, ptp4l -i eth0 and phc2sys > -a -r -r). > Then 'timemaster' would be used to synchronize the system clock to > NTP. > > 1. NTP <---- timemaster <---- system clock <---- phc2sys <--- > - 'ptp4l (master)' ----//---->ptp4l (slaves)---->phc2sys---->system > clock > > > > > III. The problem is that when phc2sys is configured to feed > chronyd/ntpd, it is not able to synchronize the PTP clock in the > reverse direction when ptp4l is master. > > It is my understanding that the ptp4l(master) will be driving phc2sys > to drive the system clock (1: 'ptp2l -i eth0 -m', 'phc2sys -a -r -r > -m'; 2: 'ptp4l -i eth0 -s -m', 'phc2sys -a -r -m'). Or another words > PTP master clock will be driving everything. > > 1. NTP <---- timemaster <---- system clock <---- phc2sys <--- > - 'ptp4l (master)' ----//---->ptp4l (slaves)---->phc2sys---->system > clock > > > > > Jake Keller > > If you run ptp4l on all your systems, and each one also running > phc2sys, it will: > > on system which is "master" > > phc2sys will drive the ptp4l hw clock based on local time > > ptp4l will sync time out the network using PTP > > on systems which are not master > > ptp4l will sync time in from network to hw clock > > phc2sys will sync hw clock to CLOCK_REALTIME. > > > Also Jake Keller response: "But if you want to also use NTP as a clock > source, then you need to use timemaster, as otherwise phc2sys and ntpd > will interfere with each other." Dosen't this imply that using > 'timemaster' will remove this issue (phc2sys will synchronize the PTP > (master) clock to the system clock and NTP will synchronize the system > clock)? > > > Harold > > > > > -----Original Message----- > From: Miroslav Lichvar [mailto:mlich...@redhat.com] > Sent: Monday, August 17, 2015 11:54 AM > To: Harold Lapprich <hlappr...@pixel-velocity.com<mailto: > hlappr...@pixel-velocity.com>> > Cc: Keller, Jacob E <jacob.e.kel...@intel.com<mailto: > jacob.e.kel...@intel.com>>; linuxptp-users@lists.sourceforge.net<mail > to:linuxptp-users@lists.sourceforge.net> > Subject: Re: [Linuxptp-users] Grandmaster Auto-Negotiation and > Reconfiguration of phc2sys to ptp4l Synchronization > > On Mon, Aug 17, 2015 at 02:14:36PM +0000, Harold Lapprich wrote: > > "If you run ptp4l on all your systems, and each one also > > running phc2sys, it will: > > > > on system which is "master" > > > > phc2sys will drive the ptp4l hw clock based on local > > time > > > > ptp4l will sync time out the network using PTP > > > > on systems which are not master > > > > ptp4l will sync time in from network to hw clock > > > > phc2sys will sync hw clock to CLOCK_REALTIME. > > > > But if you want to also use NTP as a clock source, then you > > need to use timemaster, as otherwise phc2sys and ntpd will interfere > > with each other." > > Strictly speaking you just need to configure phc2sys to use the NTP > SHM servo to feed chronyd/ntpd so they can select the best source or > combine multiple sources and synchronize the clock. That's how phc2sys > is configured by timemaster. > > Do you need NTP as a time source? Or just serve time over NTP? The > former conflicts with your requirement to allow ptp4l to be a master > as phc2sys would need to be the process that synchronizes the clock > and not chronyd/ntpd. > > If you just need to serve NTP, you can configure chronyd/ntpd to not > synchronize the clock and keep phc2sys in the control of the clock. > > > So when NTP is to be the clock source (and vice versa) then > > 'timermaster' is needed because phc2sys and ntpd will interfere with > > one another. Now the problem is GrandMaster failure, if I understand > > you correctly when another PTP system on the network becomes the > > GrandMater 'timemaster' will NOT automatically reconfigure and start > > using NTP as the clock source (using timemaster to start PTP > > configuration on all systems on the PTP network)? > > The problem is that when phc2sys is configured to feed chronyd/ntpd, > it is not able to synchronize the PTP clock in the reverse direction > when ptp4l is master. > > > If this is the case then one would have to have another application > > running in the background to detect the switch, create the > > appropriate 'timemaster' configuration file and start? > > There is currently no way to configure timemaster to serve local time > over PTP. phc2sys would need to allow that first. > > -- > Miroslav Lichvar > ------------------------------------------------------------------------------ _______________________________________________ Linuxptp-users mailing list Linuxptp-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-users