On 17 Mar 2021 at 18:01, Miroslav Lichvar wrote: [on PCI-e PTM] > > The switches are supposed to work like NTP server and client at the > same time (boundary clock in the PTP terminology), so all PCIe links > have hardware timestamping on both ends. > > BTW, at least in theory, a network using boundary clocks should > perform better than a similar network using transparent clocks, > assuming the servos are well configured and the sync interval is short > enough to minimize the time errors in the chain. Divide and conquer > :). I think transparent clocks are meant to be the simpler and cheaper > variant. >
I recall some history around the debate on BC vs TC. >From my subjective perspective anyway. PTP v1 used to be all about BC's. Then in 2008 came PTP v2, and suddenly TC was all the rage, the only right way for PTP-capable switches to function from now on. Except that, as years were passing by, PTP-aware switches did not exactly spawn in flocks everywhere. Rather, they remained few and far between. And, especially the telco industry would've loved to deploy PTP, but there was no way for their active elements in the networks to support PTP (to turn them all into TC's). So after some years since 2008, suddenly the Telecom Profiles came out, and were all about BC's and partial on-path support (or rather: no on-path support). The TC idea has a stronghold in the electric power industry, at least on paper - practical compatibility of implementations remains a fun topic, to use a euphemism, especially where multi-vendor heterogeneous setups are attempted... this is where I have a limited by ongoing practical experience :-) On the topic of BC's and their apparent renaissance, I've had a brief debate about this with someone from Meinberg, and the message was: BC's or TC's, either of them works, if done properly. Reportedly there have been tests of up to 16 BC's in a cascade, and if the local oscillators were stable, and the feedback loops were tuned appropriately, the cascade just worked, there were no ill effects (excessive residual wander at the tail end, or whatever). So it's all down to oscillator quality and feedback loop parameters = control theory and the respective math. Laplace, Nyquist, whatever... As for TC's, from my practical observations (and maybe a bit of gossip), switch chipset vendors have a hard time implementing the on-the-fly corrections in the hardware. Something that would work "out of the box" for the switch box makers. And because the silicon vendors struggle, some switch box vendors implement PTP on their own, mangling MII on the fly using custom FPGA designs etc. You probably know the ugly details an order of magnitude better than I do, i.e. what it takes to make a TC switch, for P2P mode vs. E2E mode... The "TC support in a switch" is likely never going to be pure hardware, there's always an element of switch firmware that e.g. does the talking with P2P peers... While E2E might look easier to support in a switch (than P2P), I've heard a credible opinion that in E2E mode, a switch port needs to keep track of delay transactions passing through (stateful style - ORLY?) and making E2E work for a single slave is not the same as making it work for many slaves :-) etc. I've wandered even further off topic here... In PTM, having a servoed local oscillator in every PCI-e switch (BC style) seems complicated, to the point of being impractical / difficult to implement, if the oscillator should be half decent / stable... In my wildest dreams, it would be beautiful to have a 1PPS output out of the CPU TSC. Plug that into some SDP input on an i210, and you'd have an idea, where the CPU's PPS edge is located, relative to the i210 PHC. Which can in turn be disciplined by PTP, or by another external 1PPS reference. Just a single output wire, coming from the TSC - it wouldn't even need to be adjustable. Having a timestamp for that CPU 1PPS event from the i210, you'd be able to calculate an offset in TSC granularity, and subtract that offset from every timestamp obtained via rdtsc (at the cost of a single integer sub() instruction). A single signal, for 1PPS, coming out of the CPU socket. Is that too much to ask? Compared to the rather complex and imperfect PTM implementation? Actually this is not my wildest dream. I can extrapolate this further. Such as: After the all, the TSC stands for a "Time Stamp Counter" - isn't that ironic? As it is, the TSC is just a register counting steadily forward. What reference does it actually have to the wall time in the outside world, apart from the CPU core clock? (Yes I know - now with TurboBoost, even the notion of a CPU clock is hazy.) I mean wouldn't it be nice if the CPU would have a discrete event *input* into the TSC? Typically useable as 1PPS *input*. Having a timestamping register (MSR?) on the inside, a neighbor to the TSC, which would always contain a timestamp of the last exterior 1PPS edge, in the TSC's "time domain". And maybe have another register yet, clocked the same way as the TSC, but which would always reset at every 1PPS edge. Should I patent this as the TSC-ng ? :-) Next up: make it easier to reference the relevant clock signals in a PC system to a common local oscillator. If the system builder (or even integrator) so desires, so that he can drop a VC-OCXO module on the board, and have synchronous-referenced clocks for the CPU TSC, the NIC ports, and whatever have you. This, combined with a 1PPS output or input in the CPU TSC, would allow for some nasty timing performance :-) And would make passing EMC tests more complicated, without "spread spectrum" tricks. I've tried PLL-railroading the 25MHz i210 NIC clock, I've heard about people who swap/discipline their PC motherboard's central clock XTAL. To me a problem with the motherboard's existing central oscillator is, that its frequency is not standardized, and can be "the most abundant ugly binary multiple available on the XTAL market at the moment", difficult to co-divide with 10 MHz, or 25 MHz, or whatever external reference would seem practical in a lab environment. There's obviously a theoretical chance to just plug in a "nicer" frequency reference (instead of the original crystal) and reprogram the existing clock synth - but unless you can mod the BIOS (or flash coreboot, or some such) the machine will have to boot far enough with a skewed set of system clock frequencies, for you to be able to finally reprogram the synth (and with a bit of luck the PC would not hang while you're tampering with the synth registers). And obviously this is not generic / reproducible across the PC HW market. The way it is, it would probably be easiest to equip your custom VC-OCXO timebase with a multi-channel synth output that would produce the right frequencies to "nail" all the various crystals in the system. I myself have not gone that far yet :-) in my garagey DIY experiments. Oh we still have SyncE in the subject... Yes if you provide your own external stable clock to the 25MHz clock input of your NIC, you can as well configure the PPB frequency offset to 0 in your NIC's internal digital synth, and re-wire your ptp4l servo loop to discipline your local VC-OCXO out of band / in a proprietary way. Now to combine this with SyncE, you'd need to solve the clock recovery at the "SyncE slave" end. Low-level mixed-signal electronics. How to properly hook up a PLL to the raw Ethernet signals. Even a vanilla analog PLL can weed out a lot of ugly random mess out of a noisy signal... In descriptions in the public interwebs, I haven't found information about the details of SyncE-enabled layer 1, e.g. if the inter-frame filling looks different, or what differences there are for SERDES, vs. for N-Base metallic (especially gigabit and above). Apparently it's plausible to combine SyncE with vanilla SFP transceivers, i.e. to recover clock from the SERDES RX coming out the back of an SFP socket towards the MAC block... which is making me wonder if SyncE is possible with multi-rate SGMII-based RJ45 SFP transceivers :-) Or, are there multi-rate RJ45 SFP's with a SERDES back-end? And as you have already explained, there's also a bit of messaging taking place in SyncE, in addition to the raw PLL stuff... Frank _______________________________________________ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
