[Linuxptp-devel] Just in case: ptp4l multisession into a VLAN trunk... a pipe dream? :-)

[Frantisek Rysanek]

Dear gentlemen,

this is not a pressing issue.
Just my lunatic idea that would help me in the lab at the moment...

Every once in a while, on random occasions, I have an opportunity to 
review an Ethernet switch for its practical PTP capabilities. If I'm 
lucky, I have at least two pieces of the switch on the lab desk, but 
sometimes I only have one.

A particular prospective application is the electric energy business, 
i.e. the power profile = L2 multicast P2P.

And, one of the test setups that have proven useful, is running PTP 
through a VLAN trunk, in multiple simultaneous sessions.

There have been two different styles of running PTP (power profile) 
through a VLAN trunk:

A) the old way: PTP just a single session in the default VLAN, 
possibly untagged, seeping to all the untagged VLAN access ports on 
the switch (ignoring VLAN isolation)

B) the more modern way: Announce+Sync+Follow-up properly tagged, 
running in custom VLAN's, only the PDelay running either tagged with 
the default VLAN, or untagged (on the trunk).

Now - to test the latter without a second switch, i.e. if I should 
want to plug the trunk from the switch into a Linux box, I am out of 
options how to cobble together the "traffic mix" in Linux.
I can certainly bind ptp4l to a VLAN subinterface, but in that case, 
PDelay traffic gets tagged as well.
I've noticed the nftables' "bridge" address family, allowing me to 
filter packets while passing through a soft-bridge device.
No clues if this would work per output interface for multicast 
destinations.
Plus, I haven't found any way to filter the "opaque" PTP frame 
structure (unknown to nftables), within the PTP ethertype, on 
ptp.v2.message_id (as per the Wireshark display filter syntax).
Even if I could pull this off, I would only be able to run one such 
session against the VLAN trunk, because in principle only one ptp4l 
instance can handle the shared PDelay in the default VLAN.

Principally, to approach the problem systematically, I'd have to code 
PTP support into the Linux soft-bridge :-) Not gonna happen, and 
there's hardly any point beyond my today's hallucination.

AFAICT, ptp4l doesn't support VLAN tagging internally - which is 
perfectly understandable, and probably has been debated before.

Let me know if I'm missing something :-)

Frank



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

[Linuxptp-devel] Side note: i225 / igc time sync and TSN capabilities...

[Frantisek Rysanek]

Dear gents,

this is just a slightly off topic gratuitous side note.

In the context of fumbling in the "igb" driver in the other thread, I 
also took a look inside driver "igc" for comparison = to see what the 
i225 had to offer.
Unsurprisingly, the register map related to time sync features seems 
quite similar to that of the i210. The source code of igc_ptp.c is 
pretty clean. The datasheet for the i225 is not public yet and I 
don't have it, so I cannot comment further.

Interestingly to me, the TIMADJ register is now mentioned, but only 
to set some novel flag/bit in that register during port init/reset... 
For stepwise time adjustment, the SYSTIM registers get written 
directly :-)

I'm aware of the initial bug in framing (the Inter-Packet Gap), when 
the i225 silicon was first launched... and I've seen recent 
complaints about the B3 revision, that the chip sometimes dies (and a 
NIC firmware update is supposed to fix this). Those baby maladies 
aside, it looks like a pretty sporty piece of silicon. While fumbling 
for current information, I've found the following memo:

https://cdrdv2-public.intel.com/757442/757442_I225-226%20Time-Sensitiv
e-Networking-Features-Brief.pdf

Whoa. The thing supports 802.3br + 802.1Qbu.
The dirtier secrets of the TSN.
https://www.ieee802.org/3/br/Baseline/8023-IET-TF-1405_Winkel-iet-Base
line-r3.pdf
Not that I have any practical use for these, but it sure sounds 
arcane. This is a queueing/framing technology that allows high 
priority packets to preempt a long lazy packet in the middle of 
transmission, insert the high-priority packet, and after that gets 
transmitted, resume transmission of the long lazy packet - without 
having to drop the first part before preemption, without increasing 
the RX FCS err counter. I.e. graceful fragmentation and reassembly 
within/below L2 in Ethernet. Yes it does introduce an extra preamble 
or two, and the IPG also has to be observed... Pretty crazy stuff, by 
my standards :-)
I've seen a switch or three, that claim to support the TSN, but none 
of them actually mentioned 802.3br among the plethora of TSN-related 
802.x buzzwords.

So much for my todays random rant, thanks for your attention :-)

Frank



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Is intel i350 ptp driver broken?

[Frantisek Rysanek]

I have the same doubts with that, if I didn't miss something
elsewhere, the driver
never write SYSTIMx value to the chip, how they made i350 work with
PTP?
Or just no one have ever tested that before?

I'll try modifying the driver to see if that works, then post the
result here later.

Thanks again Frank


Frantisek Rysanek  于2023年5月25日周四 21:58写道：
On 25 May 2023 at 17:40, egg car wrote:
>
> Hi,
>
> I'm trying to use i350 adapter with a hardware 1pps signal input on
a
> SDP pin.
>
> I'm testing on kernel 5.19(i350 hardware ts capture function was
> announced to be supported since kernel 5.17), using 'ts2phc' program
> could capture the 1PPS input event, but it failed to write time
> adjustment to the nic.
>
> Then I looked into igb driver source code, found these two function
> 'igb_ptp_adjtime_82576' and 'igb_ptp_settime_82576' calculated the
> target timecounter value, but doesn't write the target value to nic
> register. In contrast to  i210 routine which works
> fine,'igb_ptp_adjtime_i210()' calls 'igb_ptp_write_i210()' method to
> write target time value to SYSTIM registers.
>
> Did I miss something, or the igb_ptp driver is currently broken for
> i350?
>
This is actually interesting :-)
Thanks for the question. I myself am just a cheeky bystander, not in
any way affiliated with Intel or anyone else here...
I merely feel an urge to add my two cents worth.

If memory serves, the i210 and i350 feel very much like siblings. The
launch dates at
ark.intel.com are not identical, but they seem to
share a lot in their feature set. Including e.g. the SDP pins.
The i82576 seems like a predecessor of the i350, similarly to the
i82574 being a predecessor to the i210.

If you look into the lengthy comment at the top of the igb_ptp.c:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tre
e/drivers/net/ethernet/intel/igb/igb_ptp.c

you'll notice that it describes the structure of the SYSTIM registers
in the i82576 vs. the i82580. This seems to be outdated for the
i210/i350, whose SYSTIM registers are "flat": 32bit seconds, 32bit
nanoseconds, 8bit fractional sub-ns.
Just check the datasheets of the i210/i350 and search for "SYSTIMR"=
the sub-ns fractional register. Gets you straight to the chapter,
describing the SYSTIM register contents and meaning.

Thus, I agree that whatever the logic was for the 82576/82580, for
the i350 I'd vote for a substitution of the i210-style access
routines. I.e., in igb_ptp_init(), use the igb_ptp_settime_i210() .

But, there may be another aspect.

If I understand correctly, you're trying to respond to an event at
the SDP input (1PPS in) by manipulating the SYSTIM register
immediately = stepwise. Possibly by resetting the SYSTIML to 0 (plus
maybe some estimate of your "IRQ response latency").
Note that if you're after ultimate precision, and you have a source
of razor-sharp 1PPS, this is not the right way to adjust your PHC.
Instead of introducing steps into your timebase, you can finely
adjust the frequency of your PHC. These chips have a synth that can
create a precise "virtual" clock out of your NIC's poor 25MHz Xtal.

Several years ago, Richard Cochran has published a minimal example
here on the mailing list:
https://sourceforge.net/p/linuxptp/mailman/message/36151546/
Actually... it is appropriate to do an initial stepwise setting, and
follow with continuous and gradual fine adjustment.

Why the code is possibly broken for the i350...
There's a strong possibility that my impression is off the mark.
But I can't help thinking in the following way:

The reference PCI-e board for the i210, by Intel, aka the i210-T1,
has always had a header, or at least a pad footprint, for the SDP
pins. Which has encouraged tinkerers to play, and driver maintainers
to update the driver.
In contrast, I've never seen an i350 board, where the SDP pins would
be exposed. Thus, possibly, noone ever found out, that the timing
stuff is not quite right.
I'm actually amazed that you have access to those SDP signals on an
i350, which is a BGA package. Do you have a custom designed board?

Also, as suggested above, ptp4l and possibly other software, adjust
the PHC by way of fine frequency adjustment. In the igb_ptp.c, I can
actually see igb_ptp_adjfine_82580() used for the i210/i211, as well
as for the i350. The function manipulates the TIMINCA register, which
has probably retained its semantics through the ages...
Which doesn't make perfect sense to me, because even without the SDP
pins exposed, when using HW timestamping for PTP, I'd expect ptp4l to
initially step-adjust the PHC, before starting the
continuous-convergent servo... Should the initial stepwise adjustment
fail, the subsequent continuous convergence would last pretty long...

The truth is out there, somewhere :-)

Frank


___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Is intel i350 ptp driver broken?

[Frantisek Rysanek]

On 25 May 2023 at 17:40, egg car wrote:
> 
> Hi, 
> 
> I'm trying to use i350 adapter with a hardware 1pps signal input on a 
> SDP pin.
> 
> I'm testing on kernel 5.19(i350 hardware ts capture function was 
> announced to be supported since kernel 5.17), using 'ts2phc' program 
> could capture the 1PPS input event, but it failed to write time 
> adjustment to the nic.
> 
> Then I looked into igb driver source code, found these two function 
> 'igb_ptp_adjtime_82576' and 'igb_ptp_settime_82576' calculated the 
> target timecounter value, but doesn't write the target value to nic 
> register. In contrast to  i210 routine which works 
> fine,'igb_ptp_adjtime_i210()' calls 'igb_ptp_write_i210()' method to 
> write target time value to SYSTIM registers.
> 
> Did I miss something, or the igb_ptp driver is currently broken for 
> i350?
>
This is actually interesting :-)
Thanks for the question. I myself am just a cheeky bystander, not in 
any way affiliated with Intel or anyone else here...
I merely feel an urge to add my two cents worth.

If memory serves, the i210 and i350 feel very much like siblings. The 
launch dates at ark.intel.com are not identical, but they seem to 
share a lot in their feature set. Including e.g. the SDP pins.
The i82576 seems like a predecessor of the i350, similarly to the 
i82574 being a predecessor to the i210.

If you look into the lengthy comment at the top of the igb_ptp.c:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tre
e/drivers/net/ethernet/intel/igb/igb_ptp.c

you'll notice that it describes the structure of the SYSTIM registers 
in the i82576 vs. the i82580. This seems to be outdated for the 
i210/i350, whose SYSTIM registers are "flat": 32bit seconds, 32bit 
nanoseconds, 8bit fractional sub-ns.
Just check the datasheets of the i210/i350 and search for "SYSTIMR" = 
the sub-ns fractional register. Gets you straight to the chapter, 
describing the SYSTIM register contents and meaning.

Thus, I agree that whatever the logic was for the 82576/82580, for 
the i350 I'd vote for a substitution of the i210-style access 
routines. I.e., in igb_ptp_init(), use the igb_ptp_settime_i210() .

But, there may be another aspect.

If I understand correctly, you're trying to respond to an event at 
the SDP input (1PPS in) by manipulating the SYSTIM register 
immediately = stepwise. Possibly by resetting the SYSTIML to 0 (plus 
maybe some estimate of your "IRQ response latency").
Note that if you're after ultimate precision, and you have a source 
of razor-sharp 1PPS, this is not the right way to adjust your PHC.
Instead of introducing steps into your timebase, you can finely 
adjust the frequency of your PHC. These chips have a synth that can 
create a precise "virtual" clock out of your NIC's poor 25MHz Xtal.

Several years ago, Richard Cochran has published a minimal example 
here on the mailing list:
https://sourceforge.net/p/linuxptp/mailman/message/36151546/
Actually... it is appropriate to do an initial stepwise setting, and 
follow with continuous and gradual fine adjustment.

Why the code is possibly broken for the i350...
There's a strong possibility that my impression is off the mark.
But I can't help thinking in the following way:

The reference PCI-e board for the i210, by Intel, aka the i210-T1, 
has always had a header, or at least a pad footprint, for the SDP 
pins. Which has encouraged tinkerers to play, and driver maintainers 
to update the driver.
In contrast, I've never seen an i350 board, where the SDP pins would 
be exposed. Thus, possibly, noone ever found out, that the timing 
stuff is not quite right.
I'm actually amazed that you have access to those SDP signals on an 
i350, which is a BGA package. Do you have a custom designed board?

Also, as suggested above, ptp4l and possibly other software, adjust 
the PHC by way of fine frequency adjustment. In the igb_ptp.c, I can 
actually see igb_ptp_adjfine_82580() used for the i210/i211, as well 
as for the i350. The function manipulates the TIMINCA register, which 
has probably retained its semantics through the ages...
Which doesn't make perfect sense to me, because even without the SDP 
pins exposed, when using HW timestamping for PTP, I'd expect ptp4l to 
initially step-adjust the PHC, before starting the 
continuous-convergent servo... Should the initial stepwise adjustment 
fail, the subsequent continuous convergence would last pretty long...

The truth is out there, somewhere :-)

Frank



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Intel 210 to Intel 255

[Frantisek Rysanek]

On 7 Jun 2021 at 11:42, Richard Cochran wrote:
>
> On Mon, Jun 07, 2021 at 02:01:28PM +, Geva, Erez wrote:
> 
> > Jun  7 15:44:07 ipc01 ptp4l: [673.869] timed out while polling for tx 
> > timestamp
> > Jun  7 15:44:07 ipc01 ptp4l: [673.869] increasing tx_timestamp_timeout may 
> > correct this issue, but it is likely caused by a driver bug
> 
> Try  --tx_timestamp_timeout=100
> 
> HTH,
> Richard
> 
I've taken another look, and I have to second that.
Richard is right.
I have "--tx_timestamp_timeout=20" in all my scripts, starting ptp4l 
in various configurations. I haven't started ptp4l by bare hand for 
ages.
That old LOM with broken timestamping would throw timeouts even with 
this setting.

Frank


___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Intel 210 to Intel 255

[Frantisek Rysanek]

...just my two cents worth, I haven't seen this error message for a
few years. I did get it back in 2017-ish on some older LOM NIC's by
Intel. The i210 has always been a workhorse for me.
This message is "local" = somewhere between the NIC HW and the
kernel-mode driver.
Then again, I'm not using the recentmost kernels. Maybe I should give
them a try. And I cannot rule out the possibility that some
development has happened inside the i210 firmware. My i210 hardware
is 2-3 years old now. My ptp4l isn't exactly bleeding edge either.

Umm... PCI-e ASPM comes to mind, maybe.
The master and slave are the same HW platform (motherboard model) ?
As you tried to "turn the table" on the bug, at various layers, have
you tried switching the motherboards? (It's just a silly theory, the
chances aren't high.)

Frank



On 7 Jun 2021 at 14:01, Geva, Erez wrote:

Hi,
 
I see an issue with the ptp4l using an Intel I210 connected back to
back to an Intel i255.
I am not sure if the issue is Intel related or is it ptp4l.
Probably something wrong that I did.
 
I use a very simple configuration:
On client side:
[global]
delay_mechanism P2P
network_transport L2
time_stamping hardware
slaveOnly 1
priority1 255
priority2 255
hwts_filter full
 
On source side:
[global]
delay_mechanism P2P
network_transport L2
time_stamping hardware
slaveOnly 0
priority1 100
priority2 100
hwts_filter full
 
I get this message on client.
I tried to switch sides of source, client and have the same message,
once on the I210 and once in the i255.
 
Jun  7 15:44:06 ipc01 ptp4l: [672.901] master offset    177 s2
freq  -34616 path delay   -10
Jun  7 15:44:06 ipc01 ptp4l: [673.434] port 1: FAULTY to LISTENING on
INIT_COMPLETE
Jun  7 15:44:07 ipc01 ptp4l: [673.869] timed out while polling for tx
timestamp
Jun  7 15:44:07 ipc01 ptp4l: [673.869] increasing
tx_timestamp_timeout may correct this issue, but it is likely caused
by a driver bug
Jun  7 15:44:07 ipc01 ptp4l: [673.869] port 1: send peer delay
response failed
 
I use both interface with auto-negotiation
 
root@ipc01:~# ethtool -i enp8s0
driver: igb
version: 5.10.30-rt37-tsn3-rt-ipipe
...
root@ipc01:~# ethtool  enp8s0
Settings for enp8s0:
   Supported ports: [ TP ]
   Supported link modes:   10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Supported pause frame use: Symmetric
   Supports auto-negotiation: Yes
   Supported FEC modes: Not reported
   Advertised link modes:  10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Advertised pause frame use: Symmetric
   Advertised auto-negotiation: Yes
   Advertised FEC modes: Not reported
   Speed: 1000Mb/s
   Duplex: Full
   Port: Twisted Pair
   PHYAD: 1
   Transceiver: internal
   Auto-negotiation: on
   MDI-X: on (auto)
   Supports Wake-on: pumbg
   Wake-on: g
   Current message level: 0x0007 (7)
    drv probe link
   Link detected: yes
 
 
root@ipc02:~# ethtool -i enp7s0
driver: igc
version: 5.10.30-rt37-tsn3-rt-ipipe
...
root@ipc02:~# ethtool enp7s0
Settings for enp7s0:
   Supported ports: [ ]
   Supported link modes:   10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   2500baseT/Full
   Supported pause frame use: Symmetric
   Supports auto-negotiation: Yes
   Supported FEC modes: Not reported
   Advertised link modes:  10baseT/Half 10baseT/Full
   100baseT/Half 100baseT/Full
   1000baseT/Full
   Advertised pause frame use: Symmetric
   Advertised auto-negotiation: Yes
   Advertised FEC modes: Not reported
   Speed: 1000Mb/s
   Duplex: Full
   Port: Twisted Pair
   PHYAD: 0
   Transceiver: internal
   Auto-negotiation: on
   MDI-X: off (auto)
   Supports Wake-on: pumbg
   Wake-on: g
   Current message level: 0x0007 (7)
    drv probe link
   Link detected: yes
 
With best regards,
Erez Geva

AURELLY TECHNOLOGIES GmbH 
External service provider at Siemens AG
Digital Industries
Process Automation
Software House Nbg
DI PA DCP R 3
Gleiwitzer Str. 555
90475 Nuremberg, Germany 
mailto:erez.geva@siemens.com
 

Re: [Linuxptp-devel] FYI: a summary scribble on my i210 SDP / PPS-in + 25 MHz ref etc sprees

[Frantisek Rysanek]

On 23 Mar 2021 at 10:05, Miroslav Lichvar wrote:
>
> That issue with oscillations around resolution of the clock, I think
> you could try smaller PI constants, or you could switch to the nullf
> servo, which is intended for cases like this.
> 
Yes I've tried fiddling with the P and I contributions and some other 
coefficients that I've found in that area, also with the parameters 
of the linear regression servo - to not much avail, if memory serves. 
I haven't tried the nullf servo - will give it a try when I have 
another opportunity (time and all the boxes together in the test 
rig). 

I did suspect some rounding error in the servo feedback stats/math, 
and I'm aware that there are conversions from the native data type / 
resolution coming out of the i210 HW registers (integer / fixed 
point), to the floating-point data types being used by the LinuxPTP 
servo library - but my relatively superficial conclusion was, that 
those anomalous "outliers" indeed come from the i210 hardware 
(timestamping registers). The offset reported is essentially the raw 
i210 TS register sample taken that particular second, maybe just 
type-converted from s32 to float - but no averaging / smearing / 
filtering on that.  Obviously I cannot see what's happening in the 
silicon - those details under the i210's hood are a bit of a mystery 
to me, and that's probably the way Intel would like them to remain 
:-)

I'm actually a little surprised that the servo typically is able to 
converge to a clean 0 and stay that way, with no random glitches, no 
flapping around the 8ns lsb boundary. I understand that the i210 HW 
does support fractional (below 1ppb) frequency adjustments - so it 
wouldn't be all that surprising to see scenaria where the frequency 
adjustment (which is some weighted average product of past offsets) 
would remain non-zero (fraction of a ppb) for a couple seconds and 
then the cumulative phase offset would exceed 8 ns, the servo would 
get a kick from the TS register and "fractionally creep" the other 
way. This doesn't seem to be the case... 

I have implemented the following trigger in my own code of cmp.c and 
cmp2.c

   if ((offset == 0) && (abs(ppb) < 0.01))
  phcs[i]->aligned_periods++;
   else
  phcs[i]->aligned_periods = 0;

   if (phcs[i]->aligned_periods > 4)   {
  // switch to measurement stage
  phcs[i]->initial_sync_stage = 0;

...etc
I.e. after 4 servo periods where the ppb offset was within +/- 10^-6,
I'd call the servo settled forever.
This is my heuristic based on practical observation that, while not 
entirely settled, there would be some fraction on the order of 10^-1 
or 10^-2 fluttering in the ppb, which would result in an occasional 
non-zero phase offset - but eventually I'd get the ppb float residues 
way below 10^-6 and they'd stay that way forever. Servo converged.
Which is where I'd start to take my measurements.
I was certainly hoping for this to happen while I was building my 
25MHz synth :-)

Then afterwards, when I distilled that "anomaly of not settling",
I also tried just nailing ppb to 0 the moment when phase offset 
becomes 0 - this is still present in a commented section at line 625 
in my i210_ext_pps.c .
And, it didn't work. In the anomalous state, despite the ppb being 
bolted to a clean 0 and written to the PHC, the phase offset (TS 
register output) would run at 0 for a couple samples, then give a 
shot of 8 ns, then probably return to zero...

Actually, I've spotted a similar behavior on part of the linreg 
servo, possibly as an inherent effect of the algorithm. After a 
couple dozen samples with phase offset = 0 (and ppb fluttering on the 
order of 0.1 to 0.01), the ppb value suddenly drops to a pure 0.
(This doesn't happen with the PI, where I can always see some debris 
reported below 10^-14 ppb if I count the zeroes correctly.)

While looking for a possible explanation, in the new v3.1 I've found 
SERVO_LOCKED_STABLE and servo_offset_threshold / 
servo_num_offset_values (config parameters). This logic is absent in 
v1.8, and likely not related to the behavior of the linreg servo 
(converging to pure zero ppb).
And I'm wondering if I should add SERVO_LOCKED_STABLE to the switch 
case that does clockadj_set_freq() in my proggies :-)  So far I only 
check for SERVO_LOCKED .  == I'll have to return to that, even though 
SERVO_LOCKED_STATE is possibly quite a corner case, with the 
servo_offset_threshold configured for 0 by default.

I hope to check that stuff again in a few weeks.

Frank



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] FYI: a summary scribble on my i210 SDP / PPS-in + 25 MHz ref etc sprees

[Frantisek Rysanek]

On 23 Mar 2021 at 8:34, Frank Rysanek wrote:
>
> On 22 Mar 2021 at 21:15, Richard Cochran wrote:
> > 
> > FYI, the new ts2phc program does everything that synbc did, and
> > more.
> > 
> > Thanks,
> > Richard
> 
> Thanks for the tip, I'm gonna have to try that :-)
> 
...allright, reading through the manpage, it's tugging at my sleeve 
that my next project in this vein would be a DIY Boundary Clock (sort 
of - not necessarily a bridge and certainly not a HW switch) that 
would contain a custom 25 MHz VC-OCXO in the servo loop :-)
But this is probably gonna have to wait, as I have no immediate use 
case for that...

Frank


___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] FYI: a summary scribble on my i210 SDP / PPS-in + 25 MHz ref etc sprees

[Frantisek Rysanek]

On 22 Mar 2021 at 21:15, Richard Cochran wrote:
> 
> FYI, the new ts2phc program does everything that synbc did, and more.
> 
> Thanks,
> Richard

Thanks for the tip, I'm gonna have to try that :-)

Frank


___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

[Linuxptp-devel] FYI: a summary scribble on my i210 SDP / PPS-in + 25 MHz ref etc sprees

[Frantisek Rysanek]

Dear everybody,

I've posted some of my source code before,
based on Richard Cochran's synbc.c .

Meanwhile, I've done some DIY-level soldering and "PC system 
integration" around i210, PPS for reference, 25 MHz reference,
PPS deviation measurement... lots of fun that I thought I should not 
keep to myself.

Also, it helps my sanity to swap out things that I'm kind of done 
with "onto paper" (or HTML works for that purpose too).
Permanent write-only external memory.
There are things I can publish and there's also stuff that I should 
not talk about. I've tried to distill relevant technical info that 
does not harm anyone or violate any agreements.

Anyway the time has come for me to post a link.
Here goes my newest scribble on the aforementioned topics:

http://support.fccps.cz/industry/i210_hacks/i210_hacks.html

Thanks everybody for your respective past contributions, and in 
general for the work that you keep doing on LinuxPTP.

Frank Rysanek



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] SyncE support

[Frantisek Rysanek]

On 17 Mar 2021 at 18:01, Miroslav Lichvar wrote:

[on PCI-e PTM]
> 
> The switches are supposed to work like NTP server and client at the
> same time (boundary clock in the PTP terminology), so all PCIe links
> have hardware timestamping on both ends.
> 
> BTW, at least in theory, a network using boundary clocks should
> perform better than a similar network using transparent clocks,
> assuming the servos are well configured and the sync interval is short
> enough to minimize the time errors in the chain. Divide and conquer
> :). I think transparent clocks are meant to be the simpler and cheaper
> variant.
> 

I recall some history around the debate on BC vs TC.
>From my subjective perspective anyway.

PTP v1 used to be all about BC's.
Then in 2008 came PTP v2, and suddenly TC was all the rage, the only 
right way for PTP-capable switches to function from now on.
Except that, as years were passing by, PTP-aware switches did not 
exactly spawn in flocks everywhere. Rather, they remained few and far 
between. And, especially the telco industry would've loved to deploy 
PTP, but there was no way for their active elements in the networks 
to support PTP (to turn them all into TC's). So after some years 
since 2008, suddenly the Telecom Profiles came out, and were all 
about BC's and partial on-path support (or rather: no on-path 
support). The TC idea has a stronghold in the electric power 
industry, at least on paper - practical compatibility of 
implementations remains a fun topic, to use a euphemism, especially 
where multi-vendor heterogeneous setups are attempted... this is 
where I have a limited by ongoing practical experience :-)

On the topic of BC's and their apparent renaissance, I've had a brief 
debate about this with someone from Meinberg, and the message was: 
BC's or TC's, either of them works, if done properly. Reportedly 
there have been tests of up to 16 BC's in a cascade, and if the local 
oscillators were stable, and the feedback loops were tuned 
appropriately, the cascade just worked, there were no ill effects 
(excessive residual wander at the tail end, or whatever). So it's all 
down to oscillator quality and feedback loop parameters = control 
theory and the respective math. Laplace, Nyquist, whatever...

As for TC's, from my practical observations (and maybe a bit of 
gossip), switch chipset vendors have a hard time implementing the 
on-the-fly corrections in the hardware. Something that would work 
"out of the box" for the switch box makers. And because the silicon 
vendors struggle, some switch box vendors implement PTP on their own, 
mangling MII on the fly using custom FPGA designs etc.
You probably know the ugly details an order of magnitude better than 
I do, i.e. what it takes to make a TC switch, for P2P mode vs. E2E 
mode... The "TC support in a switch" is likely never going to be pure 
hardware, there's always an element of switch firmware that e.g. does 
the talking with P2P peers... While E2E might look easier to support 
in a switch (than P2P), I've heard a credible opinion that in E2E 
mode, a switch port needs to keep track of delay transactions passing 
through (stateful style - ORLY?) and making E2E work for a single 
slave is not the same as making it work for many slaves :-) etc.


I've wandered even further off topic here...

In PTM, having a servoed local oscillator in every PCI-e switch (BC 
style) seems complicated, to the point of being impractical / 
difficult to implement, if the oscillator should be half decent / 
stable...

In my wildest dreams, it would be beautiful to have a 1PPS output out 
of the CPU TSC. Plug that into some SDP input on an i210, and you'd 
have an idea, where the CPU's PPS edge is located, relative to the 
i210 PHC. Which can in turn be disciplined by PTP, or by another 
external 1PPS reference. Just a single output wire, coming from the 
TSC - it wouldn't even need to be adjustable. Having a timestamp for 
that CPU 1PPS event from the i210, you'd be able to calculate an 
offset in TSC granularity, and subtract that offset from every 
timestamp obtained via rdtsc (at the cost of a single integer sub() 
instruction). A single signal, for 1PPS, coming out of the CPU 
socket. Is that too much to ask? Compared to the rather complex and 
imperfect PTM implementation?

Actually this is not my wildest dream. I can extrapolate this 
further. Such as:

After the all, the TSC stands for a "Time Stamp Counter" - isn't that 
ironic?  As it is, the TSC is just a register counting steadily 
forward. What reference does it actually have to the wall time in the 
outside world, apart from the CPU core clock? (Yes I know - now with 
TurboBoost, even the notion of a CPU clock is hazy.) I mean wouldn't 
it be nice if the CPU would have a discrete event *input* into the 
TSC? Typically useable as 1PPS *input*. Having a timestamping 
register (MSR?) on the inside, a neighbor to the TSC, which would 
always contain a timestamp of the last exterior 1PPS 

Re: [Linuxptp-devel] SyncE support

[Frantisek Rysanek]

On 17 Mar 2021 at 13:04, Miroslav Lichvar wrote:
>
> Right, but in most cases I see people are interested in the accuracy
> and stability of the system clock. There don't seem to be many
> applications using hardware timestamps of the PHC.
> 
> In some cases a sub-microsecond accuracy is required. The CPU<->NIC
> connection is the most problematic link in the chain. You can buy
> switches with good PTP support and compensate for asymmetric errors in
> hardware timestamping to get the PHC accurate to few nanoseconds, but
> due to the huge PCIe latency with an unknown asymmetry it's difficult
> to tell how accurate the system clock actually is. With the same NIC
> but different CPUs/boards you can get very different results.
> 
> > As for getting the Linux system timebase precise, I guess the random
> > component to ISR latency has several contributing factors, the PCI-e
> > bus latency being only one of them...
> 
> Normally there should be no interrupts involved. The time critical
> part is just a single read of a 32-bit PCI register. With the I210
> that takes about 1700 ns and the asymmetry in my measurements was up
> to about 100 ns. The shortest delay I saw with faster NICs was about
> 450 ns, asymmetry unknown.
> 
Thanks for all the data and clarifications :-)
That's interesting...

I'm not sure the PTM (that you have mentioned previously) will help 
with asymmetry. You characterize the mechanism as "NTP-like". I'm not 
a member of the the PCI-SIG, and the only clear text about the PTM 
that can be found in Google is some change notice by the PCI-SIG from 
back in 2013, referring to PTM v1.0a. 

https://fdocuments.in/document/precision-time-measurement-ptm.html 

That paper clearly says (I'm quoting): 

"Therefore ((t4 - t1) - (t3 - t2)) effectively gives the round trip 
message transit time between the two components, and that quantity 
divided by 2 approximates the Link delay - the time difference 
between t1 and t2. It is assumed that the Link transit times from PTM 
Requester to PTM Responder and back again are symmetric,   
which is typically a good assumption."

The current revision listed at the PCI-SIG website is v4.0.
Which makes me wonder if this assumption is still considered valid 
:-)

I understand that apart from the NIC (peripheral device / endpoint 
function in general), PTM must also be supported by the root complex 
and any PCI-e switches along the path... so a PTM-capable NIC alone 
will not even suffice for this simple NTP-style mechanism to 
function, if the host chipset is unaware of PTM.

To account for asymmetry, the mechanism would have to resemble PTP = 
there would have to be a correction field in the PCI-e messages and 
the PCI-e switches would have to work like PTP TC switches :-)

Frank



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] SyncE support

[Frantisek Rysanek]

On 16 Mar 2021 at 16:59, Miroslav Lichvar wrote:
>
> I'm sorry for misleading you. I meant the ice driver (speeds up to
> 100Gb). I didn't see an out-of-tree version of the igc driver.
>
Okay, thanks for that update...
Still I'm curious what news the i225 may bring in terms of e.g. GPIO 
timestamping resolution. Does it get better than those 8 ns 
I got used to with the i210 etc.
 
> The I225 might be an interesting NIC for experiments for a different
> reason. It seems it supports the PCIe Precision Time Measurement (PTM)
> feature. It is a hardware implementation of an NTP-like protocol over
> PCIe, which should allow a highly accurate synchronization of the
> system clock, avoiding the asymmetry on PCIe. It is not supported in
> the igc driver yet, but there were some patches submitted for it.
> 
> I measured the PCIe asymmetry with the I210 on few different
> boards+CPUs and it changed a lot (between about -100ns and 100ns), so
> I think PTM could make a significant improvement.
> 
Erez Geva was quicker to respond, and I agree with him:
the packet timestamping is done by the NIC HW, so unless you have an 
application where you need accurate timing all the way to the 
software-based system clock, the determinism of PCIe bus latencies 
should not matter much...
I understand that you being a RedHat insider, a principal maintainer 
of Chrony and a maintainer of the RedHat linuxptp RPM, your 
application is precisely getting the system clock as solid as 
theoretically possible :-)

So far, I've only ever considered PTP and its breathtaking accuracy 
to be any use in dedicated hardware: switches with HW PTP support 
including a central stable oscillator, or pure slaves that syntonize 
a *stable* local hardware clock to the NIC's PHC and use that precise 
HW clock for some further, thoroughly HW-implemented purpose...

As for getting the Linux system timebase precise, I guess the random 
component to ISR latency has several contributing factors, the PCI-e 
bus latency being only one of them... also, the crystal oscillator 
serving as a "stable" local reference for the PC chipset clock  
synth is in reality not very stable. I don't have a precise figure, 
but if I take the typical i210 NIC 25 MHz xtals for a benchmark,
I've seen instability translating into ppb deviations of about +/- 10 
to +/- 20 ppb, within the 1s period of ptp4l sampling - and a PC 
chipset is likely to have an even worse oscillator...

BTW, speaking of NICs with PTP and SyncE support, the only one
I know is a custom FPGA-based design by Oregano:
https://www.oreganosystems.at/products/syn1588/hardware/syn1588r-pcie-
nic
Obviously that NIC is irrelevant to Linuxptp, as the FPGA-based 
Oregano stack implements the whole protocol...
and that NIC is obviously not cheap :-)

Thanks for spending your time, responding to my novice questions...

Frank


___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] SyncE support

[Frantisek Rysanek]

On 16 Mar 2021 at 11:25, Miroslav Lichvar wrote:
>
> In the Intel's igc driver I saw few SYNCE registers defined,
> but no code using them.
> 
Whoa. igc ? oh there's an i225... didn't know about this one, thanks 
for the pointer, this definitely looks promising :-)
Looks like a successor to the i210 generation. 

If I read the gossip correctly, the first generation of i225 are in 
circulation on some gaming motherboards from the first half of 2020, 
and have a hardware bug, something to do with the inter-packet gap.

https://www.techpowerup.com/266335/intel-i225-foxville-2-5gbe-phy-has-
a-flaw-affecting-performance-rocket-lake-s-2h-2020-production-confirme
d

A version 2 of the silicon, supposedly produced since Q4/2020, has 
this bug fixed. There's no public datasheet, just a data brief and a 
spec update (a list of SKU's + the erratum for v1 inter-packet gap 
bug.)

https://www.intel.com/content/www/us/en/design/products-and-solutions/
networking-and-io/ethernet-controller-i225/technical-library.html

And, I can actually see a stand-alone PCI-e board mentioned at 
ark.intel.com - the i225-T1 :
https://ark.intel.com/content/www/us/en/ark/products/211808/intel-ethe
rnet-network-adapter-i225-t1.html

I cannot see this in the price lists at disties yet, will keep an eye 
on that one :-) The board may surface sooner from HP, Lenovo or 
Dell... There are no photoes yet - I'm curious if it has an SDP pin 
header, similar to the i210-T1.

And this one looks cool too:
http://www.commell.com.tw/Product/Peripheral/PCI%20Express%20mini%20ca
rd/MPX-225.HTM

Frank



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] SyncE support

[Frantisek Rysanek]

Dear gentlemen,
thanks for opening this interesting topic...

Is there any stock NIC (silicon, board), with an open-source Linux 
driver, that would support SyncE off the shelf in the NIC hardware?

Other than that... SyncE sounds like a pretty self-contained L1 
feature, at least considering just a single NIC port. Obviously the 
routing of clock signals among possibly several ports in a system and 
a local master oscillator - that would be a more complicated, 
proprietary story.
But at the level of a single port... what do you need?
SyncE enable / disable, and select master vs. slave role?
It would take a fairly simple extension to some existing API - makes 
me wonder which one: MII, Ethtool, Netlink, or maybe the timestamping 
API? I'd guess Ethtool or Netlink would be the right level of 
abstraction. Or some nodes in procfs or sysfs :-)
MII is too HW-specific and the timestamping stuff operating on top of 
setsockopt() feels a little too detached from hardware.

And yes there is some additional messaging, I understand in-band in 
Ethernet, which would probably have to be handled by ptp4l software, 
unless offloaded to the kernel-space driver or hardware...

Frank Rysanek


___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] More fun with i210 EXTTS: measure some PPS against a reference PPS

[Frantisek Rysanek]

On 6 Mar 2020 at 16:42, linuxptp-devel wrote:
> And the funny point is: while the E2E transactions result in no 
> asymmetry, the P2P transactions result in about +60 nanoseconds
> of additional asymmetry. That's curious...
>
...actually I should've quoted that as -60 nanoseconds.
In addition to the -20 ns on a direct connection.
The total hovers around -80 ns offset.

Frank


___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] More fun with i210 EXTTS: measure some PPS against a reference PPS

[Frantisek Rysanek]

Just a snippet of a funny observation:

I've discovered the config parameter "clock_servo nullf" .
Very good - that answes another need I had:
to let the PHC freewheel along the ultimate external reference
(10MHz -> PLL_synth -> 25 MHz from the PTP GM out of band),
and let ptp4l do the talk and calculate the delays and offsets 
that it can obsereve on the network.
It seems to work fine. On a real-wold network I may also need 
to raise the bar for step-wise adjustment (for this measurement to 
proceed unhampered).

Just to test if it works, I've tried L2 multicast through an old 
D-Link DGS-1216T. It does work, the (p)delay is about 2 microseconds.
The switch does let both P2P Pdelay and E2E Delay through just fine.

And the funny point is: while the E2E transactions result in no 
asymmetry, the P2P transactions result in about +60 nanoseconds
of additional asymmetry. That's curious...

That's over two metallic gigabit hops.
For comparison, straight against the GM, I have about 7 or 10 ns of 
(p)delay and about -20 ns of asymmetry, which may well correspond to 
the length of my PPS cabling (that I use to pre-settle the PHC).

Yes I'm having fun with my toys :-D

Frank


___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] More fun with i210 EXTTS: measure some PPS against a reference PPS

[Frantisek Rysanek]

On 5 Mar 2020 at 17:01, Jacob Keller wrote:
> On 3/5/2020 1:11 AM, Frantisek Rysanek wrote:
> > And another sideways question is: in the i210 hardware, there's a 
> > register called SYSTIMR, supposedly holding the fraction of a 
> > nanosecond (= sub-nanosecond resolution). And this register is 
> > ignored by the igb driver in Linux - first and foremost because the 
> > internal timestamping infrastructure only supports nanosecond 
> > resolution. I know that a "ns fraction" field is present in the PTP 
> > frames, but everybody except the White Rabbit just leave that field 
> > empty (all zeroes). I'm wondering if this SYSTIMR register in the 
> > i210 hardware has some practical use, or is always zero, or what.
> > Well for my practical purposes, the SYSTIMR does not get reflected in 
> > the two AUXSTMP registers - so I can probably just ignore SYSTIMR 
> > too.
> 
> So, the SYSTIMR field is not "used" directly, but it holds and maintains
> fractional nanoseconds. When you adjust the increment value slightly,
> these get added to the SYSTIMR field of the system time. As that slowly
> increments and eventually overflows, it will then increment the SYSTIML
> register.
> 
> Essentially we always round down by cutting off SYSTIMR.
>
Mr. Keller thanks for your polite explanation :-)

I have to say that the i210 is a very nice piece of silicon to play 
with :-) I'm aware that at 1 GHz / fractions of a ns, it takes some 
cunning design to make a synth with this kind of fine adjustment, 
immediate response and nanosecond resolution,
with hardly any jitter/phase noise. It's a job well done.

The fact that timestamping external events is granular at 8 ns 
("only") does not offend me. I'm aware that it's difficult
to run counters and atomic latches that fast.
I've read something about WhiteRabbit's phase detector,
called the DDMTD - which supposedly can measure phase
differences down to the picosecond range. If I understand correctly,
that comparator block depends on having isochronous clocks
(the reference and the measured input) much faster than 1PPS
and hinges on down-mixing those clocks, then phase comparison
and statistics (filtering) to arrive at that fine-grained result.
Hence also WhiteRabbit's dependence on SyncE,
and the BC-only nature of their PTP-aware switches...
I understand that this is what it takes to run time-sync
over a WAN at ns/sub-ns level preecision...

Frank Rysanek



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] More fun with i210 EXTTS: measure some PPS against a reference PPS

[Frantisek Rysanek]

...so I've tried myselfs. 
I've re-written my proggie to first settle the PHC using EXTTS CH0
to the external reference PPS, and then switch EXTTS CH0
to the measured PPS signal (reconfigure the CH0
to take input from the second SDP input).
Guess what: the measured deviations are still quantised per 8 ns.

So to sum up:
during the initial settling of the PHC frequency / servo loop,
I can see offsets in low units of ns, not aligned in any way.
But: once the PHC settles to offset==0 && ppb==0,
any measured edges captured via EXTTS channel 0 or 1
will be quantised on 8ns boundaries.
This is kind of spooky :-)
Maybe the quantisation just doesn't jump at me so bad
while the frequency (ppb) is still a little off...
and it's true that if I keep capturing on both channels,
their mutual difference is always quantised at 8 ns,
even though the difference against the internal unsettled
PHC is not aligned that way.

I'm giving up this train of research.
Comments welcome :-)
8 ns granularity is better than nothing at all.

Frank
The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  i210_ext_pps_cmp2.c
 Date:  5 Mar 2020, 17:01
 Size:  28791 bytes.
 Type:  Program-source


i210_ext_pps_cmp2.c
Description: Binary data
The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  i210_ext_pps_cmp.c
 Date:  5 Mar 2020, 16:57
 Size:  26939 bytes.
 Type:  Program-source


i210_ext_pps_cmp.c
Description: Binary data
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

[Linuxptp-devel] More fun with i210 EXTTS: measure some PPS against a reference PPS

[Frantisek Rysanek]

Dear gentlemen,

after two years of silence, I've played some more with my multiport 
i210 test rig... I now have 4 ports of i210, two of them metallic, 
two of them fiber optic (with Gigabit SERDES SFP slots).
And I've hacked the optical cards to have SDP0 and SDP3 inputs,
as well as the obvious 25MHz reference input (referenced by a PLL 
synth to 10 MHz phase-synchronous with PPS from a GPS clock).
The metallic cards have all four SDP inputs available ex works on a 
header.

= I have a box with 4 ports of i210, their 25 MHz referenced to GPS,
phase-synchronous with a PPS reference that I'm using to discipline
the four PHC's...
And, I have an extra SDP input and EXTTS channel per i210 PHC,
available for misc timestamping purposes (right now for PPS deviation
measurements).

I'm attaching an example proggie and a snippet of its TXT output.

And finally to the point = my question for today:

The servo loop that I'm using to discipline the PHC based on external 
PPS - that converges smoothly while printing a convergent row of 
nanosecond numbers. After a couple seconds, the offset and frequency 
end up at 0. This is what I achieved two years ago, based on 
Mr.Cochran's work - I got the whole thing basically finished from 
him.

The series of numbers, produced by the servo lop convergence, seems 
clearly granular down to a nanosecond.

But, the funny point is: if I use a second EXTTS channel to timestamp 
a second PPS input, that second input seems "quantised" at 8ns 
granularity to the reference PPS input.
To the extent, that while the reference servo hasn't entirely settled 
yet, I'm getting non-quantised readings from the second EXTTS channel 
too - non-quantised against the internal timebase. But, always 
quantised at 8 ns granularity against the reference PPS input.

I've read in the i210 datasheet that 
"During run time the SYSTIM timer value in the SYSTIMH, SYSTIML and 
SYSTIMR registers, is updated periodically each 8 nS clock cycle" etc
And there's a note in drivers/net/ethernet/intel/igb/igb_ptp.c that
"The 82580 timesync updates the system timer every 8ns by 8ns,
and this update value cannot be reprogrammed."

So the i210 has probably moved ahead a little, since the 82580 
generation, but in some form that 8ns granularity is still there.

I'm wondering if I should first discipline the clock by PPS ref 
plugged in the EXTTS CH0, wait for the frequency offset to settle 
down to 0 for a few periods, then stop updating the servo
and switch the EXTTS CH0 to the "measured PPS signal".
If that can possibly yield the desired granularity down to a 
nanosecond. I will probably just try :-)

Any comments welcome.

And another sideways question is: in the i210 hardware, there's a 
register called SYSTIMR, supposedly holding the fraction of a 
nanosecond (= sub-nanosecond resolution). And this register is 
ignored by the igb driver in Linux - first and foremost because the 
internal timestamping infrastructure only supports nanosecond 
resolution. I know that a "ns fraction" field is present in the PTP 
frames, but everybody except the White Rabbit just leave that field 
empty (all zeroes). I'm wondering if this SYSTIMR register in the 
i210 hardware has some practical use, or is always zero, or what.
Well for my practical purposes, the SYSTIMR does not get reflected in 
the two AUXSTMP registers - so I can probably just ignore SYSTIMR 
too.

I'm wondering if it's safe to assume that once the servo has 
converged to 0 frequency offset, that I can stop updating the 
frequency (knowing that I have the PHC's upstream oscillator 
disciplined "out of band"). Or if there's some fractional part
that can cause the internal PHC clock to drift (the internal
PHC clock is synthesized based on the XTAL as a reference
and a correction value updated by the servo loop).
Theoretically I could just set the frequency correction to 0.
Not sure if the ptp4l internal API allows me to do that.
And the servo has its role in the initial adjustment/alignment
of the PPS event in time, as the servo mechanism ends up
being more precise than a direct register write (that's only
any good as a coarse initial stepwise adjustment).

Again any comments welcome, thanks for your attention, have a nice 
day :-)

Frank Rysanek

The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  ext_sync_example3.txt
 Date:  5 Mar 2020, 9:07
 Size:  16184 bytes.
 Type:  Text


ext_sync_example3.txt
Description: Binary data
The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should 

Re: [Linuxptp-devel] [PATCH] phc2sys: Add external time stamping support

[Frantisek Rysanek]

...again just a few bystander opinions on my part:

On 5 Feb 2019 at 9:37, Miroslav Lichvar wrote:
> On Mon, Feb 04, 2019 at 11:31:01PM +, Vladimir Oltean wrote:
> > On 2/4/19 2:53 PM, Miroslav Lichvar wrote:
> > > I was thinking about a similar approach. I'm not suggesting to use a
> > > second time source (e.g. a GNSS refclock) to pair the pulses with real
> > > time, only to synchronize the PHC to the nearest second using the PPS
> > > input. The PHC can be initially set by ptp4l from the network, by
> > > phc2sys from the system clock, or something else from the refclock.
> > > Then phc2sys using the PPS input can take over and keep it
> > > synchronized to TAI (no leap seconds). ptp4l can work as a grandmaster
> > > on the PHC. That is my use case.
> > 
> > Who guarantees that the switchover time between the states "1. NTP GNSS 
> > refclock driver disciplines system clock", "2. NIC PHC takes a sip of 
> > system time for initial time of day" and "3. PHC keeps chugging along 
> > keeping time of day based on PPS only" will not cause the NIC PHC to 
> > lean towards the wrong second, throwing its absolute value off by one?
> 
> phc2sys can require the clock to be very close to the pulse edge, e.g.
> 10 milliseconds, and refuse to operate when the offset is larger. We
> can make some assumptions about maximum frequency error of the clock
> (e.g. 500 ppm) and calculate the maximum time for phc2sys to lock.
> 
I believe that we should leave some responsibility to the admin :-)

"Dear Admin:
based on your configuration, you probably want to serve PTP with no 
upstream PTP source. You want to serve PTP using a NIC-based PHC 
disciplined by the PPS. Note that the PHC has no inherent notion of 
'time of day' etc. We can initialize the PHC's wall time down to the 
second from the system clock, but THE RESPONSIBILITY IS YOURS to 
provide your Linux system clock with valid time, before you start the 
PTP service. Principally in theory, for a successful PHC 
initialization, your Linux system time should be accurate within 
< +/- 0.5s of the PPS edge that you provide to the PHC. Preferably 
much more accurate, for optimal results / safe PHC init."

Note that low single-digit milliseconds of accuracy are possible with 
NTP off pool.ntp.org, i.e. using servers in the wild internet.

There are so many different ways of providing plausible time to the 
Linux system clock... Let the admin pick one, using his best 
judgement - and let us take that "time reference" to initialize the 
PHC, and from there use the PPS input to servo the PHC.

If the admin has an accurate PPS source that he can plug into our 
i210 PHC, does it even matter where the system clock's Time of Day 
information came from? Whether NTP from the wild internet, or 
incidentally that same GPS refclock providing the PPS ?
On a decent timing GPS, the serial string alone is enough to get
millisecond-level accuracy... 

And one more note specifically on:
> > Who guarantees that [...] will not cause the NIC PHC to lean 
> > towards the wrong second, throwing its absolute value off by one?

The algorithm in my proggie works in two steps:

1) ignoring the system clock or the absolute PHC time, just calculate 
the distance (nanoseconds) between the previous PPS event and the 
current PPS event. If the distance is short(-ish), probably the 
previous event was a leading edge and the current event is a trailing 
edge. We are interested in the leading edge, that's the whole-second 
boundary. 
(My algorithm actually looks for a particular expected pulse length 
+/- some margin, compile-time constants.)

2) now that we know the leading PPS edge, calculate its bipolar 
offset from the PHC's current whole-second boundary (which will be 
within +/- 0.5s) and start tuning the servo loop from there.
Actually there should be a step 0) where the absolute PHC time is 
initialized by copy from the Linux system clock.

So unless the system clock (used as initial reference) is off by more 
than 500 ms against the PPS edge used to discipline the PHC,
the algorithm will converge to the correct second boundary.
Simple quantisation math.

If the system admin cannot get his act together, we can hardly decide 
this for him algorithmically (not having another reference to compare 
against). Though as Mr. Lichvar says, we could use an actual offset 
past some pedantic margin as a statistically probable indication that 
the admin has not done his homework :-)

> A separate phc2sys instance can be used for monitoring the offset
> between the system clock and PHC.
> 
"just in case", I understand... to report errors into syslog, keep a 
watch in Nagios or something. Normally the deviation should not grow 
over time, if both the PHC and the system clock are disciplined, even 
if disciplined independently as we're suggesting here.

> > And why pass GPS -> NIC PHC disciplining through the system time at all? 
> 
> I think that would be the easiest approach using the existing tools.
>

Re: [Linuxptp-devel] [PATCH] phc2sys: Add external time stamping support

[Frantisek Rysanek]

d served to userspace as extts events).
>
This sure is an interesting angle :-)

You're right - the NIC's contain a PHC = effectively a timebase with 
some synthetic frequency adjustment capabilities, that can provide 
precise nanosecond-level timestamps between PPS edges. 
And apparently the cascade of counters/dividers can also count 
seconds from epoch, POSIX style.

As you correctly point out, the PHC's don't have an NMEA input.
Or some such.
I'd like to add that this was probably never intended.
The NICs' timestamping PHC is likely geared for use in PTP.
As PTP slaves in the first place.
Would anyone want a bread-and-butter NIC to contain a GPS refclock 
driver? Just NMEA, or a plethora of other proprietary string formats 
as well? It would need a serial UART on chip, and some pins to waste,
and the protocol is best crunched by an MCU running some code...

Note that once you get the PPS input ticking, the PHC will keep 
counting seconds from the UNIX epoch straight on, forever.
...so... what would you need NMEA input *for* in the first place?

Initialization of the PHC to the correct Time of Day = "seconds from 
epoch"? That's not a big problem, you can always initialize that part 
from the system time, if that's half decent. 
All you need is some other means of syncing the Time of Day.
NTP gets you to millisecond precision over a modern WAN.
NTP has refclock drivers for a number of proprietary radio clocks and 
string formats.

But wait: there's a chicken-and-egg problem.
To discipline your Linux system timebase from a NIC PHC,
you first need to turn ntpd off.
So you'd need to first run ntpd or chrony for a while, or take a sip 
of NTP using ntpdate, and only then start phc2sys.

And then there are leap seconds.
Does phc2sys know or care about leap seconds?
NTP does get to know about them, from upstream NTP or from a radio 
refclock. But we need to shut down ntpd in the first place...

Note that PTP is using the TAI as its time domain.
So the PHC's actually can tick forever straight on,
just counting seconds, oblivious of leap seconds.
Leap seconds are transported as a data attribute in the PTP protocol.

If you have NTP in the host system, then NTP disciplines the clock 
and tries its best to work around leap seconds. Nowadays with some 
support in the Linux kernel timebase, AFAIR.

POSIX doesn't know about leap seconds, and NTP can use 
several different bad ways (pick your poison) to cope with them.
PTP has its inherent TAI timebase and the UTC is derived using just a 
simple offset attribute (which accumulates any leap seconds over 
time).
The question really is, how the PTP / phc2sys software stack in Linux 
gets to know about leap seconds and who's responsible for taking care 
of them.

How about... configure ntpd to prefer the local system clock 
"pseudo-refclock driver" (which should disengage the system clock 
disciplining ambitions of ntpd), add one upstream NTP peer to get to 
know about leap seconds, and run phc2sys to discipline the system 
timebase from a PPS-based PHC? If all of this works, and ntpd gets to 
know about leap seconds, will it notify the Linux kernel timebase, 
while it's told not to tweak it? Will phc2sys get to know about leap 
seconds? Does it even care?



Let's abstract from the leap seconds again for a bit.
Forget leap seconds for now.

So far the canonical use of linuxptp has been:
- run ptp4l to turn a nic into a PTP slave = to speak the protocol,
   get to know about TAI/UTC offset etc.
- run phc2sys to sync the system time base to that "PTP slave PHC"

In that case, Miroslav's idea to just "add support for ext PPS"
in phc2sys (or in ptp4l?) would be the low hanging fruit.
Despite the fact that a PTP slave NIC is typically used as a PPS 
*source*, rather than a PPS input... now this sounds a little 
unconventional.

What if we don't have ptp4l running?
What if we don't have a PTP GM available?
Would ptp4l let the slave PHC free-wheel forever, 
if there is no upsteam PTP GM?

Would it be possible to e.g... start ptp4l in the GM role,
on a particular NIC port, enable the ext.PPS on this "PTP GM NIC"
(which would be the more typical place to enable ext.PPS)
and then * run phc2sys against that PTP GM NIC PHC * ?
Uhh... if I run ptp4l on a PC in a pure GM role, where does it take 
its time from? The system time base? Wouldn't that give rise to a 
funny feedback loop / chicken-and-egg race condition ?


In my packet capturing lab machine I do have the system clock 
disciplined by ntpd or by ptp4l on a dedicated Ethernet port. 
I'm not running phc2sys to take time from the PPS-disciplined PHC's.
External PPS is input to the PHC's dedicated to precise packet 
capture. (Accompanied by 25 MHz synthesized from GPS-based 10 MHz.)


I'm ranting about things I know nothing about.
I should probably read the code... except I don't have the time :-/
Apologies for my ignorance and thanks for your polite attention.

Frank

> Fra

Re: [Linuxptp-devel] PHC delay when calling clock_gettime

[Frantisek Rysanek]

Dear gentlemen,

I've been following your debate, and while it's a sin to comment off 
topic, it's difficult for me to hold back the following smirk:

i82579V/LM brings back a recollection of a pretty famous bug
discovered a few years ago, where PCI-e ASPM did not work properly, 
with the net result originally described as "low double digit per 
cent" of packet loss.
Possibly originally reported for this particular chip... maybe 
because it was so ubiquitous: the i82579 is the external PHY chip of 
the LOM MAC integrated in the 6-series Intel south bridges (companion 
to SandyBridge CPU's). But actually the ASPM bug used to plague 
several Intel gigabit adaptor models of that era.

I can't seem to find the bugzilla entry or mailing list thread...
but I'm pretty sure someone close to the Linux kernel development 
finally nailed the bug after several months of its first report, and 
patched it in the kernel by disabling ASPM in the Intel NIC driver.
Not sure which one it was, e1000e or igb or what, and what kernel 
version.
A month or so later, this also got "fixed" in the Windows driver by 
Intel.

I surely agree that a problem waking up the PCI-e lane from shallow 
sleep doesn't sound like something that would freeze the on-chip PHC
for a fixed time quantum every time it gets asked :-) especially if 
the workaround is just to disable ASPM altogether.

While googling in vain for traces of that bug, I've found a 
*different* bug report related to the i82579:

https://bugzilla.redhat.com/show_bug.cgi?id=713315

Again... I'm not saying that this is necessarily related.
Just that the 82579 probably had its share of issues ;-)
that possibly got worked around in drivers.
So it seems that I superficially agree with Mr. Keller on that one...

(P.S.: not mentioning the "occasional cfg EEPROM invalidation issue",
which seems totally unrelated and generic across the Intel NIC 
product line.)

Don't get me wrong - I'm a great fan of Intel x86 silicon, including 
the accompanying chipsets and peripheral stuff.
Feels like home to me.
Bugs get fixed. I've seen worse elsewhere.

Frank Rysanek



___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Despite the patch, "timed out while polling for tx timestamp" keeps happening

[Frantisek Rysanek]

Dear gents,

let me just briefly follow up on the past thread on "TX timestamp 
timeouts". To sum up: on the my part the problem seems gone.

In the three months that have passed, I've built another computer
with some i210 and i82574 NIC's inside (related to my thread on
precise timestamping when capturing). I took some unsold ATX mobo 
with quad Intel NIC's and added some i210's in PCI-e slots.
I also ended up with some fiber NIC's that cannot be synced by 
external PPS, and thus I needed to sync my system timebase too, and I 
did that by PTP on one of the Intel NIC's.
This new computer already has Debian 9 Stretch, and the kernel is 
4.14.10.

On tuesday I had another chance to try PTP against the TC switch at a 
substation with IEC61850 running. The "sampled values" (not GOOSE) 
traffic is a rapid fire of 3.5 Mbps in 124B packets, i.e. about 
3.5kpps, on a 100Mbps network. The PTP is mixed in that.
And, ptp4l on the Intel NIC's worked fine.
This time I prepared the interface for PTP (and for the capturing 
too) using the following script:

ethtool --set-eee $netdev eee off
ethtool --pause $netdev rx off tx off autoneg off
# unfortunately, forced mdix off is exclusive with autoneg off :-(
#ethtool -s $netdev mdix off
ethtool -s $netdev speed 100 duplex full autoneg off
ip link set dev $netdev up

There was not a hint of any problem, neither in ptp4l (TX timestamp 
timeout) nor in t-shark doing the HW-timestamped capturing.

This is probably my last message with this subject, 
thanks for all your attention and help.

Frank

On 9 Dec 2017 at 16:26, linuxptp-devel@lists.sourcefo wrote:
> On 7 Dec 2017 at 21:55, Keller, Jacob E wrote:
> > > About ethtool stats - I now understand that you mean the output of
> > > ethtool -S, namely the lines
> > >  tx_hwtstamp_timeouts: 0
> > >  tx_hwtstamp_skipped: 0
> > >  rx_hwtstamp_cleared: 0
> > > This is what they look like now, that the error does not occur.
> > > In a few days I will probably have a chance to try it in the field
> > > again, on a PTP TC switch wih GOOSE flooding the network... that's
> > > where the misbehavior was most stubborn. Well now I know what to look
> > > at :-) I'll report more numbers when I have some.
> > > 
> > 
> > Ok good. If you see Tx timeouts again, try to measure the stats here
> > and see if any of these increment. If they do, that's a sure
> > indication that the driver was not able to obtain and send the
> > timestamp to the stack. If they *do not* increment, then that means
> > that the driver was likely too late when responding with the Tx
> > timestamp, which is a separate problem. 
> >
> Thanks for the useful information!
> 
> > Oh.. It's possible that the
> > device might be going to sleep too quickly.. can you check to see if
> > it supports EEE? "ethtool --show-eee "? This causes the device to
> > go into low power link mode which substantially increases the latency
> > for actual Tx packets (when there's little to no traffic). That might
> > be the reason under some circumstances why you see dropped timestamps,
> > if EEE is enabled? 
> >
> 
> So uh... ASPM on PCI-e may have been eliminated a couple years ago, 
> but we still have the Ethernet-level energy saving to sort out?
> I recall that even low-end unmanaged SoHo switches now support some 
> "green" functions on Gb Eth... makes me wonder if EEE is the techical 
> substance of the "green" word printed on D-Link's packaging carton.
> Wikipedia mentions 802.3az... ahh yes, and so does "man ethtool",
> referring to "eee".
> 
> # ethtool --show-eee eth0
> EEE Settings for eth0:
> EEE status: enabled - inactive
> Tx LPI: 0 (us)
> Supported EEE link modes:  100baseT/Full
>1000baseT/Full
> Advertised EEE link modes:  100baseT/Full
> 1000baseT/Full
> Link partner advertised EEE link modes:  Not reported
> 
> The Ethernet port is now connected straight to a 2nd-generation 
> Meinberg grandmaster card, called the IMS-TSU. I understand the 
> ethtool listing in the following way:
> the IMS-TSU does not support EEE.
> All the better.
> 
> A month ago, also in my lab, the eth0 was attached directly to a 
> 3rd-generation Meinberg grandmaster card, called the HPS100.
> Not sure, maybe it supports EEE? That's when the TX timeouts were 
> occurring a couple times a day, when combined with my i219LM eth0.
> 
> On site, eth0 was attached to a switch by Ruggedcom, working as a TC. 
> Makes me wonder if the switch supports EEE. Time for me to check the 
> manual. Makes me wonder if EEE combined with the GOOSE multicasts 
> could result in the high frequency of TX timestamp timeouts 
> observed...
> 
> Thanks for all the tips! :-)
> 
> Frank



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot

Re: [Linuxptp-devel] connecting two devices clock via GPIO

[Frantisek Rysanek]

Just a short update along the lines of this thread:

On top of the previous work on PPS input to i210,
I've cobbled together a simple PLL synth with a 25MHz VCXO,
good enough to take an external 10 MHz reference 
and "influence" the crystal on an i210 to run in step with the PPS.
(I could as well just desolder the original crystal, but ahh well.)
I'm attaching the resulting output of "my" i210 PPS proggie :-)
(again based heavily on Mr. Cochran's example.)

As for the capturing side of things, both TCPDUMP (PCAP file format)
and T-Shark (PCAP-NG file format) work for nanosecond-level 
capturing. I'm working on a crude analyzer of PTP traffic. 
I've given up tapping into the Wireshark dissector framework or using 
raw libpcap (and writing my own PTP dissector from scratch). 
Instead, I tend to run tshark -V and parse what I need from its 
textual output. I got through the text parsing and transaction 
recombination/inference, I have yet to write the output side (dump 
per-transaction timing data into CSV files suitable for gnuplot). 

The key bit I'm still looking forward to is the decoding of various 
timing deltas. I have a "time of capture" timestamp, and the PTP's 
own payload timestamps, and the correction value... If I understand 
correctly, the slave will tell me in PDelay Response (+FollowUp, if 
2-step) its own timestamp, which should give me a clue about *the 
slave's* performance...

I got this far using a DIY powered tap for metallic 100Mb Ethernet.
And then there's the hope that I'd be able to convert (reconfigure) 
the DeLock gigabit i210 with SFP from SERDES mode into SGMII mode...
That's coming up. The 100Mb SGMII SFP's seem quite exotic.

If you care, keep your fingers crossed :-)

Frank Rysanek
The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  i210_ext_pps.txt
 Date:  28 Feb 2018, 14:16
 Size:  5795 bytes.
 Type:  Text


i210_ext_pps.txt
Description: Binary data
The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  capture_parser.txt
 Date:  28 Feb 2018, 14:24
 Size:  2285 bytes.
 Type:  Text


capture_parser.txt
Description: Binary data
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] connecting two devices clock via GPIO

[Frantisek Rysanek]

...due to problems with direct attachments in this list, please 
download the attachments mentioned in message text from here:
http://support.fccps.cz/download/adv/frr/ptp/i210_ext_pps.zip

On 13 Feb 2018 at 17:11, frantisek.rysa...@post.cz wrote:
>
> Dear everyone (maybe Mr. Cochran especially),
> 
> my setup with external PPS has started showing basic signs of life.
> 
[...]
> I'm attaching a crude beta of my servo proggie, based on Mr. 
> Cochran's example.
> 

For the record, I've implemented a simple software-side filter
on the ext_PPS events, so that only leading edges are considered
by the servo.
And I've re-written the program a bit, but it's mostly convenience
addons (getopt parsing, comments...) - the key parts by Mr. Cochran
are still firmly in place.

I'm attaching my "release candidate", along with a manpage.
As the program depends on existing parts of the linuxptp project,
I've also added a Makefile patch that adds my i210_ext_pps 
to the build process.

Attached you'll also find a minimal pinout of the common Intel/HP/etc 
i210 PCI-e nic (board) - note: the input is not 5V tolerant, 3.3V 
only! ...and an example output (trace, log) from the proggie.
I cannot stop grinning... within 10 ns with the basic uncompensated 
Xtals. Unbelievable.

Unlike Mr. Cochran's original that used the first NIC as a PPS 
source, my cut of the program depends on an external PPS source,
all the NIC's are PPS consumers. 
The program needs to run all the time, as it propels the servo loops
that keep the NIC PHC's aligned to external PPS (this is not an 
autonomous function of the i210 hardware).

I hope it helps someone.

Frank Rysanek


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] connecting two devices clock via GPIO

[Frantisek Rysanek]

Just for the record:

On 13 Feb 2018 at 8:05, Richard Cochran wrote:
>
> The reason for this is that that the i210 latches the time on both
> rising and falling edges of the input signal.  You can't program it
> for rising edge only, for example.
>  
I've checked the datasheet again and you're right,
no way to set the polarity of the SDP pins in general,
and no way to trigger an auxiliary timestamp on
a rising or falling edge only. Ahh well.

There are several specific functions of the chip where you can 
configure the active polarity, but this is not the case with the
SDP I/O in general.

So I'm gonna have to do this in software. 
Well at least I can make it polarity-agnostic :-)

Frank

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] connecting two devices clock via GPIO

[Frantisek Rysanek]

Dear everyone (maybe Mr. Cochran especially),

my setup with external PPS has started showing basic signs of life.

Interestingly, my two i210 cards seem to be throwing an event on both
PPS edges, rising and falling :-) They're spaced 200 ms apart.
And, somehow the system converges to the "wrong" edge,
which probably is not a problem, and which I could readily correct by 
flipping polarity at the Meinberg clock, but for other reasons (a 
25MHz synth that I'm working on) I would prefer to reconfigure the 
i210 accordingly.
I seem to recall notes from the i210 chip datasheet that the polarity 
can be adjusted. If you can give me a shortcut on how to do this in 
the code of your example proggie, it would save me some more time...
Maybe I should just check the tools from kernel.org that you've 
mentioned in your other message. Or back to devmem2.

I'm attaching a crude beta of my servo proggie, based on Mr. 
Cochran's example. See its output below.

BTW: Yippeee!!  :-D

Frank Rysanek

i210_ext_pps[1477.035]: ptp4 offset -3 s2 freq  -22542
i210_ext_pps[1477.835]: ptp1 offset -200015399 s2 freq -10
i210_ext_pps[1477.835]: ptp4 offset -200012384 s2 freq -10
i210_ext_pps[1478.035]: ptp1 offset -3 s2 freq   -3720
i210_ext_pps[1478.035]: ptp4 offset  5 s2 freq  -22534
i210_ext_pps[1478.835]: ptp1 offset -200015391 s2 freq -10
i210_ext_pps[1478.835]: ptp4 offset -200012381 s2 freq -10
i210_ext_pps[1479.035]: ptp1 offset -4 s2 freq   -3722
i210_ext_pps[1479.035]: ptp4 offset -1 s2 freq  -22539
i210_ext_pps[1479.835]: ptp1 offset -200015390 s2 freq -10
i210_ext_pps[1479.835]: ptp4 offset -200012384 s2 freq -10
i210_ext_pps[1480.035]: ptp1 offset -2 s2 freq   -3721
i210_ext_pps[1480.035]: ptp4 offset -3 s2 freq  -22541
i210_ext_pps[1480.835]: ptp1 offset -200015389 s2 freq -10
i210_ext_pps[1480.835]: ptp4 offset -200012384 s2 freq -10
i210_ext_pps[1481.035]: ptp1 offset  6 s2 freq   -3714
i210_ext_pps[1481.035]: ptp4 offset  5 s2 freq  -22534
i210_ext_pps[1481.835]: ptp1 offset -200015394 s2 freq -10
i210_ext_pps[1481.835]: ptp4 offset -200012382 s2 freq -10
i210_ext_pps[1482.035]: ptp1 offset  1 s2 freq   -3717
i210_ext_pps[1482.035]: ptp4 offset -1 s2 freq  -22539
i210_ext_pps[1482.835]: ptp1 offset -200015397 s2 freq -10
i210_ext_pps[1482.835]: ptp4 offset -200012384 s2 freq -10
i210_ext_pps[1483.035]: ptp1 offset -2 s2 freq   -3720
i210_ext_pps[1483.035]: ptp4 offset -3 s2 freq  -22541
i210_ext_pps[1483.835]: ptp1 offset -200015398 s2 freq -10
i210_ext_pps[1483.835]: ptp4 offset -200012385 s2 freq -10
i210_ext_pps[1484.035]: ptp1 offset -2 s2 freq   -3720
i210_ext_pps[1484.035]: ptp4 offset -4 s2 freq  -22543
i210_ext_pps[1484.835]: ptp1 offset -200015390 s2 freq -10
i210_ext_pps[1484.835]: ptp4 offset -200012384 s2 freq -10
i210_ext_pps[1485.035]: ptp1 offset -3 s2 freq   -3722
i210_ext_pps[1485.035]: ptp4 offset  5 s2 freq  -22535
i210_ext_pps[1485.835]: ptp1 offset -200015397 s2 freq -10
i210_ext_pps[1485.835]: ptp4 offset -200012381 s2 freq -10
i210_ext_pps[1486.035]: ptp1 offset -2 s2 freq   -3722
i210_ext_pps[1486.035]: ptp4 offset  0 s2 freq  -22539
i210_ext_pps[1486.835]: ptp1 offset -200015396 s2 freq -10
i210_ext_pps[1486.835]: ptp4 offset -200012384 s2 freq -10
i210_ext_pps[1487.035]: ptp1 offset -1 s2 freq   -3721
i210_ext_pps[1487.035]: ptp4 offset -3 s2 freq  -22542


On 13 Feb 2018 at 11:45, linuxptp-devel@lists.sourcefo wrote:
>
> Dear Mr. Cochran,
> 
> I've finally got back to my plan (ext.PPS to 2-4 i210 chips for 
> tcpdump timestamping)  and I'm reviewing your proggie. 
> Yes the code is pretty self-explanatory :-) and packed with gems.
> 
> If I understand correctly, setting up the SDP0 GPIO for external
> PPS input (which takes two function calls in your program)
> results in the respective phc throwing "events" on every PPS 
> leading edge - passing them to the user space proggie that 
> keeps waiting in a poll(), and keeps processing the events
> thus received. The event essentially contains a precise timestamp (as 
> per the i210's timebase) of the PPS leading edge.
> And, the "servo" is not autonomous (in hardware or in kernel), it is 
> in fact implemented by the synbc program, which has to explicitly 
> instruct the PHC to adjust stepwise or frequency-wise accordingly...
> the required "ppb" value actually comes from the servo->sample() 
> PTP4l library function...
> 
> Interestingly to me, your program seems to instruct the "slave PHC"
> (PPS master) to send the pulses every 2 seconds, rather than every 1 
> second... am I reading this correctly?
> 
> int index = 0, perout = 20;
> 
> I understand that the PHC's work in wall time,
> and I should be able to prime them with the system time, 
> once on program startup (as I won't have a 

Re: [Linuxptp-devel] connecting two devices clock via GPIO

[Frantisek Rysanek]

Ohh... excellent, that answers a question from my next message :-)

In fact I don't mind if I see the falling edges as well - only I 
should detect them somehow, by timing calculation if there's no other 
way, and ignore them in the servo policing loop.

As my application is tcpdump timestamping (rather than serving ptp),
I'd prefer to use UTC for the timestamps. Oh wait. I'll be 
timestamping PTP traffic. So I'd better use TAI to have a common
timescale inside the packets captured and in the timestamps :-)
Allright, I'll probably just subtract a few seconds when setting the 
PHC's.

Frank

On 13 Feb 2018 at 8:05, Richard Cochran wrote:

> On Tue, Feb 13, 2018 at 11:45:37AM +0100, Frantisek Rysanek wrote:
> > Interestingly to me, your program seems to instruct the "slave PHC"
> > (PPS master) to send the pulses every 2 seconds, rather than every 1 
> > second... am I reading this correctly?
> > 
> > int index = 0, perout = 20;
> 
> The reason for this is that that the i210 latches the time on both
> rising and falling edges of the input signal.  You can't program it
> for rising edge only, for example.
>  
> > I understand that the PHC's work in wall time,
> > and I should be able to prime them with the system time, 
> > once on program startup (as I won't have a PPS master / PTP slave PHC 
> > in my system). It's just a matter of asking clock_gettime() of the 
> > system timebase, rather than a particular PHC, in your function 
> > synbc_settime().
> 
> Normally the PHC time scale is TAI, which is offset from the system's
> UTC by an integer number of seconds.  You *can* use whatever time
> scale you want, but PTP nodes will expect TAI by default.
> 
> HTH,
> Richard



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] connecting two devices clock via GPIO

[Frantisek Rysanek]

Dear Mr. Cochran,

I've finally got back to my plan (ext.PPS to 2-4 i210 chips for 
tcpdump timestamping)  and I'm reviewing your proggie. 
Yes the code is pretty self-explanatory :-) and packed with gems.

If I understand correctly, setting up the SDP0 GPIO for external
PPS input (which takes two function calls in your program)
results in the respective phc throwing "events" on every PPS 
leading edge - passing them to the user space proggie that 
keeps waiting in a poll(), and keeps processing the events
thus received. The event essentially contains a precise timestamp (as 
per the i210's timebase) of the PPS leading edge.
And, the "servo" is not autonomous (in hardware or in kernel), it is 
in fact implemented by the synbc program, which has to explicitly 
instruct the PHC to adjust stepwise or frequency-wise accordingly...
the required "ppb" value actually comes from the servo->sample() 
PTP4l library function...

Interestingly to me, your program seems to instruct the "slave PHC"
(PPS master) to send the pulses every 2 seconds, rather than every 1 
second... am I reading this correctly?

int index = 0, perout = 20;

I understand that the PHC's work in wall time,
and I should be able to prime them with the system time, 
once on program startup (as I won't have a PPS master / PTP slave PHC 
in my system). It's just a matter of asking clock_gettime() of the 
system timebase, rather than a particular PHC, in your function 
synbc_settime().

Interesting stuff. Thank you :-)

Frank Rysanek


On 8 Dec 2017 at 15:59, Richard Cochran wrote:
>
> On Fri, Dec 08, 2017 at 11:09:40PM +, Keller, Jacob E wrote:
> > I'm thinking the best way is to use the external timestamp events setup, 
> > and then plug that in as the pps source into phc2sys?
> > 
> > Does this make any sense, or am I paddling up the wrong creek?
> 
> So you *could* extend phc2sys, but that program is complex enough as
> is.  I have made thoughts about a successor to phc2sys that would
> handle gpio based measurements, including setting the pin functions
> using the PHC ioctls.
> 
> But for now, I would just write a simple program for your specific
> setup.  Below is an example for using three i210 cards whose first SDP
> are connected.  The first card is hard coded as the PPS producer.  In
> a more realistic JBOD setting, you would want to switch the PPS
> producer to be the PHC of the port that takes on the SLAVE role.
> 
> HTH,
> Richard
> 
> --->8---
> /**
>  * @file synbc.c
>  * @brief Synchronize the ports of a JBOD Boundary Clock.
>  * @note Copyright (C) 2015 Richard Cochran 
>  *
>  * This program is free software; you can redistribute it and/or modify
>  * it under the terms of the GNU General Public License as published by
>  * the Free Software Foundation; either version 2 of the License, or
>  * (at your option) any later version.
>  *
>  * This program is distributed in the hope that it will be useful,
>  * but WITHOUT ANY WARRANTY; without even the implied warranty of
>  * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
>  * GNU General Public License for more details.
>  *
>  * You should have received a copy of the GNU General Public License along
>  * with this program; if not, write to the Free Software Foundation, Inc.,
>  * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
>  */
> #include 
> #include 
> #include 
> #include 
> #include 
> 
> #include 
> 
> #include "clockadj.h"
> #include "config.h"
> #include "phc.h"
> #include "print.h"
> #include "servo.h"
> #include "util.h"
> 
> #define N_PHC 3
> 
> static int synbc_extts(int fd, int enable)
> {
>   struct ptp_extts_request extts_request;
>   int index = 0;
>   memset(_request, 0, sizeof(extts_request));
>   extts_request.index = index;
>   extts_request.flags = enable ? PTP_ENABLE_FEATURE : 0;
>   if (ioctl(fd, PTP_EXTTS_REQUEST, _request)) {
>   pr_err("PTP_EXTTS_REQUEST");
>   return -1;
>   }
>   pr_info("external time stamp request okay");
>   return 0;
> }
> 
> static int synbc_pin_input(int fd)
> {
>   struct ptp_pin_desc desc;
>   int index = 0, pin_index = 0;
> 
>   memset(, 0, sizeof(desc));
>   desc.index = pin_index;
>   desc.func = PTP_PF_EXTTS;
>   desc.chan = index;
>   if (ioctl(fd, PTP_PIN_SETFUNC, )) {
>   pr_err("PTP_PIN_SETFUNC");
>   return -1;
>   }
>   pr_info("set pin function okay");
>   return 0;
> }
> 
> static int synbc_pin_output(int fd)
> {
>   struct ptp_pin_desc desc;
>   int index = 0, pin_index = 0;
> 
>   memset(, 0, sizeof(desc));
>   desc.index = pin_index;
>   desc.func = PTP_PF_PEROUT;
>   desc.chan = index;
>   if (ioctl(fd, PTP_PIN_SETFUNC, )) {
>   pr_err("PTP_PIN_SETFUNC");
>   return -1;
>   }
>   pr_info("set pin function okay");
>   return 0;
> }
> 
> static int 

Re: [Linuxptp-devel] Leap seconds in Linux

[Frantisek Rysanek]

Hello Petr,

AFAICT, the UNIX "epoch time" is per definition oblivious of leap 
seconds = in UNIX time, leap seconds "do not exist, and after a leap 
second actually happens, to a UNIX machine it never has happend" :-)
This approach is probably standardized in POSIX.
http://www.madore.org/~david/computers/unix-leap-seconds.html

=> the Linux kernel actually seems to an official API 
which can be used to step the time on a leap second.
It consists of two attributes within the arguments of adjtimex(),
called STA_INS and STA_DEL.
https://lwn.net/Articles/648313/
http://man7.org/linux/man-pages/man2/adjtimex.2.html

>From a recent trouble issue with an outdated ntpd release (in Linux),
when I was watching leap second transitions on peerstats and 
loopstats graphs, it seems that even ntpd (who does know about leap 
seconds for sure) has to adjust the Linux system timebase either 
gradually or possibly stepwise to "correct for" the leap second that 
has actually just occurred.
https://access.redhat.com/articles/15145
https://developers.redhat.com/blog/2015/06/01/five-different-ways-hand
le-leap-seconds-ntp/

=> the traditional wisdom that "UNIX system timebase runs in UTC"
has an exception: the leap seconds are not systematically accounted 
for. Leap seconds get skipped or smeared, and finally: ignored.

If you want a timestamp that does *not* include ignored leap seconds,
you would probably have to keep an eye on the official
leap seconds list
https://www.ietf.org/timezones/data/leap-seconds.list
or poll the local ntpd instance via ntpdc 
or keep an eye on the flags in adjtimex (in the kernel)
and add the leap seconds on your own.
And, if your system is configured to handle leap seconds by 
smearing/slewing, you would have to know that your timestamps are 
fuzzy for some minutes or hours following the actual leap.

PTP's native time scale is the "TAI": an international standard 
counting SI seconds. TAI does not add or subtract leap seconds, nor 
does it accommodate them in any way, it just runs forever with 60 
seconds per minute precisely - this works well for various machine 
synchronization purposes where unambiguous and continuous time is 
vital.
TAI has a certain offset against the UTC, and this is transported in 
PTP as an "additional attribute", called simply the UTC offset.
It's up to the PTP slave/client to decide how to process that 
information internally: the PTP timestamp + the UTC offset 
+ possibly the daylight saving offset.
Note that the TAI time scale is similar to the GPS time scale
in this respect (continuity), but these two (GPS and TAI)
have a different "UTC offset".
https://www.google.cz/search?q=GPS+UTC+TAI

=> If you have access to a PTP GM, your best bet might be
to use a NIC with PTP support in HW (= one containing a PHC)
and ask the NIC hardware for accurate timestamps :-)
independent of the UNIX epoch timebase in the host OS.

NTP does carry leap second announcement ahead of the actual leap.
Any serious "atomic clock" serial time string does carry that 
information as well (I'm familiar with the Meinberg Standard Time 
String).
The GPS satellite segment does transmit a current GPS-to-UTC offset 
information *and* an announcement ahead of the actual leap.
https://www.meinbergglobal.com/english/info/leap-second.htm
I haven't found any clues if any specific announcement flag or 
something is available in the NMEA data format.

Frank


On 15 Jan 2018 at 11:17, Petr Kulhavy wrote:
>
> Hi,
> 
> my question is not directly related to linuxptp, but the expertise here 
> might be the right one to answer it.
> 
> Is there a way to retrieve a UT1 timestamp on Linux? It seems all the 
> library functions like gettimeofday(), clock_gettime() etc. return the 
> Unix time without leap seconds.
> And the library functions that do handle leap seconds always work with 
> the broken-down time (e.g. localtime(), gmtime())
> 
> Does PTP transmit the leap second information?
> 
> Thanks
> Petr
> 
> --
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> ___
> Linuxptp-devel mailing list
> Linuxptp-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linuxptp-devel



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] i210 doesn't want to eavesdrop on RX only ? (apologies for OT)

[Frantisek Rysanek]

On 24 Dec 2017 at 8:34, linuxptp-devel@lists.sourcefo wrote:
> On 24 Dec 2017 at 1:23, linuxptp-devel@lists.sourcefo wrote:
> > I've tried forcing 100 Mbit full duplex in the eavesdropping i210 and 
> > 
> > generally super-simple behavior:
> > 
> > ethtool -s eth2 speed 100 duplex full autoneg off
> > ethtool --set-eee eth0 eee off
> > ethtool -A eth0 rx off rx off autoneg off
> > ethtool -s eth2 mdix off   (igb driver refuses this for full/100)
> > ...and obviously ifconfig up.
> > 
> > I've noticed that my distro's ethtool was a little stale (3.16)
> > so I compiled 4.13 from source (the kernel is 4.13.12).
> > Now I possibly get fewer errors from ethtool that "this combination 
> > of arguments is illegal", but still no go.
> > 
> > I've tried mii-tool, but the net-tools package was last updated in 
> > 2010 or so, and the SIOCSMIIREG is apparently unsupported
> > in e1000e and igb, only SIOCGMIIREG - so the ancient mii-tool
> > is not much use, maybe as a check what ethtool has actually done
> > (and not a very detailed check at that).
> > 
>
> Hmm... maybe I should also take another look at 
> common mode noise / signal grounding. Just in case.
> 
...and the outcome is: yes the problem was in reference ground noise.

Not sure if the mains transformer leaks a bit of something,
or what the hell, but the truth is that I've seen before on a 'scope, 
that the tap output waveforms contained some common mode 
low-frequency garbage if I left the internal common ground float 
free. The "signal pair biasing" RL networks cannot take care of all 
the noise, I'd have to make the R too small (and I feel uncomfortable 
about that).
Fortunately, simply connecting the tap's signal ground to the chassis 
of my probe computer seems to stabilize the ground enough that the 
buffered waveforms seem perfectly clear.

And exactly that grounding link has helped me get the eavesdropping 
setup to work = after I linked the tap's internal GND to the nearby
probe computer's chassis, the listening i210's link went up 
and tcpdump showed a waterfall of packets flying by.

I actually got stuck (plugged the tap in and there was no link) just 
before calling it a day at work on Friday before X-mas :-)
So I spent some christmas evenings studying the i210 datasheet, and 
trying things on the live animal. 
I've noticed the "energy detection" status bits in the i210 PHY and 
in the PHPM register (bar 0 reg 0x0E14). I found an easy way to peek 
and poke the IOMEM registers in bar 0, using the devmem2 utility. 
I was able to use that to poke things in the PHY via MII using the 
MDIC register (offset 0x0020). Peeks on the PHY are not feasible this 
way (via the MDIC), because a read attempt takes two transactions and 
something in the driver is always faster than I to slurp my result 
:-) but fortunately the mii-diag can do the PHY reads in a clean way.

I tried stuff such as 

   devmem2 0xdf100020 w 0x04105f70

...to force the link up via MII PHYREG 16 (0x10) - did bring the link 
indication up, but no data was coming through (obviously, as I now 
know, as the payload was total garbage)

   mii-diag -v eth2

...to see the effects of my configuration attempts in the PHY 
registers.

I noticed that the "energy detection" bit stays up even if I unplug 
the RJ45 patch cord, so it's probably somewhat bogus anyway :-)

And then I got back to work, did a few quick tests in the wiring,
tried getting a forced 100Mbit link to come up with one way going
through the tap instead of straight through, and then I tried the 
grounding... from then on, it's a problem solved.

Much more work down the road to turn this into something useful
in the context of my PTP trouble case.

Frank


--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] i210 doesn't want to eavesdrop on RX only ? (apologies for OT)

[Frantisek Rysanek]

Hmm... maybe I should also take another look at 
common mode noise / signal grounding. Just in case.
Frank

On 24 Dec 2017 at 1:23, linuxptp-devel@lists.sourcefo wrote:

> (
> once again my attachments were over the size limit, see them here:
> http://support.fccps.cz/download/adv/frr/tap/tap-schematic.pdf
> http://support.fccps.cz/download/adv/frr/tap/tap-board.pdf
> )
> 
> --
> Dear gentlemen,
> 
> apologies for this off-topic question.
> I should probably ask this in lkml or some specific Linux list at the 
> 
> vger, or electronics.stackexchange... yet, I've noticed some relevant 
> 
> souls in this list, maybe someone will know...
> 
> In the context of my previously mentioned idea, to sync multiple i210 
> 
> chips via external PPS for precise packet capturing/timestamping, 
> I've built a prototype pseudo-passive Ethernet tap for 100Base-TX.
> Schematic and board layout are attached. 
> The unrouted traces for power supply rails are implemented by wire 
> bridges (and a section of symmetric 100-Ohms cable for the missing 
> signal interconnect).
> The "straight through" direction is as free of stubs / impedance 
> impurities as possible, and the "tap outputs" are buffered by analog 
> amps (electric repeaters, not data buffers) to reconstruct the 
> correct voltage. I've paid meticulous attention to impedance matching 
> 
> and series termination. The op-amps used do have the needed 
> bandwidth. I can see beautiful waveforms on the output (when 
> loaded=terminated by 100 Ohms at the input of my oscilloscope).
> 
> An urban legend would have it, that Ethernet NIC's readily accept 
> this sort of tap output, even from a simple wired splitter that's 
> impedance-mismatched = suffers from lower voltage and reflections.
> I had my doubt, but wanted to try, and... it doesn't seem to work 
> against an i210.
> 
> The two peers in the straight-through direction connect just fine,
> I can see nice output on the tap-out jacks with a 'scope,
> but if I attach an i210 by a straight patch-cord, its link remains 
> down.
> 
> I've tried forcing 100 Mbit full duplex in the eavesdropping i210 and 
> 
> generally super-simple behavior:
> 
> ethtool -s eth2 speed 100 duplex full autoneg off
> ethtool --set-eee eth0 eee off
> ethtool -A eth0 rx off rx off autoneg off
> ethtool -s eth2 mdix off   (igb driver refuses this for full/100)
> ...and obviously ifconfig up.
> 
> I've noticed that my distro's ethtool was a little stale (3.16)
> so I compiled 4.13 from source (the kernel is 4.13.12).
> Now I possibly get fewer errors from ethtool that "this combination 
> of arguments is illegal", but still no go.
> 
> I've tried mii-tool, but the net-tools package was last updated in 
> 2010 or so, and the SIOCSMIIREG is apparently unsupported
> in e1000e and igb, only SIOCGMIIREG - so the ancient mii-tool
> is not much use, maybe as a check what ethtool has actually done
> (and not a very detailed check at that).
> 
> The tap outputs are attached to pins 3 and 6 in the RJ45 (the green 
> pair).
> 
> I haven't tried yet to load the orange pair from the eavesdropping 
> NIC. Makes me wonder if the NIC could wait for a 100 Ohms load
> on the TX pair. Not very likely to me...
> 
> I actually have 2 pcs of the i210 currently in the bench system: one 
> as a PTP slave, one for eavesdropping.
> I'm wondering if perhaps the i210 actually depends on the autoneg 
> handshake for "link up", in spite of it being disabled via ethtool 
> for speed and duplex negotiation. 
> The autoneg handshake is two-way, in principle. Needs both directions 
> 
> to work between the handshaking PHY peers :-( I suspect that this is 
> also an explanation why the i210 won't link against the Meinberg GM 
> if I configure both ends for 100/full without autoneg. Maybe the i210 
> 
> still awaits autoneg and won't link. If I configure both ends for 
> autoneg, and just tell the i210 to advertise 100/full, the handshake 
> happens and the link works perfectly. The autoneg handshake appears 
> to be responsible for several "smart" features, such as the eee - 
> i.e., it is no longer just a matter of speed+duplex :-/
> 
> Any ideas welcome... is there something I could tweak in the driver 
> maybe, to make "autoneg off" actually remove any dependency on a 
> bilateral handshake to bring the link up?
> 
> Frank Rysanek
> 
> 
> Attachments:
>   C:\Users\frr\AppData\Local\Temp\WPM$BQ1R.PM$



--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

[Linuxptp-devel] i210 doesn't want to eavesdrop on RX only ? (apologies for OT)

[Frantisek Rysanek]

(
once again my attachments were over the size limit, see them here:
http://support.fccps.cz/download/adv/frr/tap/tap-schematic.pdf
http://support.fccps.cz/download/adv/frr/tap/tap-board.pdf
)

--
Dear gentlemen,

apologies for this off-topic question.
I should probably ask this in lkml or some specific Linux list at the 

vger, or electronics.stackexchange... yet, I've noticed some relevant 

souls in this list, maybe someone will know...

In the context of my previously mentioned idea, to sync multiple i210 

chips via external PPS for precise packet capturing/timestamping, 
I've built a prototype pseudo-passive Ethernet tap for 100Base-TX.
Schematic and board layout are attached. 
The unrouted traces for power supply rails are implemented by wire 
bridges (and a section of symmetric 100-Ohms cable for the missing 
signal interconnect).
The "straight through" direction is as free of stubs / impedance 
impurities as possible, and the "tap outputs" are buffered by analog 
amps (electric repeaters, not data buffers) to reconstruct the 
correct voltage. I've paid meticulous attention to impedance matching 

and series termination. The op-amps used do have the needed 
bandwidth. I can see beautiful waveforms on the output (when 
loaded=terminated by 100 Ohms at the input of my oscilloscope).

An urban legend would have it, that Ethernet NIC's readily accept 
this sort of tap output, even from a simple wired splitter that's 
impedance-mismatched = suffers from lower voltage and reflections.
I had my doubt, but wanted to try, and... it doesn't seem to work 
against an i210.

The two peers in the straight-through direction connect just fine,
I can see nice output on the tap-out jacks with a 'scope,
but if I attach an i210 by a straight patch-cord, its link remains 
down.

I've tried forcing 100 Mbit full duplex in the eavesdropping i210 and 

generally super-simple behavior:

ethtool -s eth2 speed 100 duplex full autoneg off
ethtool --set-eee eth0 eee off
ethtool -A eth0 rx off rx off autoneg off
ethtool -s eth2 mdix off   (igb driver refuses this for full/100)
...and obviously ifconfig up.

I've noticed that my distro's ethtool was a little stale (3.16)
so I compiled 4.13 from source (the kernel is 4.13.12).
Now I possibly get fewer errors from ethtool that "this combination 
of arguments is illegal", but still no go.

I've tried mii-tool, but the net-tools package was last updated in 
2010 or so, and the SIOCSMIIREG is apparently unsupported
in e1000e and igb, only SIOCGMIIREG - so the ancient mii-tool
is not much use, maybe as a check what ethtool has actually done
(and not a very detailed check at that).

The tap outputs are attached to pins 3 and 6 in the RJ45 (the green 
pair).

I haven't tried yet to load the orange pair from the eavesdropping 
NIC. Makes me wonder if the NIC could wait for a 100 Ohms load
on the TX pair. Not very likely to me...

I actually have 2 pcs of the i210 currently in the bench system: one 
as a PTP slave, one for eavesdropping.
I'm wondering if perhaps the i210 actually depends on the autoneg 
handshake for "link up", in spite of it being disabled via ethtool 
for speed and duplex negotiation. 
The autoneg handshake is two-way, in principle. Needs both directions 

to work between the handshaking PHY peers :-( I suspect that this is 
also an explanation why the i210 won't link against the Meinberg GM 
if I configure both ends for 100/full without autoneg. Maybe the i210 

still awaits autoneg and won't link. If I configure both ends for 
autoneg, and just tell the i210 to advertise 100/full, the handshake 
happens and the link works perfectly. The autoneg handshake appears 
to be responsible for several "smart" features, such as the eee - 
i.e., it is no longer just a matter of speed+duplex :-/

Any ideas welcome... is there something I could tweak in the driver 
maybe, to make "autoneg off" actually remove any dependency on a 
bilateral handshake to bring the link up?

Frank Rysanek



WPM$BQ1R.PM$
Description: Mail message body
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] connecting two devices clock via GPIO

[Frantisek Rysanek]

Oh... and this one also went to Mr. Cochran directly. Apologies.
I already got an answer from him and I'm past this stage,
but I'm forwarding this into the mailing-list "for the record",
to give some food to the Google spider.

FR

On 8 Dec 2017 at 15:59, Richard Cochran wrote:
> On Fri, Dec 08, 2017 at 11:09:40PM +, Keller, Jacob E wrote:
> > I'm thinking the best way is to use the external timestamp events setup, 
> > and then plug that in as the pps source into phc2sys?
> > 
> > Does this make any sense, or am I paddling up the wrong creek?
> 
> So you *could* extend phc2sys, but that program is complex enough as
> is.  I have made thoughts about a successor to phc2sys that would
> handle gpio based measurements, including setting the pin functions
> using the PHC ioctls.
> 
> But for now, I would just write a simple program for your specific
> setup.  Below is an example for using three i210 cards whose first SDP
> are connected.  The first card is hard coded as the PPS producer.  In
> a more realistic JBOD setting, you would want to switch the PPS
> producer to be the PHC of the port that takes on the SLAVE role.
> 
> HTH,
> Richard

Apologies for the intrusion gentlemen...
I'm just an end user passing by, but this thread coincides with a 
related topic that's currently on my mind :-)

I've been wondering for a few days, if I could use Intel NIC hardware 

to capture misc network traffic (libpcap style), with hardware 
timestamping of incoming packets, at nanosecond resolution.
Timestamps on any packets captured, not just PTP
- such as, to implement the capturing back-end of a poor man's 
precision network traffic analyzer.

There are several question marks along the way.
In the following text, note that I'll answer some questions myselfs,
as I'm studying and experimenting (with freshmost upstream GIT code).

To me, the most unclear parts are especially the "general 
timestamping" bits. From the user space, I've noticed the 
SO_TIMESTAMPNS and SOF_TIMESTAMPING_RX_HARDWARE = some flags 
available from the Linux kernel. They appear to be "mutually 
exclusive" ? But the latter should be sufficient for nanosec-level 
timestamping? What is the difference between
SOF_TIMESTAMPING_RX_HARDWARE and SOF_TIMESTAMPING_RAW_HARDWARE ?

https://stackoverflow.com/questions/41805687/linux-kernel-udp-receptio
n-timestamp

Am i right to assume, that the Intel NIC's can provide RX timestamps 
to any packets received, rather than just PTP exclusively?
And, is this capability reachable via the networking driver's 
in-kernel API?

Another point is how to actually capture the data, from user space, 
preferably using tools that are ready.
Use libpcap? 
Are there any other libraries in Linux along those lines?
Or, should I roll my own capture library?
I'm asking this with respect to nanosecond-level resolution.

There appears to be a common wisdom, permeating the interwebs, that 
tcpdump and libpcap do not support nanoseconds resolution, that they 
stick to microseconds.
At a closer look, I have to say that it definitely is true about the 
PCAP file format - that alone appears to be a non-issue. 

PCAP-NG supports fairly arbitrary resolution, with "microseconds" and 
"nanoseconds" being popular choices that get actually implemented in 
libraries available to manipulate those file formats.
Wireshark/tshark has nanoseconds support in PCAP-NG files mentioned 
in the docs for version 2.5.0, but I can actually see nanoseconds 
resolution in the textual output of tshark 2.4.1. in Linux, precisely 
"tshark -i eth1" with no custom options... I'm wondering how to find 
out if tshark uses the HW timestamping capabilities of the kernel and 
hardware. 
Interestingly, a nightly build of Wireshark and TShark 2.5.0 still 
shows microseconds on Windows 64bit (capturing from a local Ethernet 
interface) and I haven't found any configuration options to switch it 
to nanoseconds...

TCPdump the user-space app doesn't seem to support PCAP-NG, 
except for some options specific to MacOS X...

LibPCAP the capture library DOES seem to support nanoseconds 
precision! In the changelog of libpcap source code (currently at 
1.8/1.9 release), I've found notes that support for nanoseconds was 
added in 1.5.0 back in 2013 or so... A good keyword to grep for 
appears to be   PCAP_TSTAMP_PRECISION_NANO, and grep also finds 
references to SOF_TIMESTAMPING_SYS_HARDWARE and 
SOF_TIMESTAMPING_RAW_HARDWARE in the libpcap source code 
(pcap-linux.c).
Support for pcap-ng seems to be in the current libpcap source code 
too. 
Heheh - when I downloaded and compiled fresh libpcap and tcpdump from 
GIT, the following prints nanosecond timestamps: 

  tcpdump -i eth1 --time-stamp-precision=nano

# tcpdump --version
tcpdump version 4.10.0-PRE-GIT
libpcap version 1.9.0-PRE-GIT (with TPACKET_V3)

Only I can't seem to save the data in PCAP-NG format, as tcpdump 
still doesn't seem to support that... :-(  The tcpdump manual 
mentions some option to save a modified PCAP 

Re: [Linuxptp-devel] connecting two devices clock via GPIO

[Frantisek Rysanek]

...oops... ...forgot to post this to the mailing list...

On 10 Dec 2017 at 19:20, Richard Cochran wrote:
> Just use
> 
> tcpdump \
>   -j adapter_unsynced \   
>   
>   
>   
>  
>   --time-stamp-precision=nano \   
>   
>   
>   
>  
>   ... other options
> 
I've read 'man pcap-tstamp' to see what -j adapter_unsynced means.
"High-precision timestamp, not synchronized to the operating system's 

clock". 
If I provide an UTC-synchronous PPS externally (from a GPS receiver), 

to maybe 4 ports of i210, how do I reference that precise timestamp 
to UTC "time of day" when analyzing a resulting PCAP file ? By the 
time of last modification of the file? Or do the four separate PHC's 
get synchronized to UTC somehow automagically?

The PCAP file normally seems to have an initial absolute timestamp in 

the header, consequently Wireshark can show absolute UTC-referenced 
timestamps per packet if asked to... Will it work like that with 
adapter_unsynced ?

> TL;DR the rest...
>
TL, admittedly, yes :-)
 
> Just get the i210 adapter.  The newer ones I've seen already have the
> header.
>
That's my conclusion too.

I'm still wondering if all this is useable for *fiber* tapping / 
capturing. When speaking about the i210, so far it was in the 
context of 1000Base-TX (or 10/100/1000) i.e. the built-in PHY.
The i210 has another SKU that can directly drive a dumb
SERDES transceiver (at 1Gb rate) or can talk to an intelligent 
external PHY via SGMII. Hence my uncertainty:

Is the HW timestamping support dependent on the internal metallic 
PHY, or does it work with an external SGMII-based fiber PHY as well?

I've actually found an off-the-shelf NIC with the i210 and an SFP 
slot:
 http://www.delock.de/produkte/G_89481/merkmale.html?setLanguage=en
For my "application", I've noticed that there are 100Base-FX SFP 
transceivers with SGMII interface which would make it feasible to 
grab 100 Mbit fiber optic traffic using an i210.

And, apparently, chances are that they are compatible with the i210:
https://embedded.communities.intel.com/thread/8856

> > So... looking at the proggie from Mr. Cochran,
> > to configure the PHC in a NIC chip to be a PPS slave,
> > using a particular SDP pin as an input, I need to open its respective 
> > /dev/phcX and run some fine-tipped ioctl()s on the open fd.
> 
> Just use the program I posted.
> 
> Or use the 'testptp' program from the Linux kernel.  It can configure
> the pins via command line.
>
coool. Thanks for those pointers :-)
 
Frank Rysanek




WPM$SQDV.PM$
Description: Mail message body
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Despite the patch, "timed out while polling for tx timestamp" keeps happening

[Frantisek Rysanek]

Dear everyone (Mr. Keller in particular),

in the process of fiddling with an addon i210-T1 received today 
(original Intel board SKU) I have discovered minor havoc in my 
previously posted data.

Over the past few days/weeks I was delighted at how marvellous the 
onboard i219LM was, while in fact, it turns out that I've been using 
the second onboard NIC, the i210, all along for PTP. 
All the praise goes to the i210. The i219LM is inferior in my PC.
I'm re-attaching some samples from a running ptp4l.
The files also contain a corresponding dump of ethtool -T .

I was originally orienting myself by the contents of "dmesg". By the 
eth0 vs. eth1 originally reported upon device detection. 
Only today I started to smell a rat (because the addon i210 behaved 
so very good) and after some fumbling in dmesg, I have noticed that 
systemd would rename (swap) eth1 for eth0, all along, probably since 
my kernel upgrade (which I did very early on).
Today with the addon board, systemd does a triple rename:
eth0 -> eth1
eth1 -> eth2
eth2 -> eth0
:-)

Trying to trace the renames in dmesg is prone to confusion.
Ultimately my favourite way of mapping the netdevice names
to PCI devices is a combination of the following two commands:
ethtool -i 
and
lspci
The output of ethtool -i contains a row labeled "bus-info", which 
quotes the familiar bus:device.function triplet, matching those 
listed by lspci.

Which means that I have a tool that's capable of HW timestamping
with an error within maybe 20-30 ns. Now for the PPS input and 
distribution across some 4 boards... I've already ordered some 
74LVC1G125 to work as level shifters.

Frank Rysanek

The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  i210_onboard_arbor.txt
 Date:  11 Dec 2017, 15:06
 Size:  2576 bytes.
 Type:  Unix-text


i210_onboard_arbor.txt
Description: Binary data
The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  i219LM_onboard_arbor_PCH_LOM.txt
 Date:  11 Dec 2017, 15:05
 Size:  2729 bytes.
 Type:  Unix-text


i219LM_onboard_arbor_PCH_LOM.txt
Description: Binary data
The following section of this message contains a file attachment
prepared for transmission using the Internet MIME message format.
If you are using Pegasus Mail, or any other MIME-compliant system,
you should be able to save it or view it from within your mailer.
If you cannot, please ask your system administrator for assistance.

    File information ---
 File:  i210_addon_intel_original_i210-T1.txt
 Date:  11 Dec 2017, 15:07
 Size:  2504 bytes.
 Type:  Unix-text


i210_addon_intel_original_i210-T1.txt
Description: Binary data
--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Despite the patch, "timed out while polling for tx timestamp" keeps happening

[Frantisek Rysanek]

On 7 Dec 2017 at 21:55, Keller, Jacob E wrote:
> > About ethtool stats - I now understand that you mean the output of
> > ethtool -S, namely the lines
> >  tx_hwtstamp_timeouts: 0
> >  tx_hwtstamp_skipped: 0
> >  rx_hwtstamp_cleared: 0
> > This is what they look like now, that the error does not occur.
> > In a few days I will probably have a chance to try it in the field
> > again, on a PTP TC switch wih GOOSE flooding the network... that's
> > where the misbehavior was most stubborn. Well now I know what to look
> > at :-) I'll report more numbers when I have some.
> > 
> 
> Ok good. If you see Tx timeouts again, try to measure the stats here
> and see if any of these increment. If they do, that's a sure
> indication that the driver was not able to obtain and send the
> timestamp to the stack. If they *do not* increment, then that means
> that the driver was likely too late when responding with the Tx
> timestamp, which is a separate problem. 
>
Thanks for the useful information!

> Oh.. It's possible that the
> device might be going to sleep too quickly.. can you check to see if
> it supports EEE? "ethtool --show-eee "? This causes the device to
> go into low power link mode which substantially increases the latency
> for actual Tx packets (when there's little to no traffic). That might
> be the reason under some circumstances why you see dropped timestamps,
> if EEE is enabled? 
>

So uh... ASPM on PCI-e may have been eliminated a couple years ago, 
but we still have the Ethernet-level energy saving to sort out?
I recall that even low-end unmanaged SoHo switches now support some 
"green" functions on Gb Eth... makes me wonder if EEE is the techical 
substance of the "green" word printed on D-Link's packaging carton.
Wikipedia mentions 802.3az... ahh yes, and so does "man ethtool",
referring to "eee".

# ethtool --show-eee eth0
EEE Settings for eth0:
EEE status: enabled - inactive
Tx LPI: 0 (us)
Supported EEE link modes:  100baseT/Full
   1000baseT/Full
Advertised EEE link modes:  100baseT/Full
1000baseT/Full
Link partner advertised EEE link modes:  Not reported

The Ethernet port is now connected straight to a 2nd-generation 
Meinberg grandmaster card, called the IMS-TSU. I understand the 
ethtool listing in the following way:
the IMS-TSU does not support EEE.
All the better.

A month ago, also in my lab, the eth0 was attached directly to a 
3rd-generation Meinberg grandmaster card, called the HPS100.
Not sure, maybe it supports EEE? That's when the TX timeouts were 
occurring a couple times a day, when combined with my i219LM eth0.

On site, eth0 was attached to a switch by Ruggedcom, working as a TC. 
Makes me wonder if the switch supports EEE. Time for me to check the 
manual. Makes me wonder if EEE combined with the GOOSE multicasts 
could result in the high frequency of TX timestamp timeouts 
observed...

Thanks for all the tips! :-)

Frank

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Despite the patch, "timed out while polling for tx timestamp" keeps happening

[Frantisek Rysanek]

On 7 Dec 2017 at 18:47, Keller, Jacob E wrote:

> > => the Intel NIC hardware is possibly sensitive to "irrelevant"
> > contents in the traffic. I can come up with the following candidate
> > culprits/theories:
> > - absence of the VLAN tag
> > - correction values of 10-20 ms
> > - other mcast traffic interfering
> > - higher/different actual jitter in the messages?
> > 
> > > Which device (and driver) are you using? (I can't see it in the history).
> > >
> > On the ptp4l client?
> > The PC is a pre-production engineering sample panel PC by Arbor/TW,
> > with Intel Skylake mobile, the NIC that I'm using is an i219LM
> > integrated on the mothereboard (not sure if this has a MAC on chip
> > within the PCH/south, or if it's a stand-alone NIC). Of the two Intel
> > NIC chips, this one is more precise. The kernel is a fresh vanilla
> > 4.13.12 and the e1000e driver came with it.
> > I'm attaching a dump of dmesg and lspci. Ask for more if you want.
> > 
> > Frank Rysanek
> 
> Do you know the packet rate for Tx packets? (How often is it
> requesting timestamps)? There was a recent-ish problem I believe we
> fixed but it appears to be in 4.13: 5012863b7347 ("e1000e: fix race
> condition around skb_tstamp_tx()", 2017-06-06), but that definitely
> should be in the 4.13 kernel.. 
> 
> There should also be statistics you can check in ethtool stats on the
> device. Could you try checking if tx_hwtstamp_timeouts is
> incrementing? Also whether tx_hwtstamp_skipped? 
> 
> Thanks,
> Jake

Dear Mr. Keller, thanks for your immediate responses and for the job 
that you're doing on the drivers. You have my deepest respect.

Yes that patch is in my e1000e driver:
https://patchwork.ozlabs.org/patch/758160/
That's the patch mentioned in the subject of this e-mail thread :-)

ptp4l sends one PDelay Request per second, and answers one
PDelay Request received from the upstream switch (per second).
That's three PTP messages transmitted per second.
There is no other TX traffic on that same port.

About ethtool stats - I now understand that you mean the output of 
ethtool -S, namely the lines
 tx_hwtstamp_timeouts: 0
 tx_hwtstamp_skipped: 0
 rx_hwtstamp_cleared: 0
This is what they look like now, that the error does not occur.
In a few days I will probably have a chance to try it in the field 
again, on a PTP TC switch wih GOOSE flooding the network... that's 
where the misbehavior was most stubborn. Well now I know what to look 
at :-) I'll report more numbers when I have some.

BTW do you know what volume of RX buffers does the i219LM have on 
chip? Or its companion MAC integrated in the PCH, if the i219 is just 
a PHY.

Frank Rysanek

--
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel

Re: [Linuxptp-devel] Despite the patch, "timed out while polling for tx timestamp" keeps happening

[Frantisek Rysanek]

Note: I'm forwarding this message with PNG attachments removed,
as I got politely and deservedly reminded that big attachments are a 
no-no in a mailing list. Here goes the message:

> > The "correction" field inserted by the RuggedCom switch contains
> > values between 10 and 20 million raw units, that's some 150 to 300ns.
> > Sounds about appropriate. Makes me wonder if the contents of the PTP
> > traffic can make the Intel hardware puke :-/ The actual jitter, or
> > the non-zero correction field... it's strange.
> > 
Actually... this is probably wrong. The value in the correction.ns 
field is about 10 to 20 million, i.e. 10 to 20 milliseconds. I can 
see the raw value in the frame (in hex) and that's what Wireshark and 
ptpTrackHound interpret, in unison. 
And, one vendor techsupport insists that a correction value of 20 ms
is perfectly allright in a TC switch, due to SW processing of the PTP 
packets. Yikes, what?
Or, is there any chance that my sniffing rig is broken? 

I've captured the PTP traffic by libpcap, 
A) either with ptp4l running in software mode as a client 
to a TC switch (with a Meinberg GM as the next upstream hop) 
B) or as a pure sniffer, listening to traffic between a 3rd-party
   client and the TC. The Intel NIC does have PTP support, but I 
   understand that it is turned off, at the time of the capture. 

Any chance that the Intel NIC hardware would mangle the correction 
field? (I hope not - after some debate in another thread, the 10-20ms 
really seem allright, even if spooky.)

I'll probably have to borrow a proper "meter" device anyway :-/

I have some other potentially interesting observations, relevant to 
ptp4l and Intel HW.

There are two GM's in play: 

GM A (older), which correlated with a problem reported on site by a 
particular 3rd-party PTP slave. Presumed buggy.

GM B (younger), whose deployment correlated with the 3rd-party slave 
becoming happy. Presumed healthy.

The 3rd-party slave is a black box, expensive, presumably 
high-quality implementation.
Let me focus on the behavior observed in ptp4l with HW accel.


I actually tried ptp4l with HW support under several slightly 
different scenaria. L2 Multicast and 2-step P2P mechanism were 
common, but details were different. 

1) with "grandmaster B", directly attached at 1 Gbps, configured for 
C37.238-2017 (including ALTERNATE_TIME_OFFSET_INDICATOR_TLV),
both ends without a VLAN tag, in my lab. That worked for the most 
part, ptp4l would throw maybe 8 TX timeouts during one night (10 
hours).

2) with "grandmaster B", on site, configured for C37.238-2017 
(including ALTERNATE_TIME_OFFSET_INDICATOR_TLV),
both ends without a VLAN tag, through a PTP-capable switch
(the one adding 10-20 ms of "correction").
Here the ptp4l with HW accel would never stop choking with TX 
timeouts. Sometimes it went for 3 to 10 PDelay transactions without a
timeout, sometimes it would run timeout after timeout.
There was 3rd-party multicast traffic on the network (IEC61850 
GOOSE).

3) with "grandmaster A", on site, direct attached, configured for 
C37.238-2011 (no local timezone TLV), but *with* a VLAN tag 
containing ID=0 configured on the GM, and *without* VLAN tag on the 
ptp4l client, the ptp4l would not sychronize to the GM. In the packet
trace I can see all the messages from the GM, and ptp4l does respond 
to the master's PDelay Requests, but the GM does *not* respond to 
ptp4l's PDelay Requests.
=> I consider this a misconfiguration on my part (PEBKAC),
even though... theoretically... VLAN ID=0 means "this packet has 
802.1p priority assigned, but does not belong to a VLAN".
The GM *could* be a little more tolerant / liberal in what it accepts
:-) Then again, I do not know the wording of the 2011 "power 
profile".

4) with "grandmaster A", direct attached, back home in the lab, 
configured for C37.238-2011 (no local timezone TLV), but *with* a 
VLAN tag containing ID=0 configured on the GM, and *with* a VLAN tag 
ID=0 on the ptp4l client (created a VLAN subinterface eth0.0), 
ptp4l now RUNS LIKE A CHEETAH FOR DAYS !
No TX timeouts in the log.

=> the Intel NIC hardware is possibly sensitive to "irrelevant" 
contents in the traffic. I can come up with the following candidate 
culprits/theories: 
- absence of the VLAN tag
- correction values of 10-20 ms
- other mcast traffic interfering
- higher/different actual jitter in the messages?

> Which device (and driver) are you using? (I can't see it in the history).
> 
On the ptp4l client?
The PC is a pre-production engineering sample panel PC by Arbor/TW, 
with Intel Skylake mobile, the NIC that I'm using is an i219LM 
integrated on the mothereboard (not sure if this has a MAC on chip 
within the PCH/south, or if it's a stand-alone NIC). Of the two Intel
NIC chips, this one is more precise. The kernel is a fresh vanilla 
4.13.12 and the e1000e driver came with it.
I'm attaching a dump of dmesg and lspci. Ask for more if you want.

Frank Rysanek



WPM$LMWC.PM$
Description: Mail