Re: [Linuxptp-devel] [Linuxptp-users] phc2sys jump into huge value
hi Jake, Thanks for your response. Not sure I follow here. You mean BC as in the boundary clock you're connected to? >> Yes. You could possibly filter or average the delay to remove outliers, but I am not sure how this impacts things. Other experts on this list are probably better to answer that. >> Once in a while, delay is observed while coping from master clock source to >> other clock source via phc2sys. As both clock are within the systems, may be >> delay is caused within the systems example like PCIe bus congestion. I hope its fine to implement Moving average delay to remove any delay/latency in phc2sys within the system. Regards,Ramesh .On Thursday, April 29, 2021, 10:57:57 PM GMT+5:30, Jacob Keller wrote: On 4/29/2021 8:04 AM, ramesh t wrote: > hello Jake, > > Did time profiling using time ticks. > Under the problem condition, observing clock_gettime of interface > connected to BC is taking more time ticks. This results in phc offset > jumping to 4 digit value momentarily. Also i'm not sure if reading > across numa is triggering this issue. > Not sure I follow here. You mean BC as in the boundary clock you're connected to? > Please suggest a way to fix in the phc2sys offset code? > 1) Should we prevent update on phc2sys value if there is momentarily jump? > You could possibly filter or average the delay to remove outliers, but I am not sure how this impacts things. Other experts on this list are probably better to answer that. To me, it sounds like the issue is that in some cases the DELAY_REQ packet is actually taking longer in one case. You might also try peer delay instead if your network setup supports it? > Please suggest. > > Thanks for your support on this issue. > > Regards, > Ramesh > On Thursday, April 29, 2021, 07:33:49 AM GMT+5:30, ramesh t via > Linuxptp-users wrote: ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
Re: [Linuxptp-devel] [Linuxptp-users] phc2sys jump into huge value
On 4/29/2021 8:04 AM, ramesh t wrote: > hello Jake, > > Did time profiling using time ticks. > Under the problem condition, observing clock_gettime of interface > connected to BC is taking more time ticks. This results in phc offset > jumping to 4 digit value momentarily. Also i'm not sure if reading > across numa is triggering this issue. > Not sure I follow here. You mean BC as in the boundary clock you're connected to? > Please suggest a way to fix in the phc2sys offset code? > 1) Should we prevent update on phc2sys value if there is momentarily jump? > You could possibly filter or average the delay to remove outliers, but I am not sure how this impacts things. Other experts on this list are probably better to answer that. To me, it sounds like the issue is that in some cases the DELAY_REQ packet is actually taking longer in one case. You might also try peer delay instead if your network setup supports it? > Please suggest. > > Thanks for your support on this issue. > > Regards, > Ramesh > On Thursday, April 29, 2021, 07:33:49 AM GMT+5:30, ramesh t via > Linuxptp-users wrote: ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
Re: [Linuxptp-devel] [Linuxptp-users] phc2sys jump into huge value
hi Richard, We have core pinned the ptp4l and phc2sys process to run on non system cores. Do think chrt is required? Please suggest. Thanks for your support. Regards,Ramesh On Thursday, April 29, 2021, 08:45:26 PM GMT+5:30, Richard Cochran wrote: On Thu, Apr 29, 2021 at 03:04:07PM +, ramesh t via Linuxptp-users wrote: > Did time profiling using time ticks. Under the problem condition, > observing clock_gettime of interface connected to BC is taking more > time ticks. This results in phc offset jumping to 4 digit value > momentarily. Also i'm not sure if reading across numa is triggering > this issue. Please suggest a way to fix in the phc2sys offset > code?1) Should we prevent update on phc2sys value if there is > momentarily jump? You can use 'chrt' to give phc2sys scheduling priority so that it will not be preempted. HTH, Richard ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
Re: [Linuxptp-devel] [Linuxptp-users] phc2sys jump into huge value
On Thu, Apr 29, 2021 at 03:04:07PM +, ramesh t via Linuxptp-users wrote: > Did time profiling using time ticks. Under the problem condition, > observing clock_gettime of interface connected to BC is taking more > time ticks. This results in phc offset jumping to 4 digit value > momentarily. Also i'm not sure if reading across numa is triggering > this issue. Please suggest a way to fix in the phc2sys offset > code?1) Should we prevent update on phc2sys value if there is > momentarily jump? You can use 'chrt' to give phc2sys scheduling priority so that it will not be preempted. HTH, Richard ___ Linuxptp-devel mailing list Linuxptp-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
Re: [Linuxptp-devel] [Linuxptp-users] phc2sys jump into huge value
hello Jake,
Did time profiling using time ticks. Under the problem condition, observing
clock_gettime of interface connected to BC is taking more time ticks. This
results in phc offset jumping to 4 digit value momentarily. Also i'm not sure
if reading across numa is triggering this issue.
Please suggest a way to fix in the phc2sys offset code?1) Should we prevent
update on phc2sys value if there is momentarily jump?
Please suggest.
Thanks for your support on this issue.
Regards,Ramesh On Thursday, April 29, 2021, 07:33:49 AM GMT+5:30, ramesh t
via Linuxptp-users wrote:
hi Jake,
> enp104s0f1 phc offset 3 s2 freq -11417 delay 3568
> enp104s0f1 phc offset -4973 s2 freq -16393 delay 19341* <<
> enp104s0f1 phc offset 4977 s2 freq -7934 delay 3594*
> enp104s0f1 phc offset 1471 s2 freq -9947 delay 3591*
> enp104s0f1 phc offset 0 s2 freq -10977 delay 3587
Main issue in hand is why there is change in delay which resulted in jump of
phc offset? which recovers back? Is there any mechanism to avoid such drifts
the phc?
>From the ptp4l code, below code is used for calculating delay
if (clock_gettime(sysclk, &tdst1) || <<< Read of local nic time
clock_gettime(clkid, &tsrc) || <<< Read of remote
nic time(interface connected to BC)
clock_gettime(sysclk,&tdst2)) { <<< Read of local nic time
pr_err("failed to read clock: %m"); return 0;
}Then we calculate the interval between two reads of localNIC with below
code:
interval = (tdst2.tv_sec - tdst1.tv_sec) * NS_PER_SEC + tdst2.tv_nsec -
tdst1.tv_nsec;This interval maps to delay.
Please suggest.
Regards,Ramesh
On Thursday, April 29, 2021, 04:05:34 AM GMT+5:30, Jacob Keller
wrote:
On 4/28/2021 12:01 AM, ramesh t via Linuxptp-users wrote:
> hi,
>
> In our system, we have 3-4 ptp4l services running as master (C1 mode)
> providing clock to external device. Our system is synced using ptp
> support/slave connected to a BC. From the synced slave/support
> interface, phc2sys is used to sync master interface. All the master and
> slave ptp4l and phc2sys are core pinned. Once in a while phc offset
> which is synced into master interface using phc2sys jumps into 4 digit
> value. Any suggestion on how to debug this further.
>
> Apr 28 03:26:51 k8s-worker-0 phc2sys: [131370.261] enp104s0f1 phc
> offset 4 s2 freq -11419 delay 3610
> Apr 28 03:26:52 k8s-worker-0 phc2sys: [131371.261] enp104s0f1 phc
> offset 3 s2 freq -11418 delay 3603
> Apr 28 03:26:53 k8s-worker-0 phc2sys: [131372.261] enp104s0f1 phc
> offset 3 s2 freq -11417 delay 3568
> *Apr 28 03:26:54 k8s-worker-0 phc2sys: [131373.262] enp104s0f1 phc
> offset -4973 s2 freq -16393 delay 19341*
> *Apr 28 03:26:55 k8s-worker-0 phc2sys: [131374.262] enp104s0f1 phc
> offset 4977 s2 freq -7934 delay 3594*
> *Apr 28 03:26:56 k8s-worker-0 phc2sys: [131375.262] enp104s0f1 phc
> offset 1471 s2 freq -9947 delay 3591*
> Apr 28 03:26:57 k8s-worker-0 phc2sys: [131376.262] enp104s0f1 phc
> offset 0 s2 freq -10977 delay 3587
> Apr 28 03:26:58 k8s-worker-0 phc2sys: [131377.262] enp104s0f1 phc
> offset -449 s2 freq -11426 delay 3596
>
Ok
> After a while, we are observing NIC interface going down also.
>
> Apr 28 03:28:22 k8s-worker-0 phc2sys: [131461.271] enp104s0f1 phc
> offset 5 s2 freq -11421 delay 3559
> Apr 28 03:28:23 k8s-worker-0 kernel: i40e :68:00.1 enp104s0f1: NIC
> Link is Down
> Apr 28 03:28:23 k8s-worker-0 kernel: br0: port 4(enp104s0f1.1) entered
> disabled state
>
> Please suggest.
This looks like the i40e driver. This hardware has a clock that is
derived from the link speed. For this reason, the clock increment value
has to change whenever link status changes. I suspect that when the link
goes down, the driver is re-programming the increment value which is why
you see the change in clock.
You'd need to figure out why the link is dropping. There isn't much that
can be done from a driver or ptp4l perspective here.
Thanks,
Jake
>
> Regards,
> Ramesh
>
>
> ___
> Linuxptp-users mailing list
> linuxptp-us...@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linuxptp-users
>
___
Linuxptp-users mailing list
linuxptp-us...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-users
___
Linuxptp-users mailing list
linuxptp-us...@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-users
___
Linuxptp-devel mailing list
Linuxptp-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linuxptp-devel
