Re: [beagleboard] Re: Beaglebone time runs slow despite starting /lib/systemd/systemd-timesyncd

[wkulecz]

Thanks for the info, I figured it was something like this going on.  I may 
try refining things to a different server if necessary.

My Ubuntu-Mate desktop time always seems synced, but sudo systemctl status  
systemd-timesyncd  gives:
systemd-timesyncd.service - Network Time Synchronization
   Loaded: loaded (/lib/systemd/system/systemd-timesyncd.service; enabled; 
vendor preset: enabled)
  Drop-In: /lib/systemd/system/systemd-timesyncd.service.d
   └─disable-with-time-daemon.conf
   Active: inactive (dead)
Condition: start condition failed at Sat 2017-12-09 11:44:12 CST; 5h 24min 
ago
   ConditionFileIsExecutable=!/usr/sbin/ntpd was not met
 Docs: man:systemd-timesyncd.service(8)

Its timekeeping has always been good.  It has /etc/ntp.conf with
# Specify one or more NTP servers.

# Use servers from the NTP Pool Project. Approved by Ubuntu Technical Board
# on 2011-02-08 (LP: #104525). See http://www.pool.ntp.org/join.html for
# more information.
pool 0.ubuntu.pool.ntp.org iburst
pool 1.ubuntu.pool.ntp.org iburst
pool 2.ubuntu.pool.ntp.org iburst
pool 3.ubuntu.pool.ntp.org iburst

# Use Ubuntu's ntp server as a fallback.
pool ntp.ubuntu.com

So apparently it tries to run both and ntp wins :)
I don't recall doing anything about its time keeping other than setting the 
locale during installation in 2016.

My BBG seems to be either right on  or two seconds slow compared to the 
others  It still seems to be updating every 35 seconds.
Still puzzled.  I'll give it more time, and maybe try getting time from a 
single server if it doesn't stablize.



On Saturday, December 9, 2017 at 4:07:35 PM UTC-6, Graham wrote:
>
> For wku...
>
>
> pool.ntp.org is not a server.
>
> It is a front end service for a pool of ntp servers, that can be anywhere 
> in the world.
> There are hundreds, if not thousands of them in the pool.
> To level the loading on the ntp servers within the pool, you will get 
> assigned to a different server for each call to pool.ntp.org. 
> To see which server you are actually using, you need to look at the IP 
> address of the server, then do a whois lookup.
>
> It is likely that each of your BBB's is running off a different ntp 
> server, even though they all called to pool.ntp.org.
>
> But, to get admitted to that pool, you need to be running a full 
> implementation of NTP, which is cross synched with at least five other 
> servers, and each will likely be less than 10 ms in error from absolute 
> time.
> Many these days have their own GPS receiver, so will have time errors less 
> than 0.1 ms.
>
> If you want all of your units to use the same server, you will need to 
> specify the IP address of the specific server, in that NTP= statement.
>
> Preferably one close to you, with low latency and a symmetric path.
>
> pool.ntp.org is the Global pool.
> If you want to restrict it to something closer, use:
>
> Africa  — africa.pool.ntp.org (33)
> Antarctica  — 
> antarctica.pool.ntp.org (0)
> Asia  — asia.pool.ntp.org (264)
> Europe  — europe.pool.ntp.org (2763)
> North America  — 
> north-america.pool.ntp.org (943)
> Oceania  — oceania.pool.ntp.org 
> (113)
> South America  — 
> south-america.pool.ntp.org (48)
>
> --- Graham
>
>
>
>

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/e9b01bd3-2daa-445a-88d3-9a92ab5a4b21%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Beaglebone time runs slow despite starting /lib/systemd/systemd-timesyncd

[Graham]

>
> For wku...


pool.ntp.org is not a server.

It is a front end service for a pool of ntp servers, that can be anywhere 
in the world.
There are hundreds, if not thousands of them in the pool.
To level the loading on the ntp servers within the pool, you will get 
assigned to a different server for each call to pool.ntp.org. 
To see which server you are actually using, you need to look at the IP 
address of the server, then do a whois lookup.

It is likely that each of your BBB's is running off a different ntp server, 
even though they all called to pool.ntp.org.

But, to get admitted to that pool, you need to be running a full 
implementation of NTP, which is cross synched with at least five other 
servers, and each will likely be less than 10 ms in error from absolute 
time.
Many these days have their own GPS receiver, so will have time errors less 
than 0.1 ms.

If you want all of your units to use the same server, you will need to 
specify the IP address of the specific server, in that NTP= statement.

Preferably one close to you, with low latency and a symmetric path.

pool.ntp.org is the Global pool.
If you want to restrict it to something closer, use:

Africa  — africa.pool.ntp.org (33)
Antarctica  — 
antarctica.pool.ntp.org (0)
Asia  — asia.pool.ntp.org (264)
Europe  — europe.pool.ntp.org (2763)
North America  — 
north-america.pool.ntp.org (943)
Oceania  — oceania.pool.ntp.org (113)
South America  — 
south-america.pool.ntp.org (48)

--- Graham



-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/cadeefbc-89fd-466d-9869-b679b8d744b1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Beaglebone time runs slow despite starting /lib/systemd/systemd-timesyncd

[Graham]

Well, I loaded Debian 8.2 on one of my BBB's, and then did a cold start and
you
can see the following in syslog.  timesyncd starts off at a 32 second sntp
resync with the server, then moves up in powers of two to a maximum sync
time of about 34 minutes.

You can see it is making only single digit millisecond corrections to the
system clock, once it converges.

This BBB has had the caps across the master crystal changed to get the
master clock error down into the single digit ppm.  Most of the factory
BBB's are in the 50 to 70 ppm error range.

Dec  9 18:28:49 beaglebone systemd-timesyncd[220]: Using NTP server
66.135.44.92:123 (2.debian.pool.ntp.org).
Dec  9 18:28:49 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 32s/-0.182s/0.030s/0.000s/+0ppm
Dec  9 18:29:21 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 64s/-0.001s/0.031s/0.000s/-7ppm
Dec  9 18:30:25 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 128s/+0.002s/0.033s/0.001s/+0ppm
Dec  9 18:32:33 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 256s/-0.001s/0.032s/0.001s/-1ppm
Dec  9 18:36:49 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 512s/-0.000s/0.030s/0.001s/-1ppm
Dec  9 18:45:22 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 1024s/-0.004s/0.036s/0.002s/-1ppm
(ignored)
Dec  9 19:02:26 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 2048s/+0.001s/0.026s/0.002s/-1ppm
(ignored)
Dec  9 19:36:34 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 2048s/+0.007s/0.037s/0.003s/+0ppm
Dec  9 20:10:42 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 2048s/-0.000s/0.038s/0.003s/+0ppm
Dec  9 20:44:51 beaglebone systemd-timesyncd[220]:
interval/delta/delay/jitter/drift 2048s/-0.003s/0.042s/0.003s/+0ppm

The major source of time error is any asymetry in the inbound and outbound
path to the ntp server. (Common issue with cable modems.)

Reporting of this activity disappeared sometime between Debian 8 and Debian
9.

Reading the source code for timesyncd suggests it should still be there, as
cron.debug messages, but so far, no joy.

I'll watchstat /var/lib/systemd/clockto make sure it is running.

If anyone knows how to get the detailed timesyncd messages back, I would
like to hear about it.

--- Graham

==

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CAC_N71p9T1%3D%3DramC2opXb7ogswFtP8Scp5y3KdSWXAkVOc7MAQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Beaglebone time runs slow despite starting /lib/systemd/systemd-timesyncd

[wkulecz]

On Saturday, December 9, 2017 at 1:16:51 PM UTC-6, William Hermans wrote:
>
>
> If you're serious about keeping time. Then get an accurate real time 
> clock, and sync your system clock off that. Then update the real time clock 
> once every month, or however often you need in order to keep time as close 
> to dead on, as you need it.
>

I require sync to 1 second,  ntp has up to now "just worked" to be that 
good everywhere I've ever used it.

I realize there can be systematic errors if all systems are not using the 
same time server, that may have been my problem, or something with 
systemd-timesyncd is FuBar on Debian 8.9 and the BBG, time will tell.  The 
BBW with Debian 9.2 seems fine now that its been running for about two 
days. maybe it just takes that lonf when booting from a fresh install 
image..

I assume this answer is really meant for Graham who wants 20 mS sync.

My BBG timesyncd seems to be trying to sync every 35 seconds or so, whereas 
the BBW seems to be checking every 34 minutes or so.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/893ca9fe-c700-4cb6-afe9-b1487eb46d00%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [beagleboard] Re: Beaglebone time runs slow despite starting /lib/systemd/systemd-timesyncd

[William Hermans]

On Sat, Dec 9, 2017 at 12:08 PM,  wrote:

> Thanks, glad to see I'm not the only one with the issue.  Did some more
> searching and it seems it only logs the sync that happens at daemon start
> up, but is supposed to resync at some compiled in interval.  Check the most
> recent sync time with the times from stat /var/lib/systemd/clock
>
> On my Debian 8.9 Beaglebone Green I changed /etc/systemd/timesyncd.conf to
> have a line:
> NTP=pool.ntp.org
> This is the server that I know is being used by the Lorex DVR device which
> is the most important wrt the time sync with  my Beaglebone Green.
>
> I then did:
> sudo systemctl stop systemd-timesyncd
> sudo systemctl start systemd-timesyncd
>
> and sudo systemctl status  systemd-timesyncd showed I was apparently now
> using the pool.ntp.org server.
>
>
> Using: stat /var/lib/systemd/clock
> Mine seems to be updating every 35 seconds, although apparently the
> interval will change as sync gets better/worse.
>
> After doing this about 45 minutes ago my Beaglebone is now within 1 second
> of the Lorex, where it was about 5 seconds off before.  Is it possible that
> extra jitter in the time functions is caused by a fairly heavy and variable
> workload?  tops shows load averages: 0.20 .010. 0.07
>
> OTOH my BBW Debian 9.2 system, which has been up about a day longer than
> the BBG (did a test reboot yesterday) now seems well synced, but it is very
> lightly loaded basically just running my ssh session to make these tests.
> The systemd-timesyncd update interval seems to be about 34 minutes.
>
> So I'm not sure what is really going on here, but as long as the BBG and
> the Lorex stay synced to +/- 1 second I don't need to understand it :)
>

If you're serious about keeping time. Then get an accurate real time clock,
and sync your system clock off that. Then update the real time clock once
every month, or however often you need in order to keep time as close to
dead on, as you need it.

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/CALHSORpA%2B2t9ZDRW-MTB5YTQW%2BQHQ6dQXKY5ji536FhMJxV%2Bxg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[beagleboard] Re: Beaglebone time runs slow despite starting /lib/systemd/systemd-timesyncd

[wkulecz]

Thanks, glad to see I'm not the only one with the issue.  Did some more 
searching and it seems it only logs the sync that happens at daemon start 
up, but is supposed to resync at some compiled in interval.  Check the most 
recent sync time with the times from stat /var/lib/systemd/clock

On my Debian 8.9 Beaglebone Green I changed /etc/systemd/timesyncd.conf to 
have a line:
NTP=pool.ntp.org
This is the server that I know is being used by the Lorex DVR device which 
is the most important wrt the time sync with  my Beaglebone Green.

I then did:
sudo systemctl stop systemd-timesyncd
sudo systemctl start systemd-timesyncd

and sudo systemctl status  systemd-timesyncd showed I was apparently now 
using the pool.ntp.org server.


Using: stat /var/lib/systemd/clock
Mine seems to be updating every 35 seconds, although apparently the 
interval will change as sync gets better/worse.

After doing this about 45 minutes ago my Beaglebone is now within 1 second 
of the Lorex, where it was about 5 seconds off before.  Is it possible that 
extra jitter in the time functions is caused by a fairly heavy and variable 
workload?  tops shows load averages: 0.20 .010. 0.07

OTOH my BBW Debian 9.2 system, which has been up about a day longer than 
the BBG (did a test reboot yesterday) now seems well synced, but it is very 
lightly loaded basically just running my ssh session to make these tests.
The systemd-timesyncd update interval seems to be about 34 minutes.

So I'm not sure what is really going on here, but as long as the BBG and 
the Lorex stay synced to +/- 1 second I don't need to understand it :)






On Friday, December 8, 2017 at 10:34:32 PM UTC-6, Graham wrote:
>
> I have been chasing something similar.
> On Debian 8.8 and prior Jessie versions, systemd-timesyncd worked very 
> well and issued a lot of (level debug) messages that told you what was 
> going on, and the level of correction being applied.
> When it is running the time would stay synced within 20 milliseconds or so.
>
> On Stretch, I have been unable to receive any of these update messages.
> Normally every 30 minutes, after initial correction convergence.
>  rsyslog is correctly configured to capture them.
>
> All I see is a single syslog message at the time of initial sync, then 
> nothing.
>
> I am unable to convince myself that systemd-timesyncd is actually running, 
> even though
>
> systemctl status systemd-timesyncd.service
>
> says that it is.
>
> --- Graham
>
> ==
>
> On Friday, December 8, 2017 at 9:39:21 PM UTC-6, wku...@gmail.com wrote:
>>
>> I've an IOT application distributed over multiple systems all on a local 
>> network and  all synchronized with some flavor of ntp running.
>>
>> These all stay syncronized to within a second over long periods:
>> Raspberry Pi2
>> Raspberry Pi3
>> Raspberry PiZero-W
>> Ubuntu 16.04 desktop
>> Commercial Lorex security DVR
>>
>> These two have drifted about three seconds slow already, although they 
>> seem well synced with each other:
>> Beaglebone Green running Debian 8.9  (rebooted this morning)
>> Beaglebone White running Debian 9.2 (booted yesterday morning)
>>
>>
>> All the Raspberries are running Raspbian, verson 8 for the Pi2 and 
>> version 9 for the rest
>>
>> My initial assumption is somehow they are using different time servers 
>> that have a systematic error.  Problem is I can't find where the 
>> systemd-timesync stores its servers to see if Beaglebones  ntp server 
>> settings matches the others.
>>
>> The Lorex box would drift until I changed it sync interval from 6 hours 
>> to 1 hour.  All the systems that now stay synced appear to be using 
>> pool.ntp.org,  I can't seem to find where the Beaglbones store the 
>> server they use.
>>
>> Like with most things systemd I'm finding lots of misleading and 
>> confusing information that is generally not matching what I'm finding on my 
>> systems, but the Beaglebone solution may be as simple as it was for the 
>> Lorex device -- shorten the sync interval.  But how do I find what the 
>> interval is and then how do I shorten it?
>>
>> This system has been running and evolving since 2015, and I've noticed 
>> the Beaglebones generally would be "off" + or - a few seconds from the 
>> beginning of development, but I'm just now getting to adding the feature 
>> that needs time sync accurate within one second.
>>
>>
>>

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/c25c66eb-d669-43f1-9674-53e6361462d6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[beagleboard] Re: Beaglebone time runs slow despite starting /lib/systemd/systemd-timesyncd

[Graham]

I have been chasing something similar.
On Debian 8.8 and prior Jessie versions, systemd-timesyncd worked very well 
and issued a lot of (level debug) messages that told you what was going on, 
and the level of correction being applied.
When it is running the time would stay synced within 20 milliseconds or so.

On Stretch, I have been unable to receive any of these update messages.
Normally every 30 minutes, after initial correction convergence.
 rsyslog is correctly configured to capture them.

All I see is a single syslog message at the time of initial sync, then 
nothing.

I am unable to convince myself that systemd-timesyncd is actually running, 
even though

systemctl status systemd-timesyncd.service

says that it is.

--- Graham

==

On Friday, December 8, 2017 at 9:39:21 PM UTC-6, wku...@gmail.com wrote:
>
> I've an IOT application distributed over multiple systems all on a local 
> network and  all synchronized with some flavor of ntp running.
>
> These all stay syncronized to within a second over long periods:
> Raspberry Pi2
> Raspberry Pi3
> Raspberry PiZero-W
> Ubuntu 16.04 desktop
> Commercial Lorex security DVR
>
> These two have drifted about three seconds slow already, although they 
> seem well synced with each other:
> Beaglebone Green running Debian 8.9  (rebooted this morning)
> Beaglebone White running Debian 9.2 (booted yesterday morning)
>
>
> All the Raspberries are running Raspbian, verson 8 for the Pi2 and version 
> 9 for the rest
>
> My initial assumption is somehow they are using different time servers 
> that have a systematic error.  Problem is I can't find where the 
> systemd-timesync stores its servers to see if Beaglebones  ntp server 
> settings matches the others.
>
> The Lorex box would drift until I changed it sync interval from 6 hours to 
> 1 hour.  All the systems that now stay synced appear to be using 
> pool.ntp.org,  I can't seem to find where the Beaglbones store the server 
> they use.
>
> Like with most things systemd I'm finding lots of misleading and confusing 
> information that is generally not matching what I'm finding on my systems, 
> but the Beaglebone solution may be as simple as it was for the Lorex device 
> -- shorten the sync interval.  But how do I find what the interval is and 
> then how do I shorten it?
>
> This system has been running and evolving since 2015, and I've noticed the 
> Beaglebones generally would be "off" + or - a few seconds from the 
> beginning of development, but I'm just now getting to adding the feature 
> that needs time sync accurate within one second.
>
>
>

-- 
For more options, visit http://beagleboard.org/discuss
--- 
You received this message because you are subscribed to the Google Groups 
"BeagleBoard" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to beagleboard+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/beagleboard/e68d2316-638c-4c02-a874-969ab3bb96e8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.