Re: bhyve clock issue

2016-04-26 Thread Ilario Febi

Il 26/04/16 16:50, Neel Natu ha scritto:

What was the timecounter being used when you were seeing the time skew?


HPET


--
*Ing. Ilario Febi

Schema 31 S.p.a. Socio Unico*
*System Administrator

*BO - FI - ROMA - PA
ITALY
Tel: +39. 06.98.358.472
Cell: +39. 366.69.45.384
Fax: +39. 055.71.880.466
Società del Gruppo *SC31 ITALIA*
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to 
"freebsd-virtualization-unsubscr...@freebsd.org"

Re: bhyve clock issue

2016-04-26 Thread Neel Natu
What was the timecounter being used when you were seeing the time skew?

Best
Neel

> On Apr 26, 2016, at 6:46 AM, Ilario Febi  wrote:
> 
> Yes, 
> 
> I found errors in /var/log/messages and fix adding: 
> 
> kern.timecounter.hardware=TSC-low
> into /etc/sysctl.conf file
> 
> Thanks for support.
> 
> 
> 
> Il 14/04/16 18:13, Neel Natu ha scritto:
>> Hi,
>> 
>> Did you see anything interesting in the ntpd log file around the time the 
>> large skew happened?
>> 
>> Best
>> Neel
>> 
>>> On Apr 13, 2016, at 3:44 AM, Ilario Febi  wrote:
>>> 
>>> Hi All
>>> 
>>> I have a strange issue about the clock of some FreeBSD Bhyve guests,
>>> maybe someone of you can help me to diagnose it and work on a
>>> solution.
>>> 
>>> In detail, sometimes (without an apparent regularity) one of four
>>> bhyve guest's clock goes exactly 7 minutes and 10 seconds in the
>>> future.
>>> 
>>> This clock stays unaligned for about 20 minutes, then, without any
>>> human intervention, the clock realign to proper condition.
>>> 
>>> My configuration is:
>>> 
>>> envrm02 (bhyve HOST, 10.2-RELEASE-p14)
>>> -> og-qa (bhyve guest - 10.2-RELEASE-p14)
>>> -> og-ci (bhyve guest - 10.2-RELEASE-p14)
>>> -> og-test (bhyve guest - 10.2-RELEASE-p14)
>>> -> og-prod-rm (bhyve guest - 10.2-RELEASE-p14)
>>> All the og-* machines just have a running copy of tomcat.
>>> 
>>> I built a simple test to catch the error with python/fabric... the test is:
>>> 
>>> fab -R ENV,OG,OG-PROD -- date | awk -F ': ' '/2016/{print $2, $1}'
>>> 
>>> which simply run the command "date" on all hosts.
>>> 
>>> So, the result is:
>>> @Mac-mini ~/p/f/> ./check_clock.sh
>>> Thu Apr  7 17:38:00 CEST 2016 [envrm02] out
>>> Thu Apr  7 17:38:00 CEST 2016 [og-qa] out
>>> Thu Apr  7 17:38:00 CEST 2016 [og-ci] out
>>> Thu Apr  7 17:38:00 CEST 2016 [og-test] out
>>> Thu Apr  7 17:38:00 CEST 2016 [og-prod-rm] out
>>> 
>>> @Mac-mini ~/p/f/> ./check_clock.sh
>>> Thu Apr  7 18:38:00 CEST 2016 [envrm02] out
>>> Thu Apr  7 18:45:10 CEST 2016 [og-qa] out
>>> Thu Apr  7 18:38:00 CEST 2016 [og-ci] out
>>> Thu Apr  7 18:38:00 CEST 2016 [og-test] out
>>> Thu Apr  7 18:38:00 CEST 2016 [og-prod-rm] out
>>> 
>>> @Mac-mini ~/p/f/> ./check_clock.sh
>>> Thu Apr  7 18:41:05 CEST 2016 [envrm02] out
>>> Thu Apr  7 18:48:15 CEST 2016 [og-qa] out
>>> Thu Apr  7 18:41:05 CEST 2016 [og-ci] out
>>> Thu Apr  7 18:41:05 CEST 2016 [og-test] out
>>> Thu Apr  7 18:41:05 CEST 2016 [og-prod-rm] out
>>> 
>>> @Mac-mini ~/p/f/> ./check_clock.sh
>>> Thu Apr  7 18:46:51 CEST 2016 [envrm02] out
>>> Thu Apr  7 18:46:52 CEST 2016 [og-qa] out
>>> Thu Apr  7 18:46:52 CEST 2016 [og-ci] out
>>> Thu Apr  7 18:46:53 CEST 2016 [og-test] out
>>> Thu Apr  7 18:46:53 CEST 2016 [og-prod-rm] out
>>> 
>>> The configuration is the same for all hosts and is this:
>>> @Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/rc.conf | grep ntpd
>>> [envrm02] out: ntpd_enable="YES"
>>> [og-qa] out: ntpd_enable="YES"
>>> [og-qa] out: ntpd_sync_on_start="YES"
>>> [og-ci] out: ntpd_enable="YES"
>>> [og-ci] out: ntpd_sync_on_start="YES"
>>> [og-test] out: ntpd_enable="YES"
>>> [og-test] out: ntpd_sync_on_start="YES"
>>> [og-prod-rm] out: ntpd_enable="YES"
>>> [og-prod-rm] out: ntpd_sync_on_start="YES"
>>> 
>>> @Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/ntp.conf | grep -v 
>>> 'out: #'
>>> [envrm02] Executing task ''
>>> [envrm02] run: cat /etc/ntp.conf
>>> [envrm02] out:
>>> [envrm02] out: server 0.freebsd.pool.ntp.org iburst
>>> [envrm02] out: server 1.freebsd.pool.ntp.org iburst
>>> [envrm02] out: server 2.freebsd.pool.ntp.org iburst
>>> [envrm02] out:
>>> [envrm02] out: restrict default limited kod nomodify notrap nopeer noquery
>>> [envrm02] out: restrict -6 default limited kod nomodify notrap nopeer 
>>> noquery
>>> [envrm02] out: restrict 127.0.0.1
>>> [envrm02] out: restrict -6 ::1
>>> [envrm02] out: restrict 127.127.1.0
>>> 
>>> [og-qa] Executing task ''
>>> [og-qa] run: cat /etc/ntp.conf
>>> [og-qa] out:
>>> [og-qa] out: server 0.freebsd.pool.ntp.org iburst
>>> [og-qa] out: server 1.freebsd.pool.ntp.org iburst
>>> [og-qa] out: server 2.freebsd.pool.ntp.org iburst
>>> [og-qa] out:
>>> [og-qa] out: restrict default limited kod nomodify notrap nopeer noquery
>>> [og-qa] out: restrict -6 default limited kod nomodify notrap nopeer noquery
>>> [og-qa] out: restrict 127.0.0.1
>>> [og-qa] out: restrict -6 ::1
>>> [og-qa] out: restrict 127.127.1.0
>>> 
>>> [og-ci] Executing task ''
>>> [og-ci] run: cat /etc/ntp.conf
>>> [og-ci] out:
>>> [og-ci] out: server 0.freebsd.pool.ntp.org iburst
>>> [og-ci] out: server 1.freebsd.pool.ntp.org iburst
>>> [og-ci] out: server 2.freebsd.pool.ntp.org iburst
>>> [og-ci] out:
>>> [og-ci] out: restrict default limited kod nomodify notrap nopeer noquery
>>> [og-ci] out: restrict -6 default limited kod nomodify notrap nopeer noquery
>>> [og-ci] out: restrict 127.0.0.1
>>> [og-ci] out: restrict -6 ::1
>>> [og-ci] out: restrict 127.127.1.0
>>> 
>>> [og-test] Executing task ''
>>> [og-test] run: cat /etc/ntp.conf
>>

Re: bhyve clock issue

2016-04-26 Thread Ilario Febi

Yes,

I found errors in /var/log/messages and fix adding:

kern.timecounter.hardware=TSC-low
into /etc/sysctl.conf file

Thanks for support.



Il 14/04/16 18:13, Neel Natu ha scritto:

Hi,

Did you see anything interesting in the ntpd log file around the time the large 
skew happened?

Best
Neel


On Apr 13, 2016, at 3:44 AM, Ilario Febi  wrote:

Hi All

I have a strange issue about the clock of some FreeBSD Bhyve guests,
maybe someone of you can help me to diagnose it and work on a
solution.

In detail, sometimes (without an apparent regularity) one of four
bhyve guest's clock goes exactly 7 minutes and 10 seconds in the
future.

This clock stays unaligned for about 20 minutes, then, without any
human intervention, the clock realign to proper condition.

My configuration is:

envrm02 (bhyve HOST, 10.2-RELEASE-p14)
-> og-qa (bhyve guest - 10.2-RELEASE-p14)
-> og-ci (bhyve guest - 10.2-RELEASE-p14)
-> og-test (bhyve guest - 10.2-RELEASE-p14)
-> og-prod-rm (bhyve guest - 10.2-RELEASE-p14)
All the og-* machines just have a running copy of tomcat.

I built a simple test to catch the error with python/fabric... the test is:

fab -R ENV,OG,OG-PROD -- date | awk -F ': ' '/2016/{print $2, $1}'

which simply run the command "date" on all hosts.

So, the result is:
@Mac-mini ~/p/f/> ./check_clock.sh
Thu Apr  7 17:38:00 CEST 2016 [envrm02] out
Thu Apr  7 17:38:00 CEST 2016 [og-qa] out
Thu Apr  7 17:38:00 CEST 2016 [og-ci] out
Thu Apr  7 17:38:00 CEST 2016 [og-test] out
Thu Apr  7 17:38:00 CEST 2016 [og-prod-rm] out

@Mac-mini ~/p/f/> ./check_clock.sh
Thu Apr  7 18:38:00 CEST 2016 [envrm02] out
Thu Apr  7 18:45:10 CEST 2016 [og-qa] out
Thu Apr  7 18:38:00 CEST 2016 [og-ci] out
Thu Apr  7 18:38:00 CEST 2016 [og-test] out
Thu Apr  7 18:38:00 CEST 2016 [og-prod-rm] out

@Mac-mini ~/p/f/> ./check_clock.sh
Thu Apr  7 18:41:05 CEST 2016 [envrm02] out
Thu Apr  7 18:48:15 CEST 2016 [og-qa] out
Thu Apr  7 18:41:05 CEST 2016 [og-ci] out
Thu Apr  7 18:41:05 CEST 2016 [og-test] out
Thu Apr  7 18:41:05 CEST 2016 [og-prod-rm] out

@Mac-mini ~/p/f/> ./check_clock.sh
Thu Apr  7 18:46:51 CEST 2016 [envrm02] out
Thu Apr  7 18:46:52 CEST 2016 [og-qa] out
Thu Apr  7 18:46:52 CEST 2016 [og-ci] out
Thu Apr  7 18:46:53 CEST 2016 [og-test] out
Thu Apr  7 18:46:53 CEST 2016 [og-prod-rm] out

The configuration is the same for all hosts and is this:
@Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/rc.conf | grep ntpd
[envrm02] out: ntpd_enable="YES"
[og-qa] out: ntpd_enable="YES"
[og-qa] out: ntpd_sync_on_start="YES"
[og-ci] out: ntpd_enable="YES"
[og-ci] out: ntpd_sync_on_start="YES"
[og-test] out: ntpd_enable="YES"
[og-test] out: ntpd_sync_on_start="YES"
[og-prod-rm] out: ntpd_enable="YES"
[og-prod-rm] out: ntpd_sync_on_start="YES"

@Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/ntp.conf | grep -v 'out: #'
[envrm02] Executing task ''
[envrm02] run: cat /etc/ntp.conf
[envrm02] out:
[envrm02] out: server 0.freebsd.pool.ntp.org iburst
[envrm02] out: server 1.freebsd.pool.ntp.org iburst
[envrm02] out: server 2.freebsd.pool.ntp.org iburst
[envrm02] out:
[envrm02] out: restrict default limited kod nomodify notrap nopeer noquery
[envrm02] out: restrict -6 default limited kod nomodify notrap nopeer noquery
[envrm02] out: restrict 127.0.0.1
[envrm02] out: restrict -6 ::1
[envrm02] out: restrict 127.127.1.0

[og-qa] Executing task ''
[og-qa] run: cat /etc/ntp.conf
[og-qa] out:
[og-qa] out: server 0.freebsd.pool.ntp.org iburst
[og-qa] out: server 1.freebsd.pool.ntp.org iburst
[og-qa] out: server 2.freebsd.pool.ntp.org iburst
[og-qa] out:
[og-qa] out: restrict default limited kod nomodify notrap nopeer noquery
[og-qa] out: restrict -6 default limited kod nomodify notrap nopeer noquery
[og-qa] out: restrict 127.0.0.1
[og-qa] out: restrict -6 ::1
[og-qa] out: restrict 127.127.1.0

[og-ci] Executing task ''
[og-ci] run: cat /etc/ntp.conf
[og-ci] out:
[og-ci] out: server 0.freebsd.pool.ntp.org iburst
[og-ci] out: server 1.freebsd.pool.ntp.org iburst
[og-ci] out: server 2.freebsd.pool.ntp.org iburst
[og-ci] out:
[og-ci] out: restrict default limited kod nomodify notrap nopeer noquery
[og-ci] out: restrict -6 default limited kod nomodify notrap nopeer noquery
[og-ci] out: restrict 127.0.0.1
[og-ci] out: restrict -6 ::1
[og-ci] out: restrict 127.127.1.0

[og-test] Executing task ''
[og-test] run: cat /etc/ntp.conf
[og-test] out:
[og-test] out: server 0.freebsd.pool.ntp.org iburst
[og-test] out: server 1.freebsd.pool.ntp.org iburst
[og-test] out: server 2.freebsd.pool.ntp.org iburst
[og-test] out:
[og-test] out: restrict default limited kod nomodify notrap nopeer noquery
[og-test] out: restrict -6 default limited kod nomodify notrap nopeer noquery
[og-test] out: restrict 127.0.0.1
[og-test] out: restrict -6 ::1
[og-test] out: restrict 127.127.1.0

Some observation:
* The clock of envrm02 (the HOST) always seems to be correct.
* The issue happens on a (apparently) random guest (not always the same one).
* The issue happens always on one host at a time.

Re: bhyve clock issue

2016-04-14 Thread Neel Natu
Hi,

Did you see anything interesting in the ntpd log file around the time the large 
skew happened?

Best
Neel

> On Apr 13, 2016, at 3:44 AM, Ilario Febi  wrote:
> 
> Hi All
> 
> I have a strange issue about the clock of some FreeBSD Bhyve guests,
> maybe someone of you can help me to diagnose it and work on a
> solution.
> 
> In detail, sometimes (without an apparent regularity) one of four
> bhyve guest's clock goes exactly 7 minutes and 10 seconds in the
> future.
> 
> This clock stays unaligned for about 20 minutes, then, without any
> human intervention, the clock realign to proper condition.
> 
> My configuration is:
> 
> envrm02 (bhyve HOST, 10.2-RELEASE-p14)
> -> og-qa (bhyve guest - 10.2-RELEASE-p14)
> -> og-ci (bhyve guest - 10.2-RELEASE-p14)
> -> og-test (bhyve guest - 10.2-RELEASE-p14)
> -> og-prod-rm (bhyve guest - 10.2-RELEASE-p14)
> All the og-* machines just have a running copy of tomcat.
> 
> I built a simple test to catch the error with python/fabric... the test is:
> 
> fab -R ENV,OG,OG-PROD -- date | awk -F ': ' '/2016/{print $2, $1}'
> 
> which simply run the command "date" on all hosts.
> 
> So, the result is:
> @Mac-mini ~/p/f/> ./check_clock.sh
> Thu Apr  7 17:38:00 CEST 2016 [envrm02] out
> Thu Apr  7 17:38:00 CEST 2016 [og-qa] out
> Thu Apr  7 17:38:00 CEST 2016 [og-ci] out
> Thu Apr  7 17:38:00 CEST 2016 [og-test] out
> Thu Apr  7 17:38:00 CEST 2016 [og-prod-rm] out
> 
> @Mac-mini ~/p/f/> ./check_clock.sh
> Thu Apr  7 18:38:00 CEST 2016 [envrm02] out
> Thu Apr  7 18:45:10 CEST 2016 [og-qa] out
> Thu Apr  7 18:38:00 CEST 2016 [og-ci] out
> Thu Apr  7 18:38:00 CEST 2016 [og-test] out
> Thu Apr  7 18:38:00 CEST 2016 [og-prod-rm] out
> 
> @Mac-mini ~/p/f/> ./check_clock.sh
> Thu Apr  7 18:41:05 CEST 2016 [envrm02] out
> Thu Apr  7 18:48:15 CEST 2016 [og-qa] out
> Thu Apr  7 18:41:05 CEST 2016 [og-ci] out
> Thu Apr  7 18:41:05 CEST 2016 [og-test] out
> Thu Apr  7 18:41:05 CEST 2016 [og-prod-rm] out
> 
> @Mac-mini ~/p/f/> ./check_clock.sh
> Thu Apr  7 18:46:51 CEST 2016 [envrm02] out
> Thu Apr  7 18:46:52 CEST 2016 [og-qa] out
> Thu Apr  7 18:46:52 CEST 2016 [og-ci] out
> Thu Apr  7 18:46:53 CEST 2016 [og-test] out
> Thu Apr  7 18:46:53 CEST 2016 [og-prod-rm] out
> 
> The configuration is the same for all hosts and is this:
> @Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/rc.conf | grep ntpd
> [envrm02] out: ntpd_enable="YES"
> [og-qa] out: ntpd_enable="YES"
> [og-qa] out: ntpd_sync_on_start="YES"
> [og-ci] out: ntpd_enable="YES"
> [og-ci] out: ntpd_sync_on_start="YES"
> [og-test] out: ntpd_enable="YES"
> [og-test] out: ntpd_sync_on_start="YES"
> [og-prod-rm] out: ntpd_enable="YES"
> [og-prod-rm] out: ntpd_sync_on_start="YES"
> 
> @Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/ntp.conf | grep -v 'out: 
> #'
> [envrm02] Executing task ''
> [envrm02] run: cat /etc/ntp.conf
> [envrm02] out:
> [envrm02] out: server 0.freebsd.pool.ntp.org iburst
> [envrm02] out: server 1.freebsd.pool.ntp.org iburst
> [envrm02] out: server 2.freebsd.pool.ntp.org iburst
> [envrm02] out:
> [envrm02] out: restrict default limited kod nomodify notrap nopeer noquery
> [envrm02] out: restrict -6 default limited kod nomodify notrap nopeer noquery
> [envrm02] out: restrict 127.0.0.1
> [envrm02] out: restrict -6 ::1
> [envrm02] out: restrict 127.127.1.0
> 
> [og-qa] Executing task ''
> [og-qa] run: cat /etc/ntp.conf
> [og-qa] out:
> [og-qa] out: server 0.freebsd.pool.ntp.org iburst
> [og-qa] out: server 1.freebsd.pool.ntp.org iburst
> [og-qa] out: server 2.freebsd.pool.ntp.org iburst
> [og-qa] out:
> [og-qa] out: restrict default limited kod nomodify notrap nopeer noquery
> [og-qa] out: restrict -6 default limited kod nomodify notrap nopeer noquery
> [og-qa] out: restrict 127.0.0.1
> [og-qa] out: restrict -6 ::1
> [og-qa] out: restrict 127.127.1.0
> 
> [og-ci] Executing task ''
> [og-ci] run: cat /etc/ntp.conf
> [og-ci] out:
> [og-ci] out: server 0.freebsd.pool.ntp.org iburst
> [og-ci] out: server 1.freebsd.pool.ntp.org iburst
> [og-ci] out: server 2.freebsd.pool.ntp.org iburst
> [og-ci] out:
> [og-ci] out: restrict default limited kod nomodify notrap nopeer noquery
> [og-ci] out: restrict -6 default limited kod nomodify notrap nopeer noquery
> [og-ci] out: restrict 127.0.0.1
> [og-ci] out: restrict -6 ::1
> [og-ci] out: restrict 127.127.1.0
> 
> [og-test] Executing task ''
> [og-test] run: cat /etc/ntp.conf
> [og-test] out:
> [og-test] out: server 0.freebsd.pool.ntp.org iburst
> [og-test] out: server 1.freebsd.pool.ntp.org iburst
> [og-test] out: server 2.freebsd.pool.ntp.org iburst
> [og-test] out:
> [og-test] out: restrict default limited kod nomodify notrap nopeer noquery
> [og-test] out: restrict -6 default limited kod nomodify notrap nopeer noquery
> [og-test] out: restrict 127.0.0.1
> [og-test] out: restrict -6 ::1
> [og-test] out: restrict 127.127.1.0
> 
> Some observation:
> * The clock of envrm02 (the HOST) always seems to be correct.
> * The issue happens on a (apparently) random guest (not alw

bhyve clock issue

2016-04-13 Thread Ilario Febi
 Hi All

I have a strange issue about the clock of some FreeBSD Bhyve guests,
maybe someone of you can help me to diagnose it and work on a
solution.

In detail, sometimes (without an apparent regularity) one of four
bhyve guest's clock goes exactly 7 minutes and 10 seconds in the
future.

This clock stays unaligned for about 20 minutes, then, without any
human intervention, the clock realign to proper condition.

My configuration is:

envrm02 (bhyve HOST, 10.2-RELEASE-p14)
-> og-qa (bhyve guest - 10.2-RELEASE-p14)
-> og-ci (bhyve guest - 10.2-RELEASE-p14)
-> og-test (bhyve guest - 10.2-RELEASE-p14)
-> og-prod-rm (bhyve guest - 10.2-RELEASE-p14)
All the og-* machines just have a running copy of tomcat.

I built a simple test to catch the error with python/fabric... the test is:

fab -R ENV,OG,OG-PROD -- date | awk -F ': ' '/2016/{print $2, $1}'

which simply run the command "date" on all hosts.

So, the result is:
@Mac-mini ~/p/f/> ./check_clock.sh
Thu Apr  7 17:38:00 CEST 2016 [envrm02] out
Thu Apr  7 17:38:00 CEST 2016 [og-qa] out
Thu Apr  7 17:38:00 CEST 2016 [og-ci] out
Thu Apr  7 17:38:00 CEST 2016 [og-test] out
Thu Apr  7 17:38:00 CEST 2016 [og-prod-rm] out

@Mac-mini ~/p/f/> ./check_clock.sh
Thu Apr  7 18:38:00 CEST 2016 [envrm02] out
Thu Apr  7 18:45:10 CEST 2016 [og-qa] out
Thu Apr  7 18:38:00 CEST 2016 [og-ci] out
Thu Apr  7 18:38:00 CEST 2016 [og-test] out
Thu Apr  7 18:38:00 CEST 2016 [og-prod-rm] out

@Mac-mini ~/p/f/> ./check_clock.sh
Thu Apr  7 18:41:05 CEST 2016 [envrm02] out
Thu Apr  7 18:48:15 CEST 2016 [og-qa] out
Thu Apr  7 18:41:05 CEST 2016 [og-ci] out
Thu Apr  7 18:41:05 CEST 2016 [og-test] out
Thu Apr  7 18:41:05 CEST 2016 [og-prod-rm] out

@Mac-mini ~/p/f/> ./check_clock.sh
Thu Apr  7 18:46:51 CEST 2016 [envrm02] out
Thu Apr  7 18:46:52 CEST 2016 [og-qa] out
Thu Apr  7 18:46:52 CEST 2016 [og-ci] out
Thu Apr  7 18:46:53 CEST 2016 [og-test] out
Thu Apr  7 18:46:53 CEST 2016 [og-prod-rm] out

The configuration is the same for all hosts and is this:
@Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/rc.conf | grep ntpd
[envrm02] out: ntpd_enable="YES"
[og-qa] out: ntpd_enable="YES"
[og-qa] out: ntpd_sync_on_start="YES"
[og-ci] out: ntpd_enable="YES"
[og-ci] out: ntpd_sync_on_start="YES"
[og-test] out: ntpd_enable="YES"
[og-test] out: ntpd_sync_on_start="YES"
[og-prod-rm] out: ntpd_enable="YES"
[og-prod-rm] out: ntpd_sync_on_start="YES"

@Mac-mini ~/p/f/> fab -R ENV,OG,OG-PROD -- cat /etc/ntp.conf | grep -v 'out: #'
[envrm02] Executing task ''
[envrm02] run: cat /etc/ntp.conf
[envrm02] out:
[envrm02] out: server 0.freebsd.pool.ntp.org iburst
[envrm02] out: server 1.freebsd.pool.ntp.org iburst
[envrm02] out: server 2.freebsd.pool.ntp.org iburst
[envrm02] out:
[envrm02] out: restrict default limited kod nomodify notrap nopeer noquery
[envrm02] out: restrict -6 default limited kod nomodify notrap nopeer noquery
[envrm02] out: restrict 127.0.0.1
[envrm02] out: restrict -6 ::1
[envrm02] out: restrict 127.127.1.0

[og-qa] Executing task ''
[og-qa] run: cat /etc/ntp.conf
[og-qa] out:
[og-qa] out: server 0.freebsd.pool.ntp.org iburst
[og-qa] out: server 1.freebsd.pool.ntp.org iburst
[og-qa] out: server 2.freebsd.pool.ntp.org iburst
[og-qa] out:
[og-qa] out: restrict default limited kod nomodify notrap nopeer noquery
[og-qa] out: restrict -6 default limited kod nomodify notrap nopeer noquery
[og-qa] out: restrict 127.0.0.1
[og-qa] out: restrict -6 ::1
[og-qa] out: restrict 127.127.1.0

[og-ci] Executing task ''
[og-ci] run: cat /etc/ntp.conf
[og-ci] out:
[og-ci] out: server 0.freebsd.pool.ntp.org iburst
[og-ci] out: server 1.freebsd.pool.ntp.org iburst
[og-ci] out: server 2.freebsd.pool.ntp.org iburst
[og-ci] out:
[og-ci] out: restrict default limited kod nomodify notrap nopeer noquery
[og-ci] out: restrict -6 default limited kod nomodify notrap nopeer noquery
[og-ci] out: restrict 127.0.0.1
[og-ci] out: restrict -6 ::1
[og-ci] out: restrict 127.127.1.0

[og-test] Executing task ''
[og-test] run: cat /etc/ntp.conf
[og-test] out:
[og-test] out: server 0.freebsd.pool.ntp.org iburst
[og-test] out: server 1.freebsd.pool.ntp.org iburst
[og-test] out: server 2.freebsd.pool.ntp.org iburst
[og-test] out:
[og-test] out: restrict default limited kod nomodify notrap nopeer noquery
[og-test] out: restrict -6 default limited kod nomodify notrap nopeer noquery
[og-test] out: restrict 127.0.0.1
[og-test] out: restrict -6 ::1
[og-test] out: restrict 127.127.1.0

Some observation:
* The clock of envrm02 (the HOST) always seems to be correct.
* The issue happens on a (apparently) random guest (not always the same one).
* The issue happens always on one host at a time.

The bhyve command line is, roughly, this one:

/usr/sbin/bhyveload -m 2048 -d /san_storage/VMfs/$machine/$machine.img $machine
/usr/sbin/bhyve -c 2 -m 2048 -A -H -P -s 0,hostbridge -s
2,ahci-hd,/san_storage/VMfs/$machine/$machine.img -s 4,virtio-net,$tap
-s 31,lpc -l com1,stdio $machine

Thanks in advance.
_