Re: [bhyve][arm64] Number of allowed vcpus
On Wed, 2021-01-13 at 17:00 +0200, Mihai Carabas wrote: > We are talking here about the device tree of the guest. If you boot > up a guest with a device tree of X vcpus, why specify something else > on the command line? Because typing a command line arg is a lot easier than recompiling a device tree. IMO, these sorts of situations should always be thought of in terms of "the source of info closest to the current/interactive user rules". In device drivers I often code logic to get paramters first from devicetree data, then it can be overriden with device hints, and then that can be overridden by sysctl or tunable info (when all 3 apply). -- Ian ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: host, bhyve vm and ntpd
On Wed, 2017-10-25 at 00:43 +0300, Boris Samorodov wrote: > Hi Ian, All! > > 22.10.2017 01:15, Ian Lepore пишет: > > > > On Sat, 2017-10-21 at 17:07 -0400, Michael Voorhis wrote: > > > > > > Ian Lepore writes: > > > > > > > > > > > > Beyond that, I'm not sure what else to try. It might be necessary to > > > > get some bhyve developers involved (I know almost nothing about it). > > > NTPD behaves more normally on uniprocessor VMs. > > > > > > A FreeBSD bhyve-guest running on a freebsd host will select a > > > different timecounter depending on whether it is a multiprocessor or a > > > uniprocessor. My uniprocessor bhyve-vm selected TSC-low as the best > > > timecounter in a uniprocessor. NTP functions there as expected. > > > > > > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) > > > dummy(-100) > > > kern.timecounter.hardware: TSC-low > > > > > > The very same VM, when given two total CPUs, selected HPET (if I > > > recall) and the timekeeping with NTPD was unreliable, with many > > > step-resets to the clock. > > > > > Hmm, I just had glance at the code in sys/amd64/vmm/io/vhpet.c and it > > looks right. I wonder if this is just a simple roundoff error in > > converting between 10.0MHz and SBT units? If so, that could be wished > > away easily by using a power-of-2 frequency for the virtual HPET. I > > wonder if the attached patch is all that's needed? > I suppose the answer is "yes", the patch helped. Here are two samples > (host for bhyve VM without your patch and after patching): > --- > https://poudriere.passap.ru/misc/ntpd.jot.log-HPET.frequency.1000.txt > https://poudriere.passap.ru/misc/ntpd.jot.log-HPET.frequency.16777216.txt > --- > > The command was: > % for t in `jot 1000`; do ntpq -pn; sleep 64; done > > The patch made the system to stabilize the process. > Ian, thank you! > Hmmm. The startup behavior wasn't great, it took a long time and several clock steps for it to figure out the frequency error and get the clock under control. But, as you say, it did eventually stabilize this time. Can you show the /var/db/ntpd.drift file contents of the host and guest? Ideally, now that it's stable, the two values should be very close. If they're not, maybe this isn't the right fix. -- Ian ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: host, bhyve vm and ntpd
On Sun, 2017-10-22 at 11:31 +0300, Boris Samorodov wrote: > 22.10.2017 01:15, Ian Lepore пишет: > > > > On Sat, 2017-10-21 at 17:07 -0400, Michael Voorhis wrote: > > > > > > Ian Lepore writes: > > > > > > > > > > > > Beyond that, I'm not sure what else to try. It might be necessary to > > > > get some bhyve developers involved (I know almost nothing about it). > > > NTPD behaves more normally on uniprocessor VMs. > > > > > > A FreeBSD bhyve-guest running on a freebsd host will select a > > > different timecounter depending on whether it is a multiprocessor or a > > > uniprocessor. My uniprocessor bhyve-vm selected TSC-low as the best > > > timecounter in a uniprocessor. NTP functions there as expected. > > > > > > kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0) > > > dummy(-100) > > > kern.timecounter.hardware: TSC-low > > > > > > The very same VM, when given two total CPUs, selected HPET (if I > > > recall) and the timekeeping with NTPD was unreliable, with many > > > step-resets to the clock. > > > > > Hmm, I just had glance at the code in sys/amd64/vmm/io/vhpet.c and it > > looks right. I wonder if this is just a simple roundoff error in > > converting between 10.0MHz and SBT units? If so, that could be wished > > away easily by using a power-of-2 frequency for the virtual HPET. I > > wonder if the attached patch is all that's needed? > I've tried the patch (at bhyve guest) and nothing has changed. Should > the patched system be tested at bhyve guest or bhyve host? > Oh, I'm sorry, I should have mentioned that's for the host side. -- Ian ___ freebsd-virtualization@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization To unsubscribe, send any mail to "freebsd-virtualization-unsubscr...@freebsd.org"
Re: host, bhyve vm and ntpd
On Sat, 2017-10-21 at 17:07 -0400, Michael Voorhis wrote:
> Ian Lepore writes:
> >
> > Beyond that, I'm not sure what else to try. It might be necessary to
> > get some bhyve developers involved (I know almost nothing about it).
> NTPD behaves more normally on uniprocessor VMs.
>
> A FreeBSD bhyve-guest running on a freebsd host will select a
> different timecounter depending on whether it is a multiprocessor or a
> uniprocessor. My uniprocessor bhyve-vm selected TSC-low as the best
> timecounter in a uniprocessor. NTP functions there as expected.
>
> kern.timecounter.choice: TSC-low(1000) ACPI-fast(900) HPET(950) i8254(0)
> dummy(-100)
> kern.timecounter.hardware: TSC-low
>
> The very same VM, when given two total CPUs, selected HPET (if I
> recall) and the timekeeping with NTPD was unreliable, with many
> step-resets to the clock.
>
Hmm, I just had glance at the code in sys/amd64/vmm/io/vhpet.c and it
looks right. I wonder if this is just a simple roundoff error in
converting between 10.0MHz and SBT units? If so, that could be wished
away easily by using a power-of-2 frequency for the virtual HPET. I
wonder if the attached patch is all that's needed?
-- Ian
Index: sys/amd64/vmm/io/vhpet.c
===
--- sys/amd64/vmm/io/vhpet.c (revision 324176)
+++ sys/amd64/vmm/io/vhpet.c (working copy)
@@ -51,7 +51,7 @@ __FBSDID("$FreeBSD$");
static MALLOC_DEFINE(M_VHPET, "vhpet", "bhyve virtual hpet");
-#define HPET_FREQ 1000 /* 10.0 Mhz */
+#define HPET_FREQ 16777216 /* 16.7 (2^24) Mhz */
#define FS_PER_S 1000ul
/* Timer N Configuration and Capabilities Register */
___
freebsd-virtualization@freebsd.org mailing list
https://lists.freebsd.org/mailman/listinfo/freebsd-virtualization
To unsubscribe, send any mail to
"freebsd-virtualization-unsubscr...@freebsd.org"
Re: host, bhyve vm and ntpd
On Fri, 2017-10-20 at 21:15 +0300, Boris Samorodov wrote: > 20.10.2017 21:04, Ian Lepore пишет: > > > > On Fri, 2017-10-20 at 20:20 +0300, Boris Samorodov wrote: > > > > > > (CC to freebsd-virtualization@) > > > > > > 20.10.2017 19:32, Ian Lepore пишет: > > > > > > > > > > > > On Fri, 2017-10-20 at 18:36 +0300, Boris Samorodov wrote: > > > > > > > > > > > > > > > 20.10.2017 18:31, Boris Samorodov пишет: > > > > > > > > > > > > > > > > > > > > > > > > 20.10.2017 18:12, Ian Lepore пишет: > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 2017-10-20 at 14:46 +0300, Boris Samorodov wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > I have got a host: > > > > > > > > --- > > > > > > > > bhyve-host% uname -a > > > > > > > > FreeBSD sm.bsnet 12.0-CURRENT FreeBSD 12.0-CURRENT #3 r322868: > > > > > > > > Fri Aug > > > > > > > > 25 05:25:26 MSK 2017 > > > > > > > > bsam@builder.bsnet:/usr/obj/usr/src/sys/GENERIC-FAST amd64 > > > > > > > > amd64 > > > > > > > > --- > > > > > > > > > > > > > > > > And a bhyve vm: > > > > > > > > --- > > > > > > > > bhyve-vm: uname -a > > > > > > > > FreeBSD builder.bsnet 12.0-CURRENT FreeBSD 12.0-CURRENT #58 > > > > > > > > r324782: Fri > > > > > > > > Oct 20 05:12:17 MSK 2017 > > > > > > > > bsam@builder.bsnet:/usr/obj/usr/src/sys/PKG64X amd64 amd64 > > > > > > > > --- > > > > > > > > > > > > > > > > The only difference at kernel configs is a colored console. :-) > > > > > > > > > > > > > > > > And here I get some weird (is it?) result at the VM (I expect > > > > > > > > ntpd to be > > > > > > > > more stable): > > > > > > > > --- > > > > > > > > bhyve-vm% for t in `jot 10`; do ntpq -pn; sleep 64; done > > > > > > > > remote refid st t when poll reach delay > > > > > > > > offset > > > > > > > > jitter > > > > > > > > == > > > > > > > > XX.XX.XX.1 XX.XX.XX.245 4 u9 6430.605 > > > > > > > > -1.202 > > > > > > > > 316.407 > > > > > > > > XX.XX.XX.1 XX.XX.XX.245 4 u7 6470.605 > > > > > > > > -1.202 > > > > > > > > 358.395 > > > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u5 64 170.615 > > > > > > > > -328.42 > > > > > > > > 181.405 > > > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u3 64 370.615 > > > > > > > > -328.42 > > > > > > > > 214.868 > > > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u 67 64 370.615 > > > > > > > > -328.42 > > > > > > > > 214.868 > > > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u 63 64 770.615 > > > > > > > > -328.42 > > > > > > > > 268.618 > > > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u 60 64 1770.615 > > > > > > > > -328.42 > > > > > > > > 333.175 > > > > > > > > XX.XX.XX.1 .STEP. 16 u 1910 6400.000 > > > > > > > > 0.000 > > > > > > > > 0.000 > > > > > > > > XX.XX.XX.1 XX.XX.XX.245 4 u 27 6410.703 > > > > > > > > -262.63 > > > > > > > > 0.004 > > > > > > > > XX.XX.XX.1 XX.XX.XX.245 4 u
Re: host, bhyve vm and ntpd
On Fri, 2017-10-20 at 20:20 +0300, Boris Samorodov wrote: > (CC to freebsd-virtualization@) > > 20.10.2017 19:32, Ian Lepore пишет: > > > > On Fri, 2017-10-20 at 18:36 +0300, Boris Samorodov wrote: > > > > > > 20.10.2017 18:31, Boris Samorodov пишет: > > > > > > > > > > > > 20.10.2017 18:12, Ian Lepore пишет: > > > > > > > > > > > > > > > On Fri, 2017-10-20 at 14:46 +0300, Boris Samorodov wrote: > > > > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > I have got a host: > > > > > > --- > > > > > > bhyve-host% uname -a > > > > > > FreeBSD sm.bsnet 12.0-CURRENT FreeBSD 12.0-CURRENT #3 r322868: Fri > > > > > > Aug > > > > > > 25 05:25:26 MSK 2017 > > > > > > bsam@builder.bsnet:/usr/obj/usr/src/sys/GENERIC-FAST amd64 amd64 > > > > > > --- > > > > > > > > > > > > And a bhyve vm: > > > > > > --- > > > > > > bhyve-vm: uname -a > > > > > > FreeBSD builder.bsnet 12.0-CURRENT FreeBSD 12.0-CURRENT #58 > > > > > > r324782: Fri > > > > > > Oct 20 05:12:17 MSK 2017 > > > > > > bsam@builder.bsnet:/usr/obj/usr/src/sys/PKG64X amd64 amd64 > > > > > > --- > > > > > > > > > > > > The only difference at kernel configs is a colored console. :-) > > > > > > > > > > > > And here I get some weird (is it?) result at the VM (I expect ntpd > > > > > > to be > > > > > > more stable): > > > > > > --- > > > > > > bhyve-vm% for t in `jot 10`; do ntpq -pn; sleep 64; done > > > > > > remote refid st t when poll reach delay > > > > > > offset > > > > > > jitter > > > > > > == > > > > > > XX.XX.XX.1 XX.XX.XX.245 4 u9 6430.605 > > > > > > -1.202 > > > > > > 316.407 > > > > > > XX.XX.XX.1 XX.XX.XX.245 4 u7 6470.605 > > > > > > -1.202 > > > > > > 358.395 > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u5 64 170.615 > > > > > > -328.42 > > > > > > 181.405 > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u3 64 370.615 > > > > > > -328.42 > > > > > > 214.868 > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u 67 64 370.615 > > > > > > -328.42 > > > > > > 214.868 > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u 63 64 770.615 > > > > > > -328.42 > > > > > > 268.618 > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u 60 64 1770.615 > > > > > > -328.42 > > > > > > 333.175 > > > > > > XX.XX.XX.1 .STEP. 16 u 1910 6400.000 > > > > > > 0.000 > > > > > > 0.000 > > > > > > XX.XX.XX.1 XX.XX.XX.245 4 u 27 6410.703 > > > > > > -262.63 > > > > > > 0.004 > > > > > > XX.XX.XX.1 XX.XX.XX.245 4 u 31 6410.649 > > > > > > -331.43 > > > > > > 68.800 > > > > > > --- > > > > > > > > > > > > At the same time host's results are very stable: > > > > > > --- > > > > > > bhyve-host% for t in `jot 10`; do ntpq -pn; sleep 64; done > > > > > > remote refid st t when poll reach delay > > > > > > offset > > > > > > jitter > > > > > > == > > > > > > > > > > > > > > > > > > > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u1 6410.401 > > > > > > 0.176 > > > > > > 0.106 > > > > > > *XX.XX.XX.1 XX.XX.XX.245 4 u6 6430.401 > > > > > > 0.176 > > > > > > 0.459 > > > >
