On Wed 2021-12-01 09:52 AM MST -0700, <000.f...@quip.cz> wrote:
On 01/12/2021 17:17, John Doherty via freebsd-virtualization wrote:
That limitation appears to still exist in FreeBSD 13.0-RELEASE:
[root@grit] # freebsd-version -k ; grep 'VM_MAXCPU'
/usr/src/sys/amd64/include/vmm.h
13.0-RELEASE
#define VM_MAXCPU 16 /* maximum
virtual cpus */
I ran into this in May 2021 and with some help from folks on this
list was able to increase it. The simplest (if not minimalist) way to
do that is:
1. edit /usr/src/sys/amd64/include/vmm.h to increase that value: I
used 48
2. make buildworld
3. make installworld
The increased value has been working fine for me since I did that. I
run a couple of VMs with 24 vCPUs each and several others with
smaller numbers all the time and have run others with as many as 48
temporarily. No problems that I have seen.
I am sorry for hijacking this thread but your information is very
interesting. I was playing with VMs in VirtualBox and Bhyve and
compared performance with increasing vCPU count. The more cores VM get
the slower was even a simple single threaded task like loading PF
rules from /etc/pf.conf. It was tested on FreeBSD 11.4 and 12.2, I
tested ULE and 4BSD schedulers. Maybe it was somewhat HW related but
it always shows VMs with more than 2 v CPUs significantly slower. VMs
with 6+ vCPU was almost unusable (loading of PF ruleset takes about 8
seconds instead of fraction on single vCPU VM).
Do you have any special tunning to have so large number of vCPU
without this penalty?
I did not do anything special other than the steps described above. I
did do some other stuff while sort of stumbling toward the eventual
solution, but that's neither here nor there anymore. The steps above are
what I used to build the primary system where I use bhyve. The physical
host has two Xeon E5-2690 v4 CPUs, 14 cores/28 threads each so 28
cores/56 threads total.
I have not seen but neither have I tried to measure any problems like
you describe. bhyve works very well for me and I especially like it in
combination with the vm-bhyve package, which I'm using to manage the
VMs.
Of the various virtualization systems I've used or tried over the years,
I like this combination more than any other. It's simple, clean,
integrates well with ZFS, and is a pleasure to use, at least by my
lights. Haven't had any trouble either before or after increasing the
VM_MAXCPU value.
