Re: nested virtualization with vmm on hyper-v
On Sat, Oct 30, 2021 at 08:56:53PM -0700, Mike Larkin wrote: [cut] > I told Microsoft years ago that their implementation of legacy event injection > was broken. This is how we inject interrupts in vmm(4). They either didn't > understand, or didn't care. Since hyper-v doesn't deliver our injected event, > no interrupts are delivered to the VM and as soon as interrupts are enabled > in autoconf, you hang. > > Note: KVM and VMware do it correctly. > > -ml > Hi Mike, Thanks for the explanation. I guess I can pack up the configs, for this host. I have a laptop that can do vmm on native OpenBSD, but it's not always on. Best Regards, -peter
Re: nested virtualization with vmm on hyper-v
On Sat, Oct 30, 2021 at 02:25:44PM +0200, Peter J. Philipp wrote: > Hi, > > I'm a little confused by this. I have an old Xeon e3-1275v3 computer that > runs windows server 2019 essentials natively. It shares half it's RAM and > 2 cores with OpenBSD as a hyper-v guest. > > So after a long time of not looking into this I asked someone how they did > virtualization nesting and I got this link: > > https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/nested-virtualization > > So this is good, I thought, I followed the instructions, turned off the > hyper-v, > did the powershell command, restarted hyper-v guest and got some indication by > OpenBSD that vmm is enabled: > > arda$ dmesg | grep vmm > cpu0: using VERW MDS workaround (except on vmm entry) > vmm0 at mainbus0: VMX/EPT > > Only when I start the vmm guest with vmctl and vmctl console, it boots me out > of console mode back into arda and the vmm guest is stopped. > > So then I rebooted the entire machine to make sure that all BIOS settings are > correct. And they are. There is no CPU option that I can set here that would > disallow this nesting. > > Here is the CPU options of arda (short of a dmesg): > > cpu0: using VERW MDS workaround (except on vmm entry) > cpu0 at mainbus0: apid 0 (boot processor) > cpu0: Intel(R) Xeon(R) CPU E3-1275 v3 @ 3.50GHz, 3498.01 MHz, 06-3c-03 > cpu0: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,HTT,SSE3,PCLMUL,VMX,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,XSAVEOPT,MELTDOWN > cpu0: 256KB 64b/line 8-way L2 cache > cpu0: smt 0, core 0, package 0 > cpu0: apic clock running at 200MHz > cpu1 at mainbus0: apid 1 (application processor) > cpu1: Intel(R) Xeon(R) CPU E3-1275 v3 @ 3.50GHz, 3497.99 MHz, 06-3c-03 > cpu1: > FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,HTT,SSE3,PCLMUL,VMX,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,XSAVEOPT,MELTDOWN > cpu1: 256KB 64b/line 8-way L2 cache > cpu1: smt 0, core 1, package 0 > cpu0: using VERW MDS workaround (except on vmm entry) > > Does anyone know how I can get vmm with nesting to function? Is it a OpenBSD > problem or should I seek elsewhere? > > Best Regards, > -peter > I told Microsoft years ago that their implementation of legacy event injection was broken. This is how we inject interrupts in vmm(4). They either didn't understand, or didn't care. Since hyper-v doesn't deliver our injected event, no interrupts are delivered to the VM and as soon as interrupts are enabled in autoconf, you hang. Note: KVM and VMware do it correctly. -ml
nested virtualization with vmm on hyper-v
Hi, I'm a little confused by this. I have an old Xeon e3-1275v3 computer that runs windows server 2019 essentials natively. It shares half it's RAM and 2 cores with OpenBSD as a hyper-v guest. So after a long time of not looking into this I asked someone how they did virtualization nesting and I got this link: https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/nested-virtualization So this is good, I thought, I followed the instructions, turned off the hyper-v, did the powershell command, restarted hyper-v guest and got some indication by OpenBSD that vmm is enabled: arda$ dmesg | grep vmm cpu0: using VERW MDS workaround (except on vmm entry) vmm0 at mainbus0: VMX/EPT Only when I start the vmm guest with vmctl and vmctl console, it boots me out of console mode back into arda and the vmm guest is stopped. So then I rebooted the entire machine to make sure that all BIOS settings are correct. And they are. There is no CPU option that I can set here that would disallow this nesting. Here is the CPU options of arda (short of a dmesg): cpu0: using VERW MDS workaround (except on vmm entry) cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Xeon(R) CPU E3-1275 v3 @ 3.50GHz, 3498.01 MHz, 06-3c-03 cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,HTT,SSE3,PCLMUL,VMX,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,XSAVEOPT,MELTDOWN cpu0: 256KB 64b/line 8-way L2 cache cpu0: smt 0, core 0, package 0 cpu0: apic clock running at 200MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Xeon(R) CPU E3-1275 v3 @ 3.50GHz, 3497.99 MHz, 06-3c-03 cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUSH,MMX,FXSR,SSE,SSE2,SS,HTT,SSE3,PCLMUL,VMX,SSSE3,FMA3,CX16,PCID,SSE4.1,SSE4.2,MOVBE,POPCNT,AES,XSAVE,AVX,F16C,RDRAND,HV,NXE,PAGE1GB,RDTSCP,LONG,LAHF,ABM,FSGSBASE,BMI1,AVX2,SMEP,BMI2,ERMS,INVPCID,MD_CLEAR,IBRS,IBPB,STIBP,L1DF,SSBD,XSAVEOPT,MELTDOWN cpu1: 256KB 64b/line 8-way L2 cache cpu1: smt 0, core 1, package 0 cpu0: using VERW MDS workaround (except on vmm entry) Does anyone know how I can get vmm with nesting to function? Is it a OpenBSD problem or should I seek elsewhere? Best Regards, -peter
