Dave Voutila <d...@sisu.io> writes:
> Dave Voutila <d...@sisu.io> writes:
>
>> di...@santanas.co.za writes:
>>
>>> Hi OpenBSD friends,
>>>
>>> Just a report, not sure if it's helpful, but @voutilad requested [1] I
>>> send the details to the mailing list.
>>>
>>> I have seen a few reports online[1][2], about some users not being able to
>>> boot newer alpine linux versions (and other linux OS' in my
>>> experience). Specifically I've seen the last version that boots is
>>> 3.5.3.
>>>
>>> My system is openbsd 7.2 on my hardware Lenovo ThinkPad E14 Gen 4
>>> laptop.
>>>
>>> The issue, when the alpine linux VM boots, it kernel panics.
>>>
>>> [ 0.052602] local IPI:
>>> [ 0.052602] invalid opcode: 0000 [#1] SMP PTI
>>> [ 0.052602] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.15.79-0-virt
>>> #1-Alpine
>>> [ 0.052602] Hardware name: OpenBSD VMM, BIOS 1.14.0p0-OpenBSD-vmm
>>> 01/01/2011
>>> [ 0.052602] RIP: 0010:delay_halt_tpause+0xd/0x20
>>> [ 0.052602] Code: 75 fb 48 ff c8 31 c0 31 ff c3 cc cc cc cc 66 66 2e
>> 0f 1f 84 00 00 00 00 00 0f 1f 40 00 48 8d 04 37 31 c9 48 89 c2 48 c1
>> ea 20 <66> 0f ae f1 31 c0 31 d2 31 c9 31 f6 31 ff c3 cc cc cc cc 53 48
>> c7
>>
>> The key issue here is the invalid opcode error coming from the
>> instruction starting with 66 0f ae, which is a TPAUSE
>> instruction. Hence the RIP pointing to "delay_halt_tpause" in the Linux
>> kernel.
>>
>> I don't think I have any newer Intel hardware that supports the "User
>> Wait" instructions (aka WAITPKG). My Intel research says it premiered in
>> Tremont, Alder Lake, Sapphire Rapids so I can't test locally, but the
>> docs from Intel (SDM Vol. 2B 4-719) say:
>>
>> Prior to executing the TPAUSE instruction, an operating system may
>> specify the maximum delay it allows the processor to suspend its
>> operation. It can do so by writing TSC-quanta value to the
>> following 32-bit MSR (IA32_UMWAIT_CONTROL at MSR index E1H)...
>>
>> We probably should be masking the CPUID value for TPAUSE in the values
>> vmm(4) communicates via vmm_handle_cpuid.
>>
>>
>
> The below diff defines the cpuid bit for detecting the WAITPKG
> feature. It adds the value to vmm's cpuid mask and also updates the
> i386/amd64 cpu identification info.
>
> Can someone with a newer Intel system try this out?
>
>
> diff refs/heads/master refs/heads/vmm-tsleep
> commit - 515b7b0d87d9ff8cd5eae1449555f3d6e625fa49
> commit + 6343cff9c1cfbbf9ba2cb06cfeca507caa06fc8c
> blob - 001a437045be145322be30288c1f47d63fb07634
> blob + 0bd908e273a1c0e6324e1bc9f8c8ca921555c86f
> --- sys/arch/amd64/amd64/identcpu.c
> +++ sys/arch/amd64/amd64/identcpu.c
> @@ -208,6 +208,7 @@ const struct {
> { SEFF0ECX_AVX512VBMI, "AVX512VBMI" },
> { SEFF0ECX_UMIP, "UMIP" },
> { SEFF0ECX_PKU, "PKU" },
> + { SEFF0ECX_WAITPKG, "WAITPKG" },
> }, cpu_seff0_edxfeatures[] = {
> { SEFF0EDX_AVX512_4FNNIW, "AVX512FNNIW" },
> { SEFF0EDX_AVX512_4FMAPS, "AVX512FMAPS" },
> blob - cbde6cf9b02fc882a8ed17aa6adb5c43249e0302
> blob + b26bd32e2d9ea7386b1f58960dea40b787d6a341
> --- sys/arch/amd64/include/specialreg.h
> +++ sys/arch/amd64/include/specialreg.h
> @@ -201,6 +201,7 @@
> #define SEFF0ECX_AVX512VBMI 0x00000002 /* AVX-512 vector bit inst */
> #define SEFF0ECX_UMIP 0x00000004 /* UMIP support */
> #define SEFF0ECX_PKU 0x00000008 /* Page prot keys for user mode */
> +#define SEFF0ECX_WAITPKG 0x00000010 /* UMONITOR/UMWAIT/TPAUSE insns */
> /* SEFF EDX bits */
> #define SEFF0EDX_AVX512_4FNNIW 0x00000004 /* AVX-512 neural network
> insns */
> #define SEFF0EDX_AVX512_4FMAPS 0x00000008 /* AVX-512 mult accum single
> prec */
> blob - 6b4802abf4b508495cdbc961bd799d3fa83b9c36
> blob + bbe10bd4cfd7e778132eca1d97594e10513ac172
> --- sys/arch/amd64/include/vmmvar.h
> +++ sys/arch/amd64/include/vmmvar.h
> @@ -672,7 +672,12 @@ struct vm_mprotect_ept_params {
> SEFF0EBX_AVX512IFMA | SEFF0EBX_AVX512PF | \
> SEFF0EBX_AVX512ER | SEFF0EBX_AVX512CD | \
> SEFF0EBX_AVX512BW | SEFF0EBX_AVX512VL)
> -#define VMM_SEFF0ECX_MASK ~(SEFF0ECX_AVX512VBMI)
> +/*
> + * Copy from host minus:
> + * AVX-512 vector bit (SEFF0ECX_AVX512VBMI)
> + * UMONITOR/UMWAIT/TPAUSE (SEFF0ECX_WAITPKG)
> + */
> +#define VMM_SEFF0ECX_MASK ~(SEFF0ECX_AVX512VBMI | SEFF0ECX_WAITPKG)
>
> /* EDX mask contains the bits to include */
> #define VMM_SEFF0EDX_MASK (SEFF0EDX_MD_CLEAR)
> blob - 310208ac4cdb262aaedfa9b78d869fd5911607b2
> blob + ccf1164fd658a69dc383e1602ae0ce1f269de4e4
> --- sys/arch/i386/i386/machdep.c
> +++ sys/arch/i386/i386/machdep.c
> @@ -1038,6 +1038,7 @@ const struct cpu_cpuid_feature cpu_seff0_ecxfeatures[]
> { SEFF0ECX_UMIP, "UMIP" },
> { SEFF0ECX_AVX512VBMI, "AVX512VBMI" },
> { SEFF0ECX_PKU, "PKU" },
> + { SEFF0ECX_WAITPKG, "WAITPKG" },
> };
>
> const struct cpu_cpuid_feature cpu_seff0_edxfeatures[] = {
> blob - 392b4ff412e2dd3c4c48ed6c9c84aa2358721c6a
> blob + 7ce77ca3fdc6bd1a51571dd0b5dbf5afc311a138
> --- sys/arch/i386/include/specialreg.h
> +++ sys/arch/i386/include/specialreg.h
> @@ -190,6 +190,7 @@
> #define SEFF0ECX_AVX512VBMI 0x00000002 /* AVX-512 vector bit inst */
> #define SEFF0ECX_UMIP 0x00000004 /* UMIP support */
> #define SEFF0ECX_PKU 0x00000008 /* Page prot keys for user mode */
> +#define SEFF0ECX_WAITPKG 0x00000010 /* UMONITOR/UMWAIT/TPAUSE insns */
> /* SEFF EDX bits */
> #define SEFF0EDX_AVX512_4FNNIW 0x00000004 /* AVX-512 neural network
> insns */
> #define SEFF0EDX_AVX512_4FMAPS 0x00000008 /* AVX-512 mult accum single
> prec */
I can confirm with these patches that hit 7.2 this issue is resolved for
me on this newer intel hardware.
72-015_vmm
72-016_vmd
Thank you so much!
