[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
During the inspection of Apple reference, I have noticed that Guest CR0 and CR0 Guest/Host Mask has incorrect value. Apple defines that Guest CR0 is writable only if: CR0.CD and CR0.NW are unset But hvf accel code follows Intel SDM "Table 9-1. IA-32 and Intel 64 Processor States Following Power-up, Reset, or INIT" and sets CR0 value to: 0x6010 Likewise, CR0 Guest/Host Mask is conditionally writable if: CR0.CD and CR0.NW are set I doubt if it's related to the HV_ERROR issue but I'll prepare a patch to fix both fields (and likely set CR0 Read Shadow). -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt] frame #4: 0x00010baa4c8f qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt] frame #5: 0x00010b988423 qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at cpus.c:1636 [opt] frame #6: 0x00010bd9dfce qemu-system-x86_64`qemu_thread_start(args=) at qemu-thread-posix.c:502 [opt] frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126 frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70 frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
It's not possible to allocate MSR bitmap in userspace because it requires a physical address to be stored in the VMCS field. However, the bitmap page is already allocated inside kernel part of Hypervisor.framework. The 4k bitmap region is aligned to page boundary. It's worth to continue inspection of the checks (26.2 CHECKS ON VMX CONTROLS AND HOST-STATE AREA). The reason why MSR Bitmap Address has weird value is because it's not necessarily the value of the VMCS field (albeit VMCS_CTRL_MSR_BITMAPS is defined in hv_arch_vmx.h). HVF uses an internal lookup table that has a limited set of VMCS fields exposed by Apple. The list is documented at the reference page: https://developer.apple.com/documentation/hypervisor/1469436-virtual_machine_control_structur It's likely that 0x3f is a field from the VMCS lookup table. Given the signature of hv_vmx_vcpu_read_vmcs, I would expect an error (e.g. HV_BAD_ARGUMENT) to be returned instead of the silent failure. I have submitted FB6858948 to Apple to correct the behaviour. So, Apple doesn't provide an explicit access to MSR Bitmap Address field but allows to control the bitmap via hv_vcpu_enable_native_msr. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt] frame #4: 0x00010baa4c8f qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt] frame #5: 0x00010b988423 qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at cpus.c:1636 [opt] frame #6: 0x00010bd9dfce qemu-system-x86_64`qemu_thread_start(args=) at qemu-thread-posix.c:502 [opt] frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126 frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70 frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
My guess is that RFLAGS.ZF == 1 and one or a few of the checks on VMX controls have failed. So far I have verified the following checks (26-2 and 26-3 in Intel SDM Vol. 3C): * Reserved bits in Pin-based VM execution controls are set according to associated capabilities MSR * Reserved bits in Primary Proc-based VM execution controls are set according to associated capabilities MSR * Reserved bits in Secondary Proc-based VM execution controls are set according to associated capabilities MSR * CR-3 target count is not greater than 4. (the count is 0) * Use I/O bitmaps check is not applicable because "use I/O bitmaps" VM-execution control is 0. * Reserved bits in VM-exit controls are set according to associated capabilities MSR * Reserved bits in VM-entry controls are set according to associated capabilities MSR However, the MSR-bitmap Address check might fail: "If the “use MSR bitmaps” VM-execution control is 1, bits 11:0 of the MSR-bitmap address must be 0. The address should not set any bits beyond the processor’s physical-address width." Bit 28 in Pin-based VM execution controls is set to 1 while the MSR address has bits 5:1 set to 1 (0x3f). There's no way to disable the "use MSR bitmaps" execution control so I'll try to make a patch that sets 4k- page aligned MSR bitmap address. Updated log lines show the VMX capabilities for the control fields and VMCS fields related to the failure: qemu-system-x86_64: hv_vcpu_run failed qemu-system-x86_64: exit reason:0x0030 qemu-system-x86_64: exit qualification: 0x0083 qemu-system-x86_64: instruction error: 0x0007 qemu-system-x86_64: VM-EXECUTION CONTROL FIELDS qemu-system-x86_64: Pin-Based VM-Execution Controls qemu-system-x86_64: pin based ctls: 0x003f qemu-system-x86_64: pin based caps: 0x007f003f qemu-system-x86_64: Processor-Based VM-Execution Controls qemu-system-x86_64: pri proc based ctls:0x95206dfa qemu-system-x86_64: pri proc based caps:0xfdf9fffe9500697a qemu-system-x86_64: sec proc based ctls:0x00a3 qemu-system-x86_64: sec proc based caps:0x00011cef00a2 qemu-system-x86_64: CR3-Target Controls qemu-system-x86_64: cr3 target count: 0x qemu-system-x86_64: MSR-Bitmap Address: 0x003f qemu-system-x86_64: VM-EXIT CONTROL FIELDS qemu-system-x86_64: VM-Exit Controls qemu-system-x86_64: vm exit ctls: 0x00236fff qemu-system-x86_64: vm exit caps: 0x00636fff00236fff qemu-system-x86_64: VM-ENTRY CONTROL FIELDS qemu-system-x86_64: VM-Entry Controls qemu-system-x86_64: vm entry ctls: 0x93ff qemu-system-x86_64: vm entry caps: 0x93ff91ff qemu-system-x86_64: Error: HV_ERROR -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt]
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
Hi Roman, thanks for the patch, we were able to reproduce this issue with our custom Android Cuttlefish based d VM (running 4.14 kernel): 2019-07-23T11:36:37.180753Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.8001H:ECX.svm [bit 2] 2019-07-23T11:36:37.182517Z qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.8001H:ECX.svm [bit 2] 2019-07-23T11:37:54.647855Z qemu-system-x86_64: hv_vcpu_run failed 2019-07-23T11:37:54.650737Z qemu-system-x86_64: exit reason: 0x0030 2019-07-23T11:37:54.661753Z qemu-system-x86_64: exit qualification: 0x0981 2019-07-23T11:37:54.661769Z qemu-system-x86_64: instruction error: 0x0007 2019-07-23T11:37:54.661780Z qemu-system-x86_64: pri proc based ctls: 0x95206dfa 2019-07-23T11:37:54.661790Z qemu-system-x86_64: sec proc based ctls: 0x00a3 2019-07-23T11:37:54.661799Z qemu-system-x86_64: eptp: 0x003f 2019-07-23T11:37:54.661810Z qemu-system-x86_64: gpa: 0x7fd05004 2019-07-23T11:37:54.661820Z qemu-system-x86_64: gla: 0xfe02f004 2019-07-23T11:37:54.661828Z qemu-system-x86_64: Error: HV_ERROR The error happened right at startup, after multiple tries. Thank you, Gergely -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt] frame #4: 0x00010baa4c8f qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt] frame #5: 0x00010b988423 qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at cpus.c:1636 [opt] frame #6: 0x00010bd9dfce qemu-system-x86_64`qemu_thread_start(args=) at qemu-thread-posix.c:502 [opt] frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126 frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70 frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
The log line I've got is the following: ➜ vms ~/dev/qemu/x86_64-softmmu/qemu-system-x86_64 -accel hvf -m 2G -cdrom ~/Downloads/ubuntu-18.04.2-desktop-amd64.iso -hda ubuntu.qc ow2 qemu-system-x86_64: warning: host doesn't support requested feature: CPUID.8001H:ECX.svm [bit 2] qemu-system-x86_64: hv_vcpu_run failed qemu-system-x86_64: instruction error: 0x0007 qemu-system-x86_64: exit reason:0x0030 qemu-system-x86_64: exit qualification: 0x0083 qemu-system-x86_64: pri proc based ctls:0x95206dfa qemu-system-x86_64: sec proc based ctls:0x00a3 qemu-system-x86_64: eptp: 0x003f qemu-system-x86_64: gpa:0x7d9ef000 qemu-system-x86_64: gla:0xfe000ec0 qemu-system-x86_64: Error: HV_ERROR Instruction error is 0x7 and Intel SDM 31-4 Vol. 3C states that: The processor checks on the VMX controls and host-state area. If any of these checks fail, the VM-entry instruction fails. RFLAGS.ZF is set to 1 and either 7 (VM entry with invalid control field(s)) or 8 (VM entry with invalid host-state field(s)) is saved in the VM-instruction error field. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt] frame #4: 0x00010baa4c8f qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt] frame #5: 0x00010b988423 qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at cpus.c:1636 [opt] frame #6: 0x00010bd9dfce qemu-system-x86_64`qemu_thread_start(args=) at qemu-thread-posix.c:502 [opt] frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126 frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70 frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
For the triage of the issue we need the following VMCS fields: * instruction error * exit reason * exit qualification On my machine (with macOS 10.14.5) each time QEMU exits with HV_ERROR, AppleHV spills the following error into system log: 2019-07-06 10:38:56.148547+0300 0x1e3ee4 Default 0x0 0 0kernel: (AppleHV) AppleHV: /BuildRoot/Library/Caches/com.apple.xbs/Sources/Hypervisor/Hypervisor-31.230.1/kext/x86/vmx/hv_vmx_vcpu.cpp : hv_return_t hv_vmx_vcpu_t::hv_vmx_vcpu_run() : 997 Such log lines can be read with the command: $ log show -predicate 'senderImagePath CONTAINS "AppleHV"' The error above can only happen if vmlaunch or vmresume has failed and RFLAGS has either CF or ZF (or both) set to 1, according to Intel SDM. Unfortunately the exact RFLAGS value is not logged by Hypervisor.framework. I have submitted a feedback report (FB6787376) to log RFLAGS if it's not zero immediately after vmlaunch/vmresume. If you wish to assist in debugging of the issue, please build and use QEMU from the branch: https://github.com/roolebo/qemu/tree/debug-hv-error Or apply the patch to your tree: https://github.com/roolebo/qemu/commit/f8098782573a89fc323d8dcae2d5445335e626f0.diff -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt] frame #4: 0x00010baa4c8f qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt] frame #5: 0x00010b988423 qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at cpus.c:1636 [opt] frame #6: 0x00010bd9dfce qemu-system-x86_64`qemu_thread_start(args=) at qemu-thread-posix.c:502 [opt] frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126 frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70 frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
We can reproduce this problem with Linux guests as well (running 4.15 Ubuntu Xenial and 4.14 Android kernels). Mac models with integrated GPU seem to be more affected according to our testing, and the crash does not always occur, needs multiple tries to be triggered. We would be happy to assist in debugging, once you have a patch that can generate more detailed logs. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt] frame #4: 0x00010baa4c8f qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt] frame #5: 0x00010b988423 qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at cpus.c:1636 [opt] frame #6: 0x00010bd9dfce qemu-system-x86_64`qemu_thread_start(args=) at qemu-thread-posix.c:502 [opt] frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126 frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70 frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
I'm looking into the issue... HV_ERROR is a high-level return value and doesn't give enough details about the nature of the error. The error is returned from vmexit handler in AppleHV.kext (which implements kernel part of Hypervisor.framework). Perhaps we should extract more data from the VMCS and print it before aborting the execution. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt] frame #4: 0x00010baa4c8f qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt] frame #5: 0x00010b988423 qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at cpus.c:1636 [opt] frame #6: 0x00010bd9dfce qemu-system-x86_64`qemu_thread_start(args=) at qemu-thread-posix.c:502 [opt] frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126 frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70 frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
^This is on version: % qemu-system-x86_64 --version QEMU emulator version 4.0.50 (v4.0.0-rc4-52-g3284aa1281-dirty) Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1818937 Title: Crash with HV_ERROR on macOS host Status in QEMU: New Bug description: On macOS host running Windows 10 guest, qemu crashed with error message: Error: HV_ERROR. Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5 4278U. QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560 QEMU parameter: qemu-system-x86_64 -m 3000 -drive file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3 thread list Process 56054 stopped thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16 + 970, queue = 'com.apple.main-thread' thread #2: tid = 0x2ffecc, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #3: tid = 0x2ffecd, 0x7fff79d715aa libsystem_kernel.dylib`__select + 10 thread #4: tid = 0x2ffece, 0x7fff79d71d9a libsystem_kernel.dylib`__sigwait + 10 * thread #6: tid = 0x2ffed0, 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT thread #7: tid = 0x2ffed1, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #8: tid = 0x2ffed2, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 thread #11: tid = 0x2fff34, 0x7fff79d6a17a libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread' thread #30: tid = 0x300c04, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #31: tid = 0x300c16, 0x7fff79e233f8 libsystem_pthread.dylib`start_wqthread thread #32: tid = 0x300c17, 0x thread #33: tid = 0x300c93, 0x7fff79d6d7de libsystem_kernel.dylib`__psynch_cvwait + 10 Crashed thread: * thread #6, stop reason = signal SIGABRT * frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10 frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285 frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127 frame #3: 0x00010baa476d qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt] frame #4: 0x00010baa4c8f qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt] frame #5: 0x00010b988423 qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at cpus.c:1636 [opt] frame #6: 0x00010bd9dfce qemu-system-x86_64`qemu_thread_start(args=) at qemu-thread-posix.c:502 [opt] frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126 frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70 frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
[Qemu-devel] [Bug 1818937] Re: Crash with HV_ERROR on macOS host
I can reproduce this by booting the Windows 10 x64 install ISO with the
command line:
+ WINIMG=Win10.iso
+ VIRTIMG=virtio-win-0.1.164.iso
+ qemu-system-x86_64 -accel hvf -drive driver=raw,file=Win10.img,if=virtio -m
1536 -net nic,model=virtio -net user -cdrom Win10.iso -drive
file=virtio-win-0.1.164.iso,index=3,media=cdrom -rtc base=localtime,clock=host
-smp cores=2 -usb -device usb-tablet -net user
qemu-system-x86_64: warning: host doesn't support requested feature:
CPUID.8001H:ECX.svm [bit 2]
qemu-system-x86_64: warning: host doesn't support requested feature:
CPUID.8001H:ECX.svm [bit 2]
Unimplemented handler (f80641601c38) for 0 (f 11)
Unimplemented handler (f8064160192f) for 0 (f 7f)
qemu-system-x86_64: Error: HV_ERROR
./qemu-boot.sh: line 20: 32294 Abort trap: 6 qemu-system-x86_64
-accel hvf -drive driver=raw,file=Win10.img,if=virtio -m 1536 -net
nic,model=virtio -net user -cdrom ${WINIMG} -drive
file=${VIRTIMG},index=3,media=cdrom -rtc base=localtime,clock=host -smp cores=2
-usb -device usb-tablet -net user
** Attachment added: "Crash log"
https://bugs.launchpad.net/qemu/+bug/1818937/+attachment/5260023/+files/crash_log.txt
--
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1818937
Title:
Crash with HV_ERROR on macOS host
Status in QEMU:
New
Bug description:
On macOS host running Windows 10 guest, qemu crashed with error
message: Error: HV_ERROR.
Host: macOS Mojave 10.14.3 (18D109) Late 2014 Mac mini presumably Core i5
4278U.
QEMU: git commit a3e3b0a7bd5de211a62cdf2d6c12b96d3c403560
QEMU parameter: qemu-system-x86_64 -m 3000 -drive
file=disk.img,if=virtio,discard=unmap -accel hvf -soundhw hda -smp 3
thread list
Process 56054 stopped
thread #1: tid = 0x2ffec8, 0x7fff48d0805a vImage`vLookupTable_Planar16
+ 970, queue = 'com.apple.main-thread'
thread #2: tid = 0x2ffecc, 0x7fff79d6d7de
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #3: tid = 0x2ffecd, 0x7fff79d715aa
libsystem_kernel.dylib`__select + 10
thread #4: tid = 0x2ffece, 0x7fff79d71d9a
libsystem_kernel.dylib`__sigwait + 10
* thread #6: tid = 0x2ffed0, 0x7fff79d7023e
libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGABRT
thread #7: tid = 0x2ffed1, 0x7fff79d6d7de
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #8: tid = 0x2ffed2, 0x7fff79d6d7de
libsystem_kernel.dylib`__psynch_cvwait + 10
thread #11: tid = 0x2fff34, 0x7fff79d6a17a
libsystem_kernel.dylib`mach_msg_trap + 10, name = 'com.apple.NSEventThread'
thread #30: tid = 0x300c04, 0x7fff79e233f8
libsystem_pthread.dylib`start_wqthread
thread #31: tid = 0x300c16, 0x7fff79e233f8
libsystem_pthread.dylib`start_wqthread
thread #32: tid = 0x300c17, 0x
thread #33: tid = 0x300c93, 0x7fff79d6d7de
libsystem_kernel.dylib`__psynch_cvwait + 10
Crashed thread:
* thread #6, stop reason = signal SIGABRT
* frame #0: 0x7fff79d7023e libsystem_kernel.dylib`__pthread_kill + 10
frame #1: 0x7fff79e26c1c libsystem_pthread.dylib`pthread_kill + 285
frame #2: 0x7fff79cd91c9 libsystem_c.dylib`abort + 127
frame #3: 0x00010baa476d
qemu-system-x86_64`assert_hvf_ok(ret=) at hvf.c:106 [opt]
frame #4: 0x00010baa4c8f
qemu-system-x86_64`hvf_vcpu_exec(cpu=0x7f8e5283de00) at hvf.c:681 [opt]
frame #5: 0x00010b988423
qemu-system-x86_64`qemu_hvf_cpu_thread_fn(arg=0x7f8e5283de00) at
cpus.c:1636 [opt]
frame #6: 0x00010bd9dfce
qemu-system-x86_64`qemu_thread_start(args=) at
qemu-thread-posix.c:502 [opt]
frame #7: 0x7fff79e24305 libsystem_pthread.dylib`_pthread_body + 126
frame #8: 0x7fff79e2726f libsystem_pthread.dylib`_pthread_start + 70
frame #9: 0x7fff79e23415 libsystem_pthread.dylib`thread_start + 13
To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1818937/+subscriptions
