Mike Larkin <[email protected]> writes:
> On Sat, Nov 20, 2021 at 09:14:31PM -0500, Dave Voutila wrote:
>> The below diff fixes an issue reported by kn@ on bugs@ [1]. joshe@ also
>> observed the issue and confirmed the below diff resolves it.
>>
>> The symptoms were quite odd: errors from fdc(4) during an OpenBSD guest
>> booting under vmm(4)/vmd(8). We don't emulate a floppy disk drive!!!
>>
>> I introduced a bug in r1.287 [2] when simplifying parts of
>> vcpu_run_{svm,vmx} by letting the functions return 0 instead of
>> voluntarily yielding. The edge case I didn't account for is if after a
>> vmexit for an IN instruction, the io port address isn't one emulated by
>> vmd(8) in userland, vmm(4) will perform the emulation (not the bug) by
>> writing the appropriate number of 0xff bytes to AL/AX/EAX. IF the
>> scheduler would like us to yield, we return setting a vrp exit code of
>> VM_EXIT_NONE (since we aren't asking userland/vmd to help with any
>> emulation).
>>
>> vmd(8) correctly handles this exit, but vmm(4) never copies out the
>> current vcpu registers and irqready state. When vmd(8) runs the vcpu
>> again, the vcpu's guest state still has a vmexit related to the IO
>> operation and presumes vmd(8) modified RAX and overwrites the vcpu's
>> RAX before re-entering the guest.
>>
>> This behavior occurs on both Intel and AMD. To confirm, I added some
>> printfs to fdc(4) and specifically checked when the dma reads returned
>> something other than 0xff on instances of both types of host. (Since
>> it's probabilistic, it's not uncommon to see it happen only 3-4 times
>> out of the 100k bus reads out_fdc() attempts, but it seems more
>> reproducible on older hardware.)
>>
>> ok?
>>
>> -dv
>>
>
> ok mlarkin if not already committed
>
Prior to commit I was was testing my diff with some others I'm working
on and found I missed one thing. We should be using the vrp exit reason,
not the one in the vcpu.vc_gueststate. While it's a bit odd to carry the
exit reason in two places, the one we send to userland is a part of
vrp...and vrp is where we check for the continue flag.
I was slightly off in my prior assessment. We emulate rax on the
vc_gueststate and set the PREVIOUS rax on the vc_exit data sent to
userland. So while my diff vastly reduced the probability of delivering
a response that would convince fdc(4) that there's a device, it was
still missing this piece to make sure we don't clobber the vc_gueststate
with the rax result that we emulated the in/out vmexit.
diff 73abbd0403c8988c9d44e9b268fc8d0b7e665e30 /usr/src
blob - 10832653c2c271a7fd1dd8bd8ae57a5f3c3c2fe3
file + sys/arch/amd64/amd64/vmm.c
--- sys/arch/amd64/amd64/vmm.c
+++ sys/arch/amd64/amd64/vmm.c
@@ -4644,7 +4644,7 @@ vcpu_run_vmx(struct vcpu *vcpu, struct vm_run_params *
irq = vrp->vrp_irq;
if (vrp->vrp_continue) {
- switch (vcpu->vc_gueststate.vg_exit_reason) {
+ switch (vrp->vrp_exit_reason) {
case VMX_EXIT_IO:
if (vcpu->vc_exit.vei.vei_dir == VEI_DIR_IN)
vcpu->vc_gueststate.vg_rax =
@@ -7099,7 +7099,7 @@ vcpu_run_svm(struct vcpu *vcpu, struct vm_run_params *
* exit data structure.
*/
if (vrp->vrp_continue) {
- switch (vcpu->vc_gueststate.vg_exit_reason) {
+ switch (vrp->vrp_exit_reason) {
case SVM_VMEXIT_IOIO:
if (vcpu->vc_exit.vei.vei_dir == VEI_DIR_IN) {
vcpu->vc_gueststate.vg_rax =
>> [1] https://marc.info/?l=openbsd-bugs&m=163682062027764&w=2
>> [2]
>> https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/sys/arch/amd64/amd64/vmm.c.diff?r1=1.286&r2=1.287
>>
>>
>> Index: sys/arch/amd64/amd64/vmm.c
>> ===================================================================
>> RCS file: /opt/cvs/src/sys/arch/amd64/amd64/vmm.c,v
>> retrieving revision 1.294
>> diff -u -p -r1.294 vmm.c
>> --- sys/arch/amd64/amd64/vmm.c 26 Oct 2021 16:29:49 -0000 1.294
>> +++ sys/arch/amd64/amd64/vmm.c 20 Nov 2021 21:46:07 -0000
>> @@ -4301,9 +4301,10 @@ vm_run(struct vm_run_params *vrp)
>> rw_exit_write(&vmm_softc->vm_lock);
>> }
>> ret = 0;
>> - } else if (ret == EAGAIN) {
>> + } else if (ret == 0 || ret == EAGAIN) {
>> /* If we are exiting, populate exit data so vmd can help. */
>> - vrp->vrp_exit_reason = vcpu->vc_gueststate.vg_exit_reason;
>> + vrp->vrp_exit_reason = (ret == 0) ? VM_EXIT_NONE
>> + : vcpu->vc_gueststate.vg_exit_reason;
>> vrp->vrp_irqready = vcpu->vc_irqready;
>> vcpu->vc_state = VCPU_STATE_STOPPED;
>>
>> @@ -4312,9 +4313,6 @@ vm_run(struct vm_run_params *vrp)
>> ret = EFAULT;
>> } else
>> ret = 0;
>> - } else if (ret == 0) {
>> - vrp->vrp_exit_reason = VM_EXIT_NONE;
>> - vcpu->vc_state = VCPU_STATE_STOPPED;
>> } else {
>> vrp->vrp_exit_reason = VM_EXIT_TERMINATED;
>> vcpu->vc_state = VCPU_STATE_TERMINATED;
--
-Dave Voutila