Short summary for Gleb and Paolo:
- qemu master + kvm_intel (3.10) run both OVMF/master and
OVMF/master+SeaBIOS/master CSM well, including booting a legacy OS
- qemu master + TCG spirals into an endless loop when trying to boot a
legacy OS with OVMF+CSM -- I'm not focusing on this right now,
- qemu master + kvm_amd (3.10) throws an emulation failure with pure
OVMF immediately at startup:
On 09/03/13 11:56, Laszlo Ersek wrote:
> On 09/03/13 01:05, Rod Smith wrote:
>> On 09/02/2013 05:32 PM, Laszlo Ersek wrote:
>
>>> Maybe -enable-kvm would make a difference...
>>
>> It causes it to crash with a new set of error messages:
>>
>> KVM internal error. Suberror: 1
>> emulation failure
>> EAX=c0000033 EBX=fffcc0e4 ECX=c0000080 EDX=00000000
>> ESI=fffcc2c4 EDI=00005042 EBP=fffcc000 ESP=00000000
>> EIP=ffffff26 EFL=00000082 [--S----] CPL=0 II=0 A20=1 SMM=0 HLT=0
>> ES =0008 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
>> CS =0010 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA]
>> SS =0008 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
>> DS =0008 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
>> FS =0008 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
>> GS =0008 00000000 ffffffff 00c09300 DPL=0 DS [-WA]
>> LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT
>> TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS64-busy
>> GDT= 00000000ffffff80 0000001f
>> IDT= 0000000000000000 0000ffff
>> CR0=c0000033 CR2=0000000000000000 CR3=00000000ffffe000 CR4=00000660
>> DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000
>> DR3=0000000000000000
>> DR6=00000000ffff0ff0 DR7=0000000000000400
>> EFER=0000000000000500
>> Code=00 c0 0f 32 0f ba e8 08 0f 30 0f 20 c0 0f ba e8 1f 0f 22 c0 <ea>
>> 2d ff ff ff 18 00 e9 93 00 00 00 fa bb 00 f0 8e db bb 6c ff 2e 66 0f
>> 01 17 66 b8 23 00
>> Connected to RFB server, using protocol version 3.8
>>
>> the kvm and kvm_amd modules are loaded on my system when this
>> happens.
In the qemu monitor:
(qemu) x /20i 0xffffffefd
0x0000000ffffffefd: mov $0xffffe000,%eax
0x0000000fffffff02: mov %eax,%cr3
0x0000000fffffff05: mov %cr4,%eax
0x0000000fffffff08: bts $0x5,%eax
0x0000000fffffff0c: mov %eax,%cr4
0x0000000fffffff0f: mov $0xc0000080,%ecx
0x0000000fffffff14: rdmsr
0x0000000fffffff16: bts $0x8,%eax
0x0000000fffffff1a: wrmsr
0x0000000fffffff1c: mov %cr0,%eax
0x0000000fffffff1f: bts $0x1f,%eax
0x0000000fffffff23: mov %eax,%cr0
0x0000000fffffff26: ljmp $0x18,$0xffffff2d <------ emul. failure here
0x0000000fffffff2d: jmp 0xffffffc5
This guest disassembly is part of the UEFI reset vector code under
UefiCpuPkg/ResetVector/Vtf0.
Main16 [Main.asm]
EarlyInit16 [Ia16/Init16.asm]
TransitionFromReal16To32BitFlat [Ia16/Real16ToFlat32.asm]
Flat32SearchForBfvBase [Ia32/SearchForBfvBase.asm]
Flat32SearchForSecEntryPoint [Ia32/SearchForSecEntry.asm]
Transition32FlatTo64Flat [Ia32/Flat32ToFlat64.asm]
The Transition32FlatTo64Flat "function" has the following source,
corresponding to the qemu disassembly:
> BITS 32
>
> ;
> ; Modified: EAX
> ;
> Transition32FlatTo64Flat:
>
> mov eax, ((ADDR_OF_START_OF_RESET_CODE & ~0xfff) - 0x1000)
> mov cr3, eax
0x0000000ffffffefd: mov $0xffffe000,%eax
0x0000000fffffff02: mov %eax,%cr3
>
> mov eax, cr4
> bts eax, 5 ; enable PAE
> mov cr4, eax
0x0000000fffffff05: mov %cr4,%eax
0x0000000fffffff08: bts $0x5,%eax
0x0000000fffffff0c: mov %eax,%cr4
>
> mov ecx, 0xc0000080
> rdmsr
> bts eax, 8 ; set LME
> wrmsr
0x0000000fffffff0f: mov $0xc0000080,%ecx
0x0000000fffffff14: rdmsr
0x0000000fffffff16: bts $0x8,%eax
0x0000000fffffff1a: wrmsr
>
> mov eax, cr0
> bts eax, 31 ; set PG
> mov cr0, eax ; enable paging
0x0000000fffffff1c: mov %cr0,%eax
0x0000000fffffff1f: bts $0x1f,%eax
0x0000000fffffff23: mov %eax,%cr0
>
> jmp LINEAR_CODE64_SEL:ADDR_OF(jumpTo64BitAndLandHere)
0x0000000fffffff26: ljmp $0x18,$0xffffff2d <------ emul. failure here
> BITS 64
> jumpTo64BitAndLandHere:
>
> debugShowPostCode POSTCODE_64BIT_MODE
>
> OneTimeCallRet Transition32FlatTo64Flat
0x0000000fffffff2d: jmp 0xffffffc5
"LINEAR_CODE64_SEL" is defined in "Ia16/Real16ToFlat32.asm" as:
> ;
> ; The Global Descriptor Table (GDT)
> ;
>
> GDT_BASE:
> ; null descriptor
> [...]
>
> ; linear data segment descriptor
> LINEAR_SEL equ $-GDT_BASE
> [...]
>
> ; linear code segment descriptor
> LINEAR_CODE_SEL equ $-GDT_BASE
> [...]
>
> %ifdef ARCH_X64
> ; linear code (64-bit) segment descriptor
> LINEAR_CODE64_SEL equ $-GDT_BASE
> DW 0xffff ; limit 15:0
> DW 0 ; base 15:0
> DB 0 ; base 23:16
> DB PRESENT_FLAG(1)|DPL(0)|SYSTEM_FLAG(1)|DESC_TYPE(CODE64_TYPE)
> DB
> GRANULARITY_FLAG(1)|DEFAULT_SIZE32(0)|CODE64_FLAG(1)|UPPER_LIMIT(0xf)
> DB 0 ; base 31:24
> %endif
>
> GDT_END:
>
I grepped the 3.10 kernel for KVM_INTERNAL_ERROR_EMULATION:
arch/x86/kvm/svm.c: svm->vcpu.run->internal.suberror =
KVM_INTERNAL_ERROR_EMULATION;
arch/x86/kvm/vmx.c: vcpu->run->internal.suberror =
KVM_INTERNAL_ERROR_EMULATION;
arch/x86/kvm/vmx.c: vcpu->run->internal.suberror =
KVM_INTERNAL_ERROR_EMULATION;
arch/x86/kvm/x86.c: vcpu->run->internal.suberror =
KVM_INTERNAL_ERROR_EMULATION;
include/uapi/linux/kvm.h:#define KVM_INTERNAL_ERROR_EMULATION 1
"x86.c" would be my first guess (after x86_decode_insn() or
x86_emulate_insn() fails), but that's probably not it, considering that
the code runs fine on VMX and fails on SVM, and "x86.c" looks like a
shared file.
This is what trace-cmd has to say:
qemu-system-x86-29149 [002] 12270.789910: kvm_update_master_clock: masterclock
0 hostclock tsc offsetmatched 0
qemu-system-x86-29149 [002] 12270.789977: kvm_fpu: load
qemu-system-x86-29149 [002] 12270.789979: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.789987: kvm_exit: reason EXIT_NPF
rip 0xfff0 info 100000014 fffffff0
qemu-system-x86-29149 [002] 12270.789992: kvm_page_fault: address
fffffff0 error_code 14
qemu-system-x86-29149 [002] 12270.790022: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790025: kvm_exit: reason
EXIT_WRITE_CR0 rip 0xff46 info 0 0
qemu-system-x86-29149 [002] 12270.790042: kvm_emulate_insn: ffff0000:ff46:
0f 22 c0
qemu-system-x86-29149 [002] 12270.790060: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790063: kvm_exit: reason
EXIT_WRITE_CR4 rip 0xffffff56 info 0 0
qemu-system-x86-29149 [002] 12270.790065: kvm_emulate_insn: 0:ffffff56: 0f
22 e0
qemu-system-x86-29149 [002] 12270.790069: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790076: kvm_exit: reason
EXIT_READ_CR4 rip 0xffffff05 info 0 0
qemu-system-x86-29149 [002] 12270.790079: kvm_emulate_insn: 0:ffffff05: 0f
20 e0
qemu-system-x86-29149 [002] 12270.790082: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790084: kvm_exit: reason
EXIT_WRITE_CR4 rip 0xffffff0c info 0 0
qemu-system-x86-29149 [002] 12270.790086: kvm_emulate_insn: 0:ffffff0c: 0f
22 e0
qemu-system-x86-29149 [002] 12270.790092: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790095: kvm_exit: reason EXIT_MSR
rip 0xffffff14 info 0 0
qemu-system-x86-29149 [002] 12270.790097: kvm_msr: msr_read
c0000080 = 0x0
qemu-system-x86-29149 [002] 12270.790098: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790100: kvm_exit: reason EXIT_MSR
rip 0xffffff1a info 1 0
qemu-system-x86-29149 [002] 12270.790102: kvm_msr: msr_write
c0000080 = 0x100
qemu-system-x86-29149 [002] 12270.790103: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790105: kvm_exit: reason
EXIT_READ_CR0 rip 0xffffff1c info 0 0
qemu-system-x86-29149 [002] 12270.790108: kvm_emulate_insn: 0:ffffff1c: 0f
20 c0
qemu-system-x86-29149 [002] 12270.790109: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790112: kvm_exit: reason
EXIT_WRITE_CR0 rip 0xffffff23 info 0 0
qemu-system-x86-29149 [002] 12270.790113: kvm_emulate_insn: 0:ffffff23: 0f
22 c0
qemu-system-x86-29149 [002] 12270.790121: kvm_entry: vcpu 0
qemu-system-x86-29149 [002] 12270.790124: kvm_exit: reason EXIT_NPF
rip 0xffffff26 info 200000007 ffffe000
qemu-system-x86-29149 [002] 12270.790125: kvm_page_fault: address
ffffe000 error_code 7
qemu-system-x86-29149 [002] 12270.790129: kvm_emulate_insn: 0:ffffff26: 0f
22 c0
qemu-system-x86-29149 [002] 12270.790131: kvm_emulate_insn: 0:ffffff26: 0f
22 c0 FAIL
qemu-system-x86-29149 [002] 12270.790134: kvm_userspace_exit: reason
KVM_EXIT_INTERNAL_ERROR (17)
qemu-system-x86-29149 [002] 12270.790137: kvm_fpu: unload
After all this does implicate x86_decode_insn():
x86_emulate_instruction() [arch/x86/kvm/x86.c]
x86_decode_insn() [arch/x86/kvm/emulate.c]
trace_kvm_emulate_insn_start()
handle_emulation_failure() [arch/x86/kvm/x86.c]
trace_kvm_emulate_insn_failed
sets KVM_INTERNAL_ERROR_EMULATION
Interestingly, the trace lists "0f 22 c0", which corresponds to
mov %eax,%cr0
(see near "enable paging" above). So I'm not sure if the "ljmp" fails,
or the mov-to-cr0.
Volume 2 of the Intel SDM says under "MOV -- Move to/from Control
Registers" (we're in 32-bit mode, executing Transition32FlatTo64Flat):
Opcode Instruction Op/En 64-Bit Mode Compat/Leg Mode Description
-------- ------------ ----- ----------- --------------- -----------
0F 22 /r MOV CR0-CR7, A Not Valid Move r32 to
r32 Encodable control register
A reference is made to Volume 3, Chapter 22.4, "CHANGES TO INSTRUCTION
BEHAVIOR IN VMX NON-ROOT OPERATION"... which reads:
MOV to CR0. An execution of MOV to CR0 that does not cause a VM exit
(see Section 22.1.3) leaves unmodified any bit in CR0 corresponding to
a bit set in the CR0 guest/host mask. Treatment of attempts to modify
other bits in CR0 depends on the setting of the unrestricted guest
VM-execution control: ^1
- If the control is 0, MOV to CR0 causes a general-protection
exception if it attempts to set any bit in CR0 to a value not
supported in VMX operation (see Section 20.8).
- If the control is 1, MOV to CR0 causes a general-protection
exception if it attempts to set any bit in CR0 other than bit 0 (PE)
or bit 31 (PG) to a value not supported in VMX operation. It remains
the case, however, that MOV to CR0 causes a general-protection
exception if it would result in CR0.PE = 0 and CR0.PG = 1 or if it
would result in CR0.PG = 1, CR4.PAE = 0, and IA32_EFER.LME = 1.
Ugh, thanks anyway...
So, what's going on here? Perhaps, does this mov-to-cr0 *not* exit on
Intel, but unexpectedly exits on AMD, and the instruction emulator is
not prepared to handle it?
In x86_decode_insn():
/* Opcode byte(s). */
opcode = opcode_table[ctxt->b];
/* Two-byte opcode? */
if (ctxt->b == 0x0f) {
ctxt->twobyte = 1;
ctxt->b = insn_fetch(u8, ctxt);
opcode = twobyte_table[ctxt->b];
}
ctxt->d = opcode.flags;
twobyte_table[0x22] is
IIP(ModRM | SrcMem | Priv | Op3264, em_cr_write, cr_write,
check_cr_write),
which at least appears to match mov-to-cr0.
I guess Rod is the first user who tried OVMF on SVM...
Thanks
Laszlo
------------------------------------------------------------------------------
Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more!
Discover the easy way to master current and previous Microsoft technologies
and advance your career. Get an incredible 1,500+ hours of step-by-step
tutorial videos with LearnDevNow. Subscribe today and save!
http://pubads.g.doubleclick.net/gampad/clk?id=58040911&iu=/4140/ostg.clktrk
_______________________________________________
edk2-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/edk2-devel
