On 10/29/15 03:28, Yao, Jiewen wrote: > Thanks for the info. I think we might have to dig out why LONG_JUMP fail here. > Most our IA BIOS support Ia32X64 mode, so I am sure it should work.
Right. I'll ask Paolo for help when he's back from his trip. > > For "RSM from 64-bit mode ", I have already confirmed with IA32 SDM owner > that it is just typo. > RSM should work in 64bit mode, so this patch is unnecessary, I believe. Indeed. > For debug purpose, you can apply it at first in your branch, just in case it > is emulator error. That's the case; it should become unnecessary on v4.4+ Linux hosts. > > Have a good night! Thanks! Laszlo > > Thank you > Yao Jiewen > > -----Original Message----- > From: edk2-devel [mailto:[email protected]] On Behalf Of Laszlo > Ersek > Sent: Thursday, October 29, 2015 9:50 AM > To: Yao, Jiewen; Kinney, Michael D; Fan, Jeff > Cc: edk2-devel-01 > Subject: Re: [edk2] about the SMM_S3_RESUME_SMM_64 branch in S3Resume2Pei > > On 10/29/15 02:32, Laszlo Ersek wrote: >> On 10/29/15 01:31, Yao, Jiewen wrote: >>> Hi Ersek >>> I think S3ResumePei supports Ia32 and Ia32X64. It does not support X64. So >>> I believe Ia32X64 crash is a bug somewhere. >>> >>> Since you already run into SmmRestoreCpu(), would you please help to check >>> where is last instruction causing crash? >> >> Sure. The crash occurs on the following call path (starting with >> SmmRestoreCpu()): >> >> SmmRestoreCpu() >> [UefiCpuPkg/PiSmmCpuDxeSmm/PiSmmCpuDxeSmm.c] >> EarlyInitializeCpu() [UefiCpuPkg/PiSmmCpuDxeSmm/CpuS3.c] >> SendInitSipiSipiAllExcludingSelf() >> [UefiCpuPkg/Library/BaseXApicX2ApicLib/BaseXApicX2ApicLib.c] >> >> I have a debug print right after the >> SendInitSipiSipiAllExcludingSelf(), and the BSP starts printing it, >> but before the message is completely printed, one of the APs (just >> started) seems to crash the VM. >> >> It is perfectly possible that the ACPI_CPU_DATA structure that >> OvmfPkg/QuarkPort/CpuS3DataDxe collects earlier causes this. Because, >> before EarlyInitializeCpu() calls SendInitSipiSipiAllExcludingSelf(), >> the startup vector is prepared from ACPI_CPU_DATA, in the >> PrepareApStartupVector() function. No clue what goes wrong there. >> >> Let me check the KVM trace though... Okay, I massaged it a little bit >> for easier readability. Here's when VCPU#0 runs >> SendInitSipiSipiAllExcludingSelf(): >> >>> kvm_exit: reason APIC_ACCESS rip 0x7ffc9d7f info 1300 0 >>> kvm_emulate_insn: 0:7ffc9d7f: 89 10 >>> kvm_mmio: mmio write len 4 gpa 0xfee00300 val 0xc4697 >>> kvm_apic: apic_write APIC_ICR = 0xc4697 >>> kvm_apic_ipi: dst 0 vec 151 (SIPI|physical|assert|edge|all-but-self) >>> kvm_apic_accept_irq: apicid 1 vec 151 (SIPI|edge) >>> kvm_apic_accept_irq: apicid 2 vec 151 (SIPI|edge) >>> kvm_apic_accept_irq: apicid 3 vec 151 (SIPI|edge) >>> kvm_entry: vcpu 0 >> >>> kvm_exit: reason APIC_ACCESS rip 0x7ffc9d06 info 300 0 >>> kvm_emulate_insn: 0:7ffc9d06: 8b 00 >>> kvm_apic: apic_read APIC_ICR = 0xc4697 >>> kvm_mmio: mmio read len 4 gpa 0xfee00300 val 0xc4697 >>> kvm_entry: vcpu 0 >> >>> kvm_exit: reason APIC_ACCESS rip 0x7ffc9d7f info 1310 0 >>> kvm_emulate_insn: 0:7ffc9d7f: 89 10 >>> kvm_mmio: mmio write len 4 gpa 0xfee00310 val 0x0 >>> kvm_apic: apic_write APIC_ICR2 = 0x0 >>> kvm_entry: vcpu 0 >> >> And in response, this is how VCPU#1 behaves (I guess this matches >> "UefiCpuPkg/PiSmmCpuDxeSmm/X64/MpFuncs.asm"?): >> >>> kvm_exit: reason CR_ACCESS rip 0x35 info 0 0 >>> kvm_cr: cr_write 0 = 0x60000011 >>> kvm_entry: vcpu 1 >> >>> kvm_exit: reason CR_ACCESS rip 0x9705b info 4 0 >>> kvm_cr: cr_write 4 = 0x20 >>> kvm_entry: vcpu 1 >> >>> kvm_exit: reason CR_ACCESS rip 0x9705e info 103 0 >>> kvm_cr: cr_write 3 = 0x7ff83000 >>> kvm_entry: vcpu 1 >> >>> kvm_exit: reason MSR_READ rip 0x97068 info 0 0 >>> kvm_msr: msr_read c0000080 = 0x0 >>> kvm_entry: vcpu 1 >> >>> kvm_exit: reason MSR_WRITE rip 0x9706e info 0 0 >>> kvm_msr: msr_write c0000080 = 0x100 >>> kvm_entry: vcpu 1 >> >>> kvm_exit: reason CR_ACCESS rip 0x97077 info 0 0 >>> kvm_cr: cr_write 0 = 0xe0000011 >>> kvm_entry: vcpu 1 >> >>> kvm_exit: reason TRIPLE_FAULT rip 0x9707a info 0 0 >>> kvm_userspace_exit: reason KVM_EXIT_SHUTDOWN (8) >> >> From counting the bytes in "MpFuncs.asm", and correlating them with >> the RIP values in the trace, I *think* that the triple fault happens >> right here: >> >> LONG_JUMP:: >> >> db 67h, 0EAh ; far jump >> dd 0h ; 32-bit offset >> dw 38h ; 16-bit selector >> >> Do you want me to log some values from from PrepareApStartupVector()? >> Like the offset that gets patched in here, or the GDT (to see that >> 0x38 makes sense)... I might have to do that tomorrow ^W after getting >> some sleep however, it's 02:32 AM here. > > FWIW I retested with TCG too (i.e., emulation, not hw virtualization), and > the VM crashes the same way. > > Does it matter that I have this patch in my build, temporarily: > > UefiCpuPkg: PiSmmCpuDxeSmm: do not execute RSM from 64-bit mode > http://thread.gmane.org/gmane.comp.bios.edk2.devel/3020/focus=3023 > > (I don't think it matters at this stage in "MpFuncs.asm", but I should be > clear about my environment...) > > Thanks > Laszlo > > >>> -----Original Message----- >>> From: Laszlo Ersek [mailto:[email protected]] >>> Sent: Thursday, October 29, 2015 7:59 AM >>> To: Yao, Jiewen; Kinney, Michael D; Fan, Jeff >>> Cc: edk2-devel-01 >>> Subject: Re: about the SMM_S3_RESUME_SMM_64 branch in S3Resume2Pei >>> >>> On 10/28/15 23:41, Laszlo Ersek wrote: >>>> On 10/28/15 23:26, Yao, Jiewen wrote: >>>>> Right. It seems S3Resume2Pei does not consider X64 mode. I found at least >>>>> 3 functions need enhancement on mode transition: >>>>> 1) S3RestoreConfig2() - S3Resume <-> SmmCpu (DXE mode); >>>>> 2) S3ResumeExecuteBootScript() - S3Resume <-> BootScriptExecutor >>>>> (DXE >>>>> mode) >>>>> 3) S3ResumeBootOs() - S3Resume -> OS WakingVector (OS decide). >>>> >>>> In practice at least, these problems appear specific to SMM / SMRAM >>>> usage. When we use OVMF's custom (insecure) LockBoxLib instance, the >>>> X64 build of S3Resume2Pei (actually, a fully X64 build of OVMF) >>>> provides a working S3 feature, including Windows 7 and later guests, >>>> and Linux guests. Even a minimal boot script is executed correctly >>>> (it has just an INFO opcode). >>>> >>>> If I remember correctly, quite a few code paths are possible through >>>> S3Resume2Pei. I don't exactly recall which one is taken in the above >>>> case, but I thought I'd point out that it works very well in practice. >>>> (The fact notwithstanding that the lockbox is not protected from the >>>> runtime guest OS.) >>>> >>>> The pure Ia32 case works well both with and without OVMF's SMM feature. >>>> >>>> I don't recall ever testing S3 with the Ia32X64 build; I plan to do >>>> that soonish. >>> >>> Ia32X64 crashes (with SMM enabled) with the following messages leading up >>> to it: >>> >>> -------- >>> SmmLockBoxPeiLib RestoreAllLockBoxInPlace - Exit (Success) >>> S3NvsPageTableAddress - 7DFDE000 (1) >>> SMM S3 Signature = 534D4D53 >>> SMM S3 Stack Base = 7FF8A000 >>> SMM S3 Stack Size = 8000 >>> SMM S3 Resume Entry Point = 7FFB5617 >>> SMM S3 CR0 = 80000033 >>> SMM S3 CR3 = 7FF84000 >>> SMM S3 CR4 = 668 >>> SMM S3 Return CS = 10 >>> SMM S3 Return Entry Point = 846C69 >>> SMM S3 Return Context1 = 7F6FA000 >>> SMM S3 Return Context2 = 7E039000 >>> SMM S3 Return Stack Pointer = 81730C >>> SMM S3 Smst = 7FFFDE00 >>> SmmRestoreCpu() >>> <CRASH> >>> -------- >>> >>> If I build without SMM, then Ia32X64 works fine as well. >>> >>> Summary: >>> - without SMM: S3 works in all three of the Ia32, Ia32X64, and X64 >>> OVMF builds >>> - with SMM: Ia32 works, the other two crash. >>> >>> I guess this just confirms what you've already determined from the code. >>> But, at least, it confirms it. :) >>> >>> Thank you all for looking into it! >>> Laszlo >>> >>> >>>> >>>> Thanks, >>>> Laszlo >>>> >>>>> Thank you >>>>> Yao Jiewen >>>>> >>>>> -----Original Message----- >>>>> From: Laszlo Ersek [mailto:[email protected]] >>>>> Sent: Thursday, October 29, 2015 1:34 AM >>>>> To: Kinney, Michael D; Fan, Jeff; Yao, Jiewen >>>>> Cc: edk2-devel-01 >>>>> Subject: Re: about the SMM_S3_RESUME_SMM_64 branch in S3Resume2Pei >>>>> >>>>> On 10/28/15 17:54, Kinney, Michael D wrote: >>>>>> Laszlo, >>>>>> >>>>>> I do not believe any X64 PEI testing has not been performed with this >>>>>> module. We will investigate a fix. >>>>> >>>>> Thank you. >>>>> >>>>> In any case, in OVMF we might be able to use this module nonetheless, >>>>> with the OvmfPkgIa32X64.dsc build (== 32-bit PEI, 64-bit DXE). >>>>> >>>>> Thanks! >>>>> Laszlo >>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Mike >>>>>> >>>>>>> -----Original Message----- >>>>>>> From: Laszlo Ersek [mailto:[email protected]] >>>>>>> Sent: Wednesday, October 28, 2015 8:57 AM >>>>>>> To: Fan, Jeff; Yao, Jiewen >>>>>>> Cc: edk2-devel-01; Kinney, Michael D >>>>>>> Subject: about the SMM_S3_RESUME_SMM_64 branch in S3Resume2Pei >>>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I have a question about the following code in >>>>>>> "UefiCpuPkg/Universal/Acpi/S3Resume2Pei/S3Resume.c", function >>>>>>> S3RestoreConfig2(): >>>>>>> >>>>>>>> if (SmmS3ResumeState->Signature == SMM_S3_RESUME_SMM_64) { >>>>>>>> // >>>>>>>> // Switch to long mode to complete resume. >>>>>>>> // >>>>>>>> >>>>>>>> InterruptStatus = SaveAndDisableInterrupts (); >>>>>>>> // >>>>>>>> // Need to make sure the GDT is loaded with values that >>>>>>>> support long >>>>>>> mode and real mode. >>>>>>>> // >>>>>>>> AsmWriteGdtr (&mGdt); >>>>>>>> // >>>>>>>> // update segment selectors per the new GDT. >>>>>>>> // >>>>>>>> AsmSetDataSelectors (DATA_SEGEMENT_SELECTOR); >>>>>>>> // >>>>>>>> // Restore interrupt state. >>>>>>>> // >>>>>>>> SetInterruptState (InterruptStatus); >>>>>>>> >>>>>>>> AsmWriteCr3 ((UINTN)SmmS3ResumeState->SmmS3Cr3); >>>>>>>> >>>>>>>> // >>>>>>>> // Disable interrupt of Debug timer, since IDT table >>>>>>>> cannot work in long >>>>>>> mode. >>>>>>>> // NOTE: On x64 platforms, because DisablePaging64() will >>>>>>>> disable >>>>>>> interrupts, >>>>>>>> // the code in S3ResumeExecuteBootScript() cannot be >>>>>>>> halted by soft >>>>>>> debugger. >>>>>>>> // >>>>>>>> SaveAndSetDebugTimerInterrupt (FALSE); >>>>>>>> >>>>>>>> AsmEnablePaging64 ( >>>>>>>> 0x38, >>>>>>>> SmmS3ResumeState->SmmS3ResumeEntryPoint, >>>>>>>> (UINT64)(UINTN)AcpiS3Context, >>>>>>>> 0, >>>>>>>> SmmS3ResumeState->SmmS3StackBase + SmmS3ResumeState- >>>>>>>> SmmS3StackSize >>>>>>>> ); >>>>>>>> } >>>>>>> >>>>>>> At the end of this block, the AsmEnablePaging64() function is called. >>>>>>> That call results in the following call tree, *if* the module was built >>>>>>> for X64: >>>>>>> >>>>>>> AsmEnablePaging64() >>>>>>> [MdePkg/Library/BaseLib/X86EnablePaging64.c] >>>>>>> InternalX86EnablePaging64() [MdePkg/Library/BaseLib/X64/Non-existing.c] >>>>>>> ASSERT (FALSE) >>>>>>> >>>>>>> This is because the InternalX86EnablePaging64() functionality is >>>>>>> unavailable in BaseLib on X64. >>>>>>> >>>>>>> My question: how is this branch in S3RestoreConfig2() supposed to >>>>>>> work *at >>>>>>> all* in an X64 PEI build? >>>>>>> >>>>>>> Thank you, >>>>>>> Laszlo >>>>> >>>> >>> >> > > _______________________________________________ > edk2-devel mailing list > [email protected] > https://lists.01.org/mailman/listinfo/edk2-devel > _______________________________________________ edk2-devel mailing list [email protected] https://lists.01.org/mailman/listinfo/edk2-devel
