On 10/15/15 05:30, Kinney, Michael D wrote: > Laszlo, > > I have 32 VCPUs booting to UEFI Shell using Windows build of 32-bit QEMU.
Great! > If more than 32, then we run out of SMRAM. If you'd like to experiment with more, please locate the "gUefiOvmfPkgTokenSpaceGuid.PcdQ35TsegMbytes" PCD in the DSC file(s). The documentation for the PCD is in the DEC file. It is possible to configure 1, 2, or 8 MB of TSEG (SMRAM). The Ia32 DSC sets 1 MB currently, while the other two (Ia32X64, and X64) DSCs set 2 MB. > > I have made 2 changes from your branch to make this stable: > 1) Force 32KB SMBASE alignment. Required for me because I am using pre-built > QEMU binary. Makes sense. > 2) Your patch to sync MTRRs to all APs in CpuDxe CpuMp.c. I changed > SingleThread parameter from TRUE to FALSE and it resolved all the MP > boot stability issues on my test config. Wow! That's very interesting. > This should work with > either TRUE or FALSE, so there is something else wrong here. I agree. I also thought that the invocation should work regardless of the SingleThread setting, but I figured SingleThread=TRUE would be more "conservative", hence safer / more portable. It is interesting to see the opposite being true in practice! I can flip the SingleThread arguments in the next version, but it would be nice to understand what's happening. Thanks Laszlo > > // > // Synchronize MTRR settings to APs. > // > MtrrGetAllMtrrs (&MtrrSettings); > Status = mMpServicesTemplate.StartupAllAPs ( > &mMpServicesTemplate, // This > SetMtrrsFromBuffer, // Procedure > FALSE, // SingleThread > NULL, // WaitEvent > 0, // > TimeoutInMicrosecsond > &MtrrSettings, // ProcedureArgument > NULL // FailedCpuList > ); > ASSERT (Status == EFI_SUCCESS || Status == EFI_NOT_STARTED); > > > Mike > >> -----Original Message----- >> From: Laszlo Ersek [mailto:[email protected]] >> Sent: Wednesday, October 14, 2015 3:26 PM >> To: [email protected] >> Cc: Paolo Bonzini; Justen, Jordan L; Kinney, Michael D >> Subject: [PATCH v3 00/52] OvmfPkg: support SMM for better security (steps >> towards MP and X64) >> >> Public branch: <https://github.com/lersek/edk2/commits/smm3_v3>. >> >> (Mike's v1 patches are again contained in this branch, but I have >> converted them to CRLF.) >> >> ----*---- >> >> Relative to v2, the following patches are new in v3 (also marked >> individually in the Notes sections): >> >> * PiSmmCpuDxeSmm fixes from Paolo, to be reviewed and squashed in by >> Mike, included here for completeness: >> >> 02/52 UefiCpuPkg: PiSmmCpuDxeSmm: prepare PT in InitPaging before filling >> in PDE >> 03/52 UefiCpuPkg: PiSmmCpuDxeSmm: do not execute RSM from 64-bit >> mode >> >> * Build failure fix for -D SMM_REQUIRE -D SOURCE_DEBUG_ENABLE, from >> Mike: >> >> 21/52 OvmfPkg: resolve DebugAgentLib for DXE_SMM_DRIVER modules >> >> * Features for MP and X64 support, from Paolo (kudos!!!): >> >> 23/52 OvmfPkg: import SmmCpuFeaturesLib from UefiCpuPkg >> 24/52 OvmfPkg: SmmCpuFeaturesLib: remove unnecessary bits >> 25/52 OvmfPkg: SmmCpuFeaturesLib: implement SMRAM state save map >> access >> 26/52 OvmfPkg: SmmCpuFeaturesLib: customize state save map format >> 27/52 OvmfPkg: use relaxed AP SMM synchronization mode >> 37/52 OvmfPkg: port CpuS3DataDxe to X64 >> >> * Patch that adapts the build to the former, from Laszlo: >> >> 38/52 OvmfPkg: build QuarkPort/CpuS3DataDxe for -D SMM_REQUIRE >> >> * SMRAM / TSEG size update for the (Ia32)X64 builds, from Laszlo: >> >> 51/52 OvmfPkg: double the SMRAM (TSEG) size for the 64-bit DXE phase >> builds >> >> ----*---- >> >> The following patches have been changed (and/or reworded) from v2 (also >> marked individually in the Notes sections). Identifying them by their v3 >> numbers here: >> >> * Picked up Reviewed-by tag from Jeff, dropped Cc's: >> >> 04/52 UefiCpuPkg: CpuDxe: broadcast MTRR changes to APs >> >> * Addressed v2 feedback from Mike & Jordan: >> >> 06/52 OvmfPkg: Sec: force reinit of BaseExtractGuidedSectionLib handler >> table >> 11/52 OvmfPkg: add PEIM for providing TSEG-as-SMRAM during PEI >> >> * Adapted to Paolo's patches: >> >> 28/52 OvmfPkg: build PiSmmCpuDxeSmm for -D SMM_REQUIRE >> 29/52 OvmfPkg: add skeleton QuarkPort/CpuS3DataDxe >> 30/52 OvmfPkg: QuarkPort/CpuS3DataDxe: fill in >> ACPI_CPU_DATA.StartupVector >> >> ----*---- >> >> Test environment and results: >> >> Host kernel: >> - latest RHEL-7 development kernel (3.10.0-323.el7), with Paolo's >> following patches backported by yours truly: >> - KVM: x86: clean up kvm_arch_vcpu_runnable >> - KVM: x86: fix SMI to halted VCPU >> >> QEMU: >> - current upstream (c49d3411faae), with Paolo's patch applied: >> - target-i386: allow any alignment for SMBASE >> >> Below, the meaning of "bitness=32" is: >> * qemu-system-i386 >> * -cpu coreduo,-nx >> >> Whereas "bitness=64" means: >> * qemu-system-x86_64 >> * no special -cpu flag >> >> For variable access verification, "efibootmgr" is invoked (without >> options) at the guest OS (Fedlet 20141209) root prompt. >> >> bitness accel VCPUs result >> ------- ----- ----- ----------------------------------------------- >> 32 KVM 1 Fedlet 20141209 boots, S3 works, variables work >> >> 32 KVM 2 stuck in SMBASE relocation, APIC IDs look valid >> >> 32 TCG 1 Fedlet 20141209 boots, S3 works, variables work >> >> 32 TCG 2 Fedlet 20141209 boots, variables (efibootmgr) >> are broken -- nothing is printed >> >> 64 KVM >=1 "KVM: entry failed, hardware error 0x80000021" >> while guest in SMBASE relocation >> >> 64 TCG 1 F21 XFCE LiveCD boots, variable access OK, S3 >> resume triggers InternalX86EnablePaging64() >> ASSERT() in >> "MdePkg/Library/BaseLib/X64/Non-existing.c". >> Looks like a bug in S3Resume2Pei? >> >> 64 TCG 2 F21 XFCE LiveCD boots, variable access >> (efibootmgr) is broken -- reports EINVAL >> >> These results are not consistent with the ones seen by Paolo, which is >> why I didn't update the last (OvmfPkg/README) patch in the series, about >> the current level of functionality. (Added a note to it about this >> fact.) >> >> What makes me glad is that the scenario that I've been testing and >> re-testing successfully since May, i.e., 32-bit/KVM/UP, *does* work. >> >> Thanks >> Laszlo >> >> Cc: Paolo Bonzini <[email protected]> >> Cc: Jordan Justen <[email protected]> >> Cc: Michael Kinney <[email protected]> >> >> Laszlo Ersek (42): >> UefiCpuPkg: CpuDxe: broadcast MTRR changes to APs >> OvmfPkg: introduce -D SMM_REQUIRE and PcdSmmSmramRequire >> OvmfPkg: Sec: force reinit of BaseExtractGuidedSectionLib handler >> table >> OvmfPkg: Sec: assert the build-time calculated end of the scratch >> buffer >> OvmfPkg: decompress FVs on S3 resume if SMM_REQUIRE is set >> OvmfPkg: PlatformPei: allow caching in AddReservedMemoryBaseSizeHob() >> OvmfPkg: PlatformPei: account for TSEG size with PcdSmmSmramRequire >> set >> OvmfPkg: add PEIM for providing TSEG-as-SMRAM during PEI >> OvmfPkg: add DXE_DRIVER for providing TSEG-as-SMRAM during boot-time >> DXE >> OvmfPkg: implement EFI_SMM_CONTROL2_PROTOCOL with a >> DXE_RUNTIME_DRIVER >> OvmfPkg: pull in the SMM IPL and SMM core >> OvmfPkg: pull in CpuIo2Smm driver >> OvmfPkg: AcpiS3SaveDxe: don't fake LockBox protocol if SMM_REQUIRE >> OvmfPkg: LockBox: -D SMM_REQUIRE excludes our fake lockbox >> OvmfPkg: LockBox: use SMM stack with -D SMM_REQUIRE >> OvmfPkg: resolve ReportStatusCodeLib for DXE_SMM_DRIVER modules >> OvmfPkg: resolve CpuExceptionHandlerLib for DXE_SMM_DRIVER modules >> OvmfPkg: set gUefiCpuPkgTokenSpaceGuid.PcdCpuSmmEnableBspElection to >> FALSE >> OvmfPkg: build PiSmmCpuDxeSmm for -D SMM_REQUIRE >> OvmfPkg: add skeleton QuarkPort/CpuS3DataDxe >> OvmfPkg: QuarkPort/CpuS3DataDxe: fill in ACPI_CPU_DATA.StartupVector >> OvmfPkg: QuarkPort/CpuS3DataDxe: handle IDT, GDT and MCE in >> ACPI_CPU_DATA >> OvmfPkg: QuarkPort/CpuS3DataDxe: handle StackAddress and StackSize >> OvmfPkg: import CpuConfigLib header from >> Quark_EDKII_v1.1.0/IA32FamilyCpuBasePkg >> OvmfPkg: QuarkPort/CpuS3DataDxe: fill in ACPI_CPU_DATA.NumberOfCpus >> OvmfPkg: QuarkPort/CpuS3DataDxe: fill in ACPI_CPU_DATA.MtrrTable >> OvmfPkg: QuarkPort/CpuS3DataDxe: handle register tables in >> ACPI_CPU_DATA >> OvmfPkg: build QuarkPort/CpuS3DataDxe for -D SMM_REQUIRE >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: strip trailing whitespace >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: rewrap source code to 79 >> chars >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: fix VALID_ARCHITECTURES in >> INF >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: remove FvbDevLock field >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: remove FvbScratchSpace field >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: no dual addressing needed >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: split out runtime DXE >> specifics >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: clean up includes and >> libraries >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: add DXE_SMM_DRIVER build >> OvmfPkg: QemuFlashFvbServicesRuntimeDxe: adhere to -D SMM_REQUIRE >> OvmfPkg: consolidate variable driver stack in DSC and FDF files >> OvmfPkg: pull in SMM-based variable driver stack >> OvmfPkg: double the SMRAM (TSEG) size for the 64-bit DXE phase builds >> OvmfPkg: README: document SMM status >> >> Michael Kinney (2): >> UefiCpuPkg: CpuDxe: Fix ASSERT() when only 1 CPU detected >> OvmfPkg: resolve DebugAgentLib for DXE_SMM_DRIVER modules >> >> Paolo Bonzini (8): >> UefiCpuPkg: PiSmmCpuDxeSmm: prepare PT in InitPaging before filling in >> PDE >> UefiCpuPkg: PiSmmCpuDxeSmm: do not execute RSM from 64-bit mode >> OvmfPkg: import SmmCpuFeaturesLib from UefiCpuPkg >> OvmfPkg: SmmCpuFeaturesLib: remove unnecessary bits >> OvmfPkg: SmmCpuFeaturesLib: implement SMRAM state save map access >> OvmfPkg: SmmCpuFeaturesLib: customize state save map format >> OvmfPkg: use relaxed AP SMM synchronization mode >> OvmfPkg: port CpuS3DataDxe to X64 >> >> OvmfPkg/OvmfPkg.dec >> | 29 + >> OvmfPkg/OvmfPkgIa32.dsc >> | 119 ++- >> OvmfPkg/OvmfPkgIa32X64.dsc >> | 119 ++- >> OvmfPkg/OvmfPkgX64.dsc >> | 119 ++- >> OvmfPkg/OvmfPkgIa32.fdf >> | 57 +- >> OvmfPkg/OvmfPkgIa32X64.fdf >> | 57 +- >> OvmfPkg/OvmfPkgX64.fdf >> | 57 +- >> OvmfPkg/AcpiS3SaveDxe/AcpiS3SaveDxe.inf >> | 3 >> +- >> OvmfPkg/Library/LockBoxLib/LockBoxBaseLib.inf >> | >> 3 + >> OvmfPkg/Library/LockBoxLib/LockBoxDxeLib.inf >> | >> 3 + >> OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf >> | 38 + >> OvmfPkg/PlatformPei/PlatformPei.inf >> | 5 + >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbServicesRuntimeDxe.inf >> | 44 +- >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/{FvbServicesRuntimeDxe.inf => >> FvbServicesSmm.inf} | 53 +- >> OvmfPkg/QuarkPort/CpuS3DataDxe/CpuS3DataDxe.inf >> | 107 +++ >> OvmfPkg/Sec/SecMain.inf >> | 6 + >> OvmfPkg/SmmAccess/SmmAccess2Dxe.inf >> | >> 57 ++ >> OvmfPkg/SmmAccess/SmmAccessPei.inf >> | 70 >> ++ >> OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf >> | 65 ++ >> OvmfPkg/Include/Register/QemuSmramSaveStateMap.h >> | 184 +++++ >> OvmfPkg/PlatformPei/Platform.h >> | 3 +- >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FwBlockService.h >> | 74 +- >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlash.h >> | 9 +- >> OvmfPkg/QuarkPort/CpuS3DataDxe/Cpu.h >> | >> 97 +++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/IA32/ArchSpecificDef.h >> | 57 ++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/MpApic.h >> | >> 69 ++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/MpCommon.h >> | 187 +++++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/MpService.h >> | >> 49 ++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/X64/ArchSpecificDef.h >> | 59 ++ >> OvmfPkg/QuarkPort/Include/Library/CpuConfigLib.h >> | >> 671 +++++++++++++++++ >> OvmfPkg/SmmAccess/SmramInternal.h >> | 89 >> +++ >> UefiCpuPkg/CpuDxe/CpuMp.h >> | 13 + >> OvmfPkg/AcpiS3SaveDxe/AcpiS3Save.c >> | 14 >> +- >> OvmfPkg/Library/LockBoxLib/LockBoxLib.c >> | 2 + >> OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c >> | 790 ++++++++++++++++++++ >> OvmfPkg/PlatformPei/Fv.c >> | 27 +- >> OvmfPkg/PlatformPei/MemDetect.c >> | 85 >> ++- >> OvmfPkg/PlatformPei/Platform.c >> | 9 +- >> OvmfPkg/PlatformPei/Xen.c >> | 2 +- >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FvbInfo.c >> | 56 +- >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FwBlockService.c >> | 653 +++++++--------- >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FwBlockServiceDxe.c >> | 157 ++++ >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FwBlockServiceSmm.c >> | 69 ++ >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlash.c >> | 27 +- >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlashDxe.c >> | 28 + >> OvmfPkg/QuarkPort/CpuS3DataDxe/IA32/ArchSpecific.c >> | 105 +++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/MpApic.c >> | >> 93 +++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/MpCommon.c >> | 298 ++++++++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/ProcessorConfig.c >> | 335 +++++++++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/X64/ArchSpecific.c >> | 108 +++ >> OvmfPkg/Sec/SecMain.c >> | 52 +- >> OvmfPkg/SmmAccess/SmmAccess2Dxe.c >> | >> 156 ++++ >> OvmfPkg/SmmAccess/SmmAccessPei.c >> | 446 >> +++++++++++ >> OvmfPkg/SmmAccess/SmramInternal.c >> | 188 >> +++++ >> OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.c >> | >> 365 +++++++++ >> UefiCpuPkg/CpuDxe/CpuDxe.c >> | 26 + >> UefiCpuPkg/CpuDxe/CpuMp.c >> | 36 +- >> UefiCpuPkg/PiSmmCpuDxeSmm/SmmProfile.c >> | >> 8 +- >> OvmfPkg/DecomprScratchEnd.fdf.inc >> | 72 ++ >> OvmfPkg/OvmfPkg.fdf.inc >> | 2 + >> OvmfPkg/QuarkPort/CpuS3DataDxe/IA32/CpuAsm.S >> | 56 ++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/IA32/CpuAsm.asm >> | 69 ++ >> OvmfPkg/QuarkPort/CpuS3DataDxe/X64/CpuAsm.nasm >> | 58 ++ >> OvmfPkg/README >> | 43 ++ >> UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmiEntry.S >> | 13 + >> UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmiEntry.asm >> | 13 + >> UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmInit.S >> | 13 + >> UefiCpuPkg/PiSmmCpuDxeSmm/X64/SmmInit.asm >> | 13 + >> 68 files changed, 6468 insertions(+), 591 deletions(-) >> create mode 100644 >> OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.inf >> copy >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/{FvbServicesRuntimeDxe.inf => >> FvbServicesSmm.inf} (59%) >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/CpuS3DataDxe.inf >> create mode 100644 OvmfPkg/SmmAccess/SmmAccess2Dxe.inf >> create mode 100644 OvmfPkg/SmmAccess/SmmAccessPei.inf >> create mode 100644 OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.inf >> create mode 100644 OvmfPkg/Include/Register/QemuSmramSaveStateMap.h >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/Cpu.h >> create mode 100644 >> OvmfPkg/QuarkPort/CpuS3DataDxe/IA32/ArchSpecificDef.h >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/MpApic.h >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/MpCommon.h >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/MpService.h >> create mode 100644 >> OvmfPkg/QuarkPort/CpuS3DataDxe/X64/ArchSpecificDef.h >> create mode 100644 OvmfPkg/QuarkPort/Include/Library/CpuConfigLib.h >> create mode 100644 OvmfPkg/SmmAccess/SmramInternal.h >> create mode 100644 >> OvmfPkg/Library/SmmCpuFeaturesLib/SmmCpuFeaturesLib.c >> create mode 100644 >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FwBlockServiceDxe.c >> create mode 100644 >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/FwBlockServiceSmm.c >> create mode 100644 >> OvmfPkg/QemuFlashFvbServicesRuntimeDxe/QemuFlashDxe.c >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/IA32/ArchSpecific.c >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/MpApic.c >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/MpCommon.c >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/ProcessorConfig.c >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/X64/ArchSpecific.c >> create mode 100644 OvmfPkg/SmmAccess/SmmAccess2Dxe.c >> create mode 100644 OvmfPkg/SmmAccess/SmmAccessPei.c >> create mode 100644 OvmfPkg/SmmAccess/SmramInternal.c >> create mode 100644 OvmfPkg/SmmControl2Dxe/SmmControl2Dxe.c >> create mode 100644 OvmfPkg/DecomprScratchEnd.fdf.inc >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/IA32/CpuAsm.S >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/IA32/CpuAsm.asm >> create mode 100644 OvmfPkg/QuarkPort/CpuS3DataDxe/X64/CpuAsm.nasm >> >> -- >> 1.8.3.1 > _______________________________________________ edk2-devel mailing list [email protected] https://lists.01.org/mailman/listinfo/edk2-devel
