Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Nov 27, 2019, at 04:16, Jan Beulich wrote: > > On 26.11.2019 22:20, Rich Persaud wrote: >> As an intermediate step, could we have an umbrella opt-in >> Kconfig option (CONFIG_EFI_NONSPEC_COMPATIBILITY?) that >> enables multiple EFI options for maximum hardware compatibility? >> For this thread and Xen 4.13, that would be >> EFI_SET_VIRTUAL_ADDRESS_MAP and efi=attr=uc. If more >> options/quirks are added in the future, downstreams using >> EFI_NONSPEC_COMPATIBILITY would get them by default. > > While I don't particularly like it, I'd be okay with having such > an option, provided it doesn't hamper code readability too much. > However - why would you stop at those two things? Why not also > exclude reboot through UEFI (as indicated by Andrew), or use of > runtime services as a whole? What about /mapbs? The fundamental > problem I see here really is - where would we draw the line? If we take this thread as an example, a middle ground was found among developers motivated to maintain the workarounds for downstream projects with affected hardware. Qubes, EVE & OpenXT are used on edge/client devices that often have (relative to servers) a shorter lifetime, with more device churn and support costs. These two initial options would address current pain points and enable the use of upstream Xen + EFI RS on more devices, e.g. for OTA updates with forward-sealed integrity measurements. The line could change if more downstreams adopt the option and/or new devices appear that have both customer adoption and problematic firmware behavior. Rich ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Wed, Nov 27, 2019 at 10:14:56AM +0100, Jan Beulich wrote: > On 26.11.2019 22:20, Rich Persaud wrote: > > As an intermediate step, could we have an umbrella opt-in > > Kconfig option (CONFIG_EFI_NONSPEC_COMPATIBILITY?) that > > enables multiple EFI options for maximum hardware compatibility? > > For this thread and Xen 4.13, that would be > > EFI_SET_VIRTUAL_ADDRESS_MAP and efi=attr=uc. If more > > options/quirks are added in the future, downstreams using > > EFI_NONSPEC_COMPATIBILITY would get them by default. > > While I don't particularly like it, I'd be okay with having such > an option, provided it doesn't hamper code readability too much. > However - why would you stop at those two things? Why not also > exclude reboot through UEFI (as indicated by Andrew), or use of > runtime services as a whole? What about /mapbs? The fundamental > problem I see here really is - where would we draw the line? Yes, it isn't easy to draw that line for all the downstream projects at once. For example it looks like efi=no-rs is an acceptable compromise for Project EVE, while it isn't for Qubes or OpenXT. But moving from "apply this set of patches" to "enable those options" would be an improvement. Ideally Xen should work out of the box on as many boxes as possible. If that means enabling some workarounds by default, I'm fine with it (unless it _severely_ impact other configurations). In Qubes we struggle with hardware compatibility because of large variety of client hardware, firmware and configuration. Whatever we say here, in the end it boils down to "does project X work on my hardware?". Not sure about other Xen use cases, but we prefer to have the answer "yes", whenever it's reasonably possible. I think enabling efi=attr=uc and EFI_SET_VIRTUAL_ADDRESS_MAP by default is a reasonable approach. Defaulting to a different reboot method may be too, but I haven't seen too many machines impacted by this particular issue. Maybe because Xen+UEFI breaks much earlier there. FWIW we do enable efi=attr=uc, /mapbs and /noexitboot by default (until EFI_SET_VIRTUAL_ADDRESS_MAP was added). -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? signature.asc Description: PGP signature ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On 26.11.2019 22:20, Rich Persaud wrote: > As an intermediate step, could we have an umbrella opt-in > Kconfig option (CONFIG_EFI_NONSPEC_COMPATIBILITY?) that > enables multiple EFI options for maximum hardware compatibility? > For this thread and Xen 4.13, that would be > EFI_SET_VIRTUAL_ADDRESS_MAP and efi=attr=uc. If more > options/quirks are added in the future, downstreams using > EFI_NONSPEC_COMPATIBILITY would get them by default. While I don't particularly like it, I'd be okay with having such an option, provided it doesn't hamper code readability too much. However - why would you stop at those two things? Why not also exclude reboot through UEFI (as indicated by Andrew), or use of runtime services as a whole? What about /mapbs? The fundamental problem I see here really is - where would we draw the line? Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes
On 26.11.2019 21:18, Andrew Cooper wrote: > On 26/11/2019 20:12, Roman Shaposhnik wrote: >> On Tue, Nov 26, 2019 at 10:32 AM Marek Marczykowski-Górecki >>> So, to improve Xen's hardware/firmware compatibility, I have two ideas: >>> >>> 1. Make efi=attr=uc the default (it's still possible to disable it with >>> efi=attr=no). >> I'd be very much in favor of that too (especially since it seems to match >> Linux behaviour) What do others think? > > Its more than just this. Linux also doesn't use EFI reboot because it > is broken almost everywhere (because Windows doesn't use it because its > broken almost everywhere, so it never gets fixed). > > Xen should be following Linux, but I'm exhausted arguing this point. Where it makes sense, yes. But there are cases where it doesn't (we don't, for example, want to blindly inherit bugs). Nor do I see why Linux should be the only possible reference. If other OSes work around issues in a better way than Linux does, why should we follow Linux rather than such alternative implementation? > A consequence is that downstream tend to share a pile of "unbreak Xen on > UEFI" patches which have been rejected upstream on philosophical rather > than technical grounds, despite this being a toxic environment to work in. We'll get out of this recurring debate only if you or anyone else propose to have someone other than me be the UEFI code maintainer. No matter that you call them philosophical rather than technical arguments, I continue to be of the firm opinion that workarounds for all sorts of things are acceptable, but shouldn't impact in any way systems adhering to standards. (It is probably [bad] luck that I've not myself been severely impacted by UEFI implementation issues with any of the boxes I routinely test on.) Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes
On 26.11.2019 19:32, Marek Marczykowski-Górecki wrote: > Anyway, if I understand correctly, MMIO region should be mapped as UC, > right? While MMIO typically would want to be UC, there are clearly cases where they'd better be WC, and there may even be cases where they want to be WT, WP, or WB. Hence the lack of firmware indication is a problem even for this specific memory type. Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Tue, Nov 26, 2019 at 1:20 PM Rich Persaud wrote: > > On Nov 26, 2019, at 15:23, Andrew Cooper wrote: > > > On 26/11/2019 20:12, Roman Shaposhnik wrote: > > On Tue, Nov 26, 2019 at 10:32 AM Marek Marczykowski-Górecki > > wrote: > > On Tue, Nov 26, 2019 at 09:56:25AM -0800, Roman Shaposhnik wrote: > > Hi Marek, after applying Jan's patch I'm making much further progress. > > Xen boots fine and Dom0 seems to be OK (more tests are needed tho on > > my end). > > > I'm attaching the logs from Xen and Dom0. > > > At this point it seems that adding efi=attr=uc is a better option for > > these boxes than a wholesale efi=no-rs > > > Question #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was > > supposed to cover by default (so I don't have to add efi=attr=uc)? > > No, this looks like some different firmware (?) issue. > > > Question #2: is there any downside to *always* specifying efi=attr=uc? > > Even for servers that, strictly speaking, don't need it? > > TL;DR: It should be fine. It is what Linux does too. > > > Details: > > > Lets take a look why 'efi=attr=uc' helps, and how can we make it work > > out of the box: > > > The issue is about memory marked as type=11 (EfiMemoryMappedIO) with > > attr=8000 (EFI_MEMORY_RUNTIME). Indeed none of cachability > > attribute is defined. For the record, defined attributes are (UEFI spec > > .6): > > >EFI_MEMORY_UC Memory cacheability attribute: The memory region supports > >being configured as not cacheable. > > >EFI_MEMORY_WC Memory cacheability attribute: The memory region supports > >being configured as write combining. > > >EFI_MEMORY_WT Memory cacheability attribute: The memory region supports > >being configured as cacheable with a “write through” policy. > >Writes that hit in the cache will also be written to main memory. > > >EFI_MEMORY_WB Memory cacheability attribute: The memory region supports > >being configured as cacheable with a “write back” policy. Reads > >and writes that hit in the cache do not propagate to main memory. > >Dirty data is written back to main memory when a new cache line > >is allocated. > > >EFI_MEMORY_UCE Memory cacheability attribute: The memory region supports > >being configured as not cacheable, exported, and supports the > >“fetch and add” semaphore mechanism. > > > My reading of UEFI spec doesn't give much hints what to do with memory > > mappings without any cachability attribute. The only related info I've > > found is about EfiMemoryMappedIO: > > >This memory is not used by the OS. All system memory-mapped IO > >information should come from ACPI tables. > > > So, maybe there is some more info? > > > Anyway, if I understand correctly, MMIO region should be mapped as UC, > > right? > > > I've also taken look at what Linux does. And basically, the only bit > > Linux care about is EFI_MEMORY_WB - if it's absent, then set the region > > as uncachable (page cache disabled bit in page table entry). So, > > basically Linux by default does what Xen's efi=attr=uc does. > > Very interesting! Thanks for doing the research. > > > So, to improve Xen's hardware/firmware compatibility, I have two ideas: > > > 1. Make efi=attr=uc the default (it's still possible to disable it with > > efi=attr=no). > > I'd be very much in favor of that too (especially since it seems to match > > Linux behaviour) What do others think? > > > Its more than just this. Linux also doesn't use EFI reboot because it > is broken almost everywhere (because Windows doesn't use it because its > broken almost everywhere, so it never gets fixed). > > Xen should be following Linux, but I'm exhausted arguing this point. > > A consequence is that downstream tend to share a pile of "unbreak Xen on > UEFI" patches which have been rejected upstream on philosophical rather > than technical grounds, despite this being a toxic environment to work in. > > > As an intermediate step, could we have an umbrella opt-in Kconfig option > (CONFIG_EFI_NONSPEC_COMPATIBILITY?) that enables multiple EFI options for > maximum hardware compatibility? For this thread and Xen 4.13, that would be > EFI_SET_VIRTUAL_ADDRESS_MAP and efi=attr=uc. If more options/quirks are > added in the future, downstreams using EFI_NONSPEC_COMPATIBILITY would get > them by default. As one of those downstream users I have to say I like this A LOT! > The long-term solution is an OSS virtualization-security test tool (e.g. with > Xen and QEMU KVM) that can be run by OEM/ODM QA factory teams on > pre-production firmware and hardware. That is the most OEM-actionable > development window where firmware quality issues can be detected and fixed. > Microsoft's hardware logo/certification work with Windows 10 OEMs on "secured > core" features is also tackling firmware improvements for > virtualization-based security. That's a good proposal, but the question, as always becomes who moves the needle on this one so we avoid
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Nov 26, 2019, at 15:23, Andrew Cooper wrote: > On 26/11/2019 20:12, Roman Shaposhnik wrote: >>> On Tue, Nov 26, 2019 at 10:32 AM Marek Marczykowski-Górecki >>> wrote: >>> On Tue, Nov 26, 2019 at 09:56:25AM -0800, Roman Shaposhnik wrote: Hi Marek, after applying Jan's patch I'm making much further progress. Xen boots fine and Dom0 seems to be OK (more tests are needed tho on my end). I'm attaching the logs from Xen and Dom0. At this point it seems that adding efi=attr=uc is a better option for these boxes than a wholesale efi=no-rs Question #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was supposed to cover by default (so I don't have to add efi=attr=uc)? >>> No, this looks like some different firmware (?) issue. Question #2: is there any downside to *always* specifying efi=attr=uc? Even for servers that, strictly speaking, don't need it? >>> TL;DR: It should be fine. It is what Linux does too. >>> Details: >>> Lets take a look why 'efi=attr=uc' helps, and how can we make it work >>> out of the box: >>> The issue is about memory marked as type=11 (EfiMemoryMappedIO) with >>> attr=8000 (EFI_MEMORY_RUNTIME). Indeed none of cachability >>> attribute is defined. For the record, defined attributes are (UEFI spec >>> .6): >>>EFI_MEMORY_UC Memory cacheability attribute: The memory region supports >>>being configured as not cacheable. >>>EFI_MEMORY_WC Memory cacheability attribute: The memory region supports >>>being configured as write combining. >>>EFI_MEMORY_WT Memory cacheability attribute: The memory region supports >>>being configured as cacheable with a “write through” policy. >>>Writes that hit in the cache will also be written to main memory. >>>EFI_MEMORY_WB Memory cacheability attribute: The memory region supports >>>being configured as cacheable with a “write back” policy. Reads >>>and writes that hit in the cache do not propagate to main memory. >>>Dirty data is written back to main memory when a new cache line >>>is allocated. >>>EFI_MEMORY_UCE Memory cacheability attribute: The memory region supports >>>being configured as not cacheable, exported, and supports the >>>“fetch and add” semaphore mechanism. >>> My reading of UEFI spec doesn't give much hints what to do with memory >>> mappings without any cachability attribute. The only related info I've >>> found is about EfiMemoryMappedIO: >>>This memory is not used by the OS. All system memory-mapped IO >>>information should come from ACPI tables. >>> So, maybe there is some more info? >>> Anyway, if I understand correctly, MMIO region should be mapped as UC, >>> right? >>> I've also taken look at what Linux does. And basically, the only bit >>> Linux care about is EFI_MEMORY_WB - if it's absent, then set the region >>> as uncachable (page cache disabled bit in page table entry). So, >>> basically Linux by default does what Xen's efi=attr=uc does. >> Very interesting! Thanks for doing the research. >> >>> So, to improve Xen's hardware/firmware compatibility, I have two ideas: >>> 1. Make efi=attr=uc the default (it's still possible to disable it with >>> efi=attr=no). >> I'd be very much in favor of that too (especially since it seems to match >> Linux behaviour) What do others think? > > Its more than just this. Linux also doesn't use EFI reboot because it > is broken almost everywhere (because Windows doesn't use it because its > broken almost everywhere, so it never gets fixed). > > Xen should be following Linux, but I'm exhausted arguing this point. > > A consequence is that downstream tend to share a pile of "unbreak Xen on > UEFI" patches which have been rejected upstream on philosophical rather > than technical grounds, despite this being a toxic environment to work in. As an intermediate step, could we have an umbrella opt-in Kconfig option (CONFIG_EFI_NONSPEC_COMPATIBILITY?) that enables multiple EFI options for maximum hardware compatibility? For this thread and Xen 4.13, that would be EFI_SET_VIRTUAL_ADDRESS_MAP and efi=attr=uc. If more options/quirks are added in the future, downstreams using EFI_NONSPEC_COMPATIBILITY would get them by default. The long-term solution is an OSS virtualization-security test tool (e.g. with Xen and QEMU KVM) that can be run by OEM/ODM QA factory teams on pre-production firmware and hardware. That is the most OEM-actionable development window where firmware quality issues can be detected and fixed. Microsoft's hardware logo/certification work with Windows 10 OEMs on "secured core" features is also tackling firmware improvements for virtualization-based security. From the business side, Dell/HP/Lenovo + other OEMs and ODMs could add premium "FirmCare" SKUs into their custom build ordering systems, where customers could pay a small fee for additional firmware support, custom root-of-trust (e.g. BootGuard) key management, or even coreboot.
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On 26/11/2019 20:12, Roman Shaposhnik wrote: > On Tue, Nov 26, 2019 at 10:32 AM Marek Marczykowski-Górecki > wrote: >> On Tue, Nov 26, 2019 at 09:56:25AM -0800, Roman Shaposhnik wrote: >>> Hi Marek, after applying Jan's patch I'm making much further progress. >>> Xen boots fine and Dom0 seems to be OK (more tests are needed tho on >>> my end). >>> >>> I'm attaching the logs from Xen and Dom0. >>> >>> At this point it seems that adding efi=attr=uc is a better option for >>> these boxes than a wholesale efi=no-rs >>> >>> Question #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was >>> supposed to cover by default (so I don't have to add efi=attr=uc)? >> No, this looks like some different firmware (?) issue. >> >>> Question #2: is there any downside to *always* specifying efi=attr=uc? >>> Even for servers that, strictly speaking, don't need it? >> TL;DR: It should be fine. It is what Linux does too. >> >> Details: >> >> Lets take a look why 'efi=attr=uc' helps, and how can we make it work >> out of the box: >> >> The issue is about memory marked as type=11 (EfiMemoryMappedIO) with >> attr=8000 (EFI_MEMORY_RUNTIME). Indeed none of cachability >> attribute is defined. For the record, defined attributes are (UEFI spec >> .6): >> >> EFI_MEMORY_UC Memory cacheability attribute: The memory region supports >> being configured as not cacheable. >> >> EFI_MEMORY_WC Memory cacheability attribute: The memory region supports >> being configured as write combining. >> >> EFI_MEMORY_WT Memory cacheability attribute: The memory region supports >> being configured as cacheable with a “write through” policy. >> Writes that hit in the cache will also be written to main memory. >> >> EFI_MEMORY_WB Memory cacheability attribute: The memory region supports >> being configured as cacheable with a “write back” policy. Reads >> and writes that hit in the cache do not propagate to main memory. >> Dirty data is written back to main memory when a new cache line >> is allocated. >> >> EFI_MEMORY_UCE Memory cacheability attribute: The memory region supports >> being configured as not cacheable, exported, and supports the >> “fetch and add” semaphore mechanism. >> >> My reading of UEFI spec doesn't give much hints what to do with memory >> mappings without any cachability attribute. The only related info I've >> found is about EfiMemoryMappedIO: >> >> This memory is not used by the OS. All system memory-mapped IO >> information should come from ACPI tables. >> >> So, maybe there is some more info? >> >> Anyway, if I understand correctly, MMIO region should be mapped as UC, >> right? >> >> I've also taken look at what Linux does. And basically, the only bit >> Linux care about is EFI_MEMORY_WB - if it's absent, then set the region >> as uncachable (page cache disabled bit in page table entry). So, >> basically Linux by default does what Xen's efi=attr=uc does. > Very interesting! Thanks for doing the research. > >> So, to improve Xen's hardware/firmware compatibility, I have two ideas: >> >> 1. Make efi=attr=uc the default (it's still possible to disable it with >> efi=attr=no). > I'd be very much in favor of that too (especially since it seems to match > Linux behaviour) What do others think? Its more than just this. Linux also doesn't use EFI reboot because it is broken almost everywhere (because Windows doesn't use it because its broken almost everywhere, so it never gets fixed). Xen should be following Linux, but I'm exhausted arguing this point. A consequence is that downstream tend to share a pile of "unbreak Xen on UEFI" patches which have been rejected upstream on philosophical rather than technical grounds, despite this being a toxic environment to work in. ~Andrew ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Tue, Nov 26, 2019 at 10:32 AM Marek Marczykowski-Górecki wrote: > > On Tue, Nov 26, 2019 at 09:56:25AM -0800, Roman Shaposhnik wrote: > > Hi Marek, after applying Jan's patch I'm making much further progress. > > Xen boots fine and Dom0 seems to be OK (more tests are needed tho on > > my end). > > > > I'm attaching the logs from Xen and Dom0. > > > > At this point it seems that adding efi=attr=uc is a better option for > > these boxes than a wholesale efi=no-rs > > > > Question #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was > > supposed to cover by default (so I don't have to add efi=attr=uc)? > > No, this looks like some different firmware (?) issue. > > > Question #2: is there any downside to *always* specifying efi=attr=uc? > > Even for servers that, strictly speaking, don't need it? > > TL;DR: It should be fine. It is what Linux does too. > > Details: > > Lets take a look why 'efi=attr=uc' helps, and how can we make it work > out of the box: > > The issue is about memory marked as type=11 (EfiMemoryMappedIO) with > attr=8000 (EFI_MEMORY_RUNTIME). Indeed none of cachability > attribute is defined. For the record, defined attributes are (UEFI spec > .6): > > EFI_MEMORY_UC Memory cacheability attribute: The memory region supports > being configured as not cacheable. > > EFI_MEMORY_WC Memory cacheability attribute: The memory region supports > being configured as write combining. > > EFI_MEMORY_WT Memory cacheability attribute: The memory region supports > being configured as cacheable with a “write through” policy. > Writes that hit in the cache will also be written to main memory. > > EFI_MEMORY_WB Memory cacheability attribute: The memory region supports > being configured as cacheable with a “write back” policy. Reads > and writes that hit in the cache do not propagate to main memory. > Dirty data is written back to main memory when a new cache line > is allocated. > > EFI_MEMORY_UCE Memory cacheability attribute: The memory region supports > being configured as not cacheable, exported, and supports the > “fetch and add” semaphore mechanism. > > My reading of UEFI spec doesn't give much hints what to do with memory > mappings without any cachability attribute. The only related info I've > found is about EfiMemoryMappedIO: > > This memory is not used by the OS. All system memory-mapped IO > information should come from ACPI tables. > > So, maybe there is some more info? > > Anyway, if I understand correctly, MMIO region should be mapped as UC, > right? > > I've also taken look at what Linux does. And basically, the only bit > Linux care about is EFI_MEMORY_WB - if it's absent, then set the region > as uncachable (page cache disabled bit in page table entry). So, > basically Linux by default does what Xen's efi=attr=uc does. Very interesting! Thanks for doing the research. > So, to improve Xen's hardware/firmware compatibility, I have two ideas: > > 1. Make efi=attr=uc the default (it's still possible to disable it with > efi=attr=no). I'd be very much in favor of that too (especially since it seems to match Linux behaviour) What do others think? > 2. Map type=11 (MMIO) as UC, unless attributes specify otherwise. This seems to be the subset of the #1 option. As such -- perhaps it is "safer" than a wholesale efi=attr=uc but at the same time Linux behaviour gives me pretty good confidence that we should probably be safe, no? > Any preference? I can prepare a patch for either version. But I guess > it's too late for getting it into 4.13. Good question as well. Thanks, Roman. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Tue, Nov 26, 2019 at 09:56:25AM -0800, Roman Shaposhnik wrote: > Hi Marek, after applying Jan's patch I'm making much further progress. > Xen boots fine and Dom0 seems to be OK (more tests are needed tho on > my end). > > I'm attaching the logs from Xen and Dom0. > > At this point it seems that adding efi=attr=uc is a better option for > these boxes than a wholesale efi=no-rs > > Question #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was > supposed to cover by default (so I don't have to add efi=attr=uc)? No, this looks like some different firmware (?) issue. > Question #2: is there any downside to *always* specifying efi=attr=uc? > Even for servers that, strictly speaking, don't need it? TL;DR: It should be fine. It is what Linux does too. Details: Lets take a look why 'efi=attr=uc' helps, and how can we make it work out of the box: The issue is about memory marked as type=11 (EfiMemoryMappedIO) with attr=8000 (EFI_MEMORY_RUNTIME). Indeed none of cachability attribute is defined. For the record, defined attributes are (UEFI spec .6): EFI_MEMORY_UC Memory cacheability attribute: The memory region supports being configured as not cacheable. EFI_MEMORY_WC Memory cacheability attribute: The memory region supports being configured as write combining. EFI_MEMORY_WT Memory cacheability attribute: The memory region supports being configured as cacheable with a “write through” policy. Writes that hit in the cache will also be written to main memory. EFI_MEMORY_WB Memory cacheability attribute: The memory region supports being configured as cacheable with a “write back” policy. Reads and writes that hit in the cache do not propagate to main memory. Dirty data is written back to main memory when a new cache line is allocated. EFI_MEMORY_UCE Memory cacheability attribute: The memory region supports being configured as not cacheable, exported, and supports the “fetch and add” semaphore mechanism. My reading of UEFI spec doesn't give much hints what to do with memory mappings without any cachability attribute. The only related info I've found is about EfiMemoryMappedIO: This memory is not used by the OS. All system memory-mapped IO information should come from ACPI tables. So, maybe there is some more info? Anyway, if I understand correctly, MMIO region should be mapped as UC, right? I've also taken look at what Linux does. And basically, the only bit Linux care about is EFI_MEMORY_WB - if it's absent, then set the region as uncachable (page cache disabled bit in page table entry). So, basically Linux by default does what Xen's efi=attr=uc does. So, to improve Xen's hardware/firmware compatibility, I have two ideas: 1. Make efi=attr=uc the default (it's still possible to disable it with efi=attr=no). 2. Map type=11 (MMIO) as UC, unless attributes specify otherwise. Any preference? I can prepare a patch for either version. But I guess it's too late for getting it into 4.13. -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab A: Because it messes up the order in which people normally read text. Q: Why is top-posting such a bad thing? signature.asc Description: PGP signature ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
Hi Marek, after applying Jan's patch I'm making much further progress. Xen boots fine and Dom0 seems to be OK (more tests are needed tho on my end). I'm attaching the logs from Xen and Dom0. At this point it seems that adding efi=attr=uc is a better option for these boxes than a wholesale efi=no-rs Question #1: is this something that EFI_SET_VIRTUAL_ADDRESS_MAP was supposed to cover by default (so I don't have to add efi=attr=uc)? Question #2: is there any downside to *always* specifying efi=attr=uc? Even for servers that, strictly speaking, don't need it? Thanks, Roman. On Mon, Nov 25, 2019 at 11:02 PM Roman Shaposhnik wrote: > > On Mon, Nov 25, 2019 at 7:55 PM Marek Marczykowski-Górecki > wrote: > > > > On Mon, Nov 25, 2019 at 07:44:03PM -0800, Roman Shaposhnik wrote: > > > On Sun, Nov 24, 2019 at 4:48 PM Marek Marczykowski-Górecki > > > wrote: > > > > Do you have by > > > > a chance messages of that crash (without efi=no-rs, but with > > > > EFI_SET_VIRTUAL_ADDRESS_MAP enabled)? Or even a photo if no serial > > > > output is > > > > available? > > > > > > With my awesome soldering skills ;-) I managed to rig a serial console. > > > > > > Output is attached. Please let me know if you'd like me to run any > > > other experiments. > > > > Looks helpful, lets try to do something: > > > > > Xen 4.13.0-rc > > > (XEN) Xen version 4.13.0-rc (@) (gcc (Alpine 6.4.0) 6.4.0) debug=y Tue > > > Nov 26 03:19:38 UTC 2019 > > > (XEN) Latest ChangeSet: > > > (XEN) build-id: 07aa9f711fe09a91be2588ee7df10d93ebe34c80 > > > (XEN) Bootloader: GRUB 2.03 > > > (XEN) Command line: com1=115200,8n1 console=com1 loglvl=all noreboot > > > dom0_mem=640M,max:640M dom0_max_vcpus=1 dom0_vcpus_pin smt=false > > (...) > > > (XEN) EFI memory map: > > (...) > > > (XEN) 077587000-0775f4fff type=5 attr=800f > > > > This is code that crashes - runtime services code, so somewhere with > > actual UEFI code. > > Yup -- that was my hunch with adding efi=no-rs option. > > > (...) > > > (XEN) 0ff90-0 type=11 attr=8000 > > > (XEN) Unknown cachability for MFNs 0xff900-0xf > > > > The faulting address is in this range. And because of unknown > > cachability, it isn't mapped. Try adding 'efi=attr=uc' to the Xen > > cmdline. > > Feels like we're getting exactly the same failure. Log attached. > > Thanks, > Roman. (XEN) Xen version 4.13.0-rc (@) (gcc (Alpine 6.4.0) 6.4.0) debug=y Tue Nov 26 16:59:33 UTC 2019 (XEN) Latest ChangeSet: (XEN) build-id: 65fa4bd58d340488ec6963bd8ca5418747541fe5 (XEN) Bootloader: GRUB 2.03 (XEN) Command line: com1=115200,8n1 console=com1 efi=attr=uc loglvl=all noreboot dom0_mem=640M,max:640M dom0_max_vcpus=1 dom0_vcpus_pin smt=false (XEN) Xen image load base address: 0x70e0 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) Disc information: (XEN) Found 0 MBR signatures (XEN) Found 1 EDD information structures (XEN) EFI RAM map: (XEN) - 0003f000 (usable) (XEN) 0003f000 - 0004 (ACPI NVS) (XEN) 0004 - 000a (usable) (XEN) 0010 - 2000 (usable) (XEN) 2000 - 2010 (reserved) (XEN) 2010 - 76ccb000 (usable) (XEN) 76ccb000 - 76d43000 (reserved) (XEN) 76d43000 - 76d54000 (ACPI data) (XEN) 76d54000 - 772de000 (ACPI NVS) (XEN) 772de000 - 775f5000 (reserved) (XEN) 775f5000 - 775f6000 (usable) (XEN) 775f6000 - 77638000 (reserved) (XEN) 77638000 - 789e5000 (usable) (XEN) 789e5000 - 78ffa000 (reserved) (XEN) 78ffa000 - 7900 (usable) (XEN) e000 - f000 (reserved) (XEN) fec0 - fec01000 (reserved) (XEN) fed01000 - fed02000 (reserved) (XEN) fed03000 - fed04000 (reserved) (XEN) fed08000 - fed09000 (reserved) (XEN) fed0c000 - fed1 (reserved) (XEN) fed1c000 - fed1d000 (reserved) (XEN) fee0 - fee01000 (reserved) (XEN) fef0 - ff00 (reserved) (XEN) ff90 - 0001 (reserved) (XEN) System RAM: 1919MB (1965176kB) (XEN) ACPI: RSDP 76D46000, 0024 (r2 DELL) (XEN) ACPI: XSDT 76D46088, 0094 (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FACP 76D52560, 010C (r5 DELL AS09 1072009 AMI 10013) (XEN) ACPI: DSDT 76D461B0, C3AF (r2 DELL AS09 1072009 INTL 20120913) (XEN) ACPI: FACS 772DDE80, 0040 (XEN) ACPI: APIC 76D52670, 0068 (r3 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FPDT 76D526D8, 0044 (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FIDT 76D52720, 009C (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: MCFG 76D527C0, 003C (r1 DELL AS09 1072009 MSFT 97) (XEN) ACPI: LPIT 76D52800, 0104 (r1 DELL AS09
Re: [Xen-devel] UEFI support on Dell boxes
On Tue, Nov 26, 2019 at 12:31 AM Jan Beulich wrote: > > On 26.11.2019 08:02, Roman Shaposhnik wrote: > > On Mon, Nov 25, 2019 at 7:55 PM Marek Marczykowski-Górecki > > wrote: > >> On Mon, Nov 25, 2019 at 07:44:03PM -0800, Roman Shaposhnik wrote: > >>> (XEN) 0ff90-0 type=11 attr=8000 > >>> (XEN) Unknown cachability for MFNs 0xff900-0xf > >> > >> The faulting address is in this range. And because of unknown > >> cachability, it isn't mapped. Try adding 'efi=attr=uc' to the Xen > >> cmdline. > > > > Feels like we're getting exactly the same failure. Log attached. > > Clearly the option hasn't been taking effect. Could you please > retry with this fix > https://lists.xenproject.org/archives/html/xen-devel/2019-11/msg01494.html > in place? This works very well indeed! I acked it in the patch thread. Thanks, Roman. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes
On 26.11.2019 08:02, Roman Shaposhnik wrote: > On Mon, Nov 25, 2019 at 7:55 PM Marek Marczykowski-Górecki > wrote: >> On Mon, Nov 25, 2019 at 07:44:03PM -0800, Roman Shaposhnik wrote: >>> (XEN) 0ff90-0 type=11 attr=8000 >>> (XEN) Unknown cachability for MFNs 0xff900-0xf >> >> The faulting address is in this range. And because of unknown >> cachability, it isn't mapped. Try adding 'efi=attr=uc' to the Xen >> cmdline. > > Feels like we're getting exactly the same failure. Log attached. Clearly the option hasn't been taking effect. Could you please retry with this fix https://lists.xenproject.org/archives/html/xen-devel/2019-11/msg01494.html in place? Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Mon, Nov 25, 2019 at 7:55 PM Marek Marczykowski-Górecki wrote: > > On Mon, Nov 25, 2019 at 07:44:03PM -0800, Roman Shaposhnik wrote: > > On Sun, Nov 24, 2019 at 4:48 PM Marek Marczykowski-Górecki > > wrote: > > > Do you have by > > > a chance messages of that crash (without efi=no-rs, but with > > > EFI_SET_VIRTUAL_ADDRESS_MAP enabled)? Or even a photo if no serial output > > > is > > > available? > > > > With my awesome soldering skills ;-) I managed to rig a serial console. > > > > Output is attached. Please let me know if you'd like me to run any > > other experiments. > > Looks helpful, lets try to do something: > > > Xen 4.13.0-rc > > (XEN) Xen version 4.13.0-rc (@) (gcc (Alpine 6.4.0) 6.4.0) debug=y Tue Nov > > 26 03:19:38 UTC 2019 > > (XEN) Latest ChangeSet: > > (XEN) build-id: 07aa9f711fe09a91be2588ee7df10d93ebe34c80 > > (XEN) Bootloader: GRUB 2.03 > > (XEN) Command line: com1=115200,8n1 console=com1 loglvl=all noreboot > > dom0_mem=640M,max:640M dom0_max_vcpus=1 dom0_vcpus_pin smt=false > (...) > > (XEN) EFI memory map: > (...) > > (XEN) 077587000-0775f4fff type=5 attr=800f > > This is code that crashes - runtime services code, so somewhere with > actual UEFI code. Yup -- that was my hunch with adding efi=no-rs option. > (...) > > (XEN) 0ff90-0 type=11 attr=8000 > > (XEN) Unknown cachability for MFNs 0xff900-0xf > > The faulting address is in this range. And because of unknown > cachability, it isn't mapped. Try adding 'efi=attr=uc' to the Xen > cmdline. Feels like we're getting exactly the same failure. Log attached. Thanks, Roman. Xen 4.13.0-rc (XEN) Xen version 4.13.0-rc (@) (gcc (Alpine 6.4.0) 6.4.0) debug=y Tue Nov 26 03:19:38 UTC 2019 (XEN) Latest ChangeSet: (XEN) build-id: 07aa9f711fe09a91be2588ee7df10d93ebe34c80 (XEN) Bootloader: GRUB 2.03 (XEN) Command line: com1=115200,8n1 console=com1 loglvl=all noreboot efi=attr=uc dom0_mem=640M,max:640M dom0_max_vcpus=1 dom0_vcpus_pin smt=false (XEN) Xen image load base address: 0x70e0 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) Disc information: (XEN) Found 0 MBR signatures (XEN) Found 1 EDD information structures (XEN) EFI RAM map: (XEN) - 0003f000 (usable) (XEN) 0003f000 - 0004 (ACPI NVS) (XEN) 0004 - 000a (usable) (XEN) 0010 - 2000 (usable) (XEN) 2000 - 2010 (reserved) (XEN) 2010 - 76ccb000 (usable) (XEN) 76ccb000 - 76d43000 (reserved) (XEN) 76d43000 - 76d54000 (ACPI data) (XEN) 76d54000 - 772de000 (ACPI NVS) (XEN) 772de000 - 775f5000 (reserved) (XEN) 775f5000 - 775f6000 (usable) (XEN) 775f6000 - 77638000 (reserved) (XEN) 77638000 - 789e5000 (usable) (XEN) 789e5000 - 78ffa000 (reserved) (XEN) 78ffa000 - 7900 (usable) (XEN) e000 - f000 (reserved) (XEN) fec0 - fec01000 (reserved) (XEN) fed01000 - fed02000 (reserved) (XEN) fed03000 - fed04000 (reserved) (XEN) fed08000 - fed09000 (reserved) (XEN) fed0c000 - fed1 (reserved) (XEN) fed1c000 - fed1d000 (reserved) (XEN) fee0 - fee01000 (reserved) (XEN) fef0 - ff00 (reserved) (XEN) ff90 - 0001 (reserved) (XEN) System RAM: 1919MB (1965176kB) (XEN) ACPI: RSDP 76D46000, 0024 (r2 DELL) (XEN) ACPI: XSDT 76D46088, 0094 (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FACP 76D52560, 010C (r5 DELL AS09 1072009 AMI 10013) (XEN) ACPI: DSDT 76D461B0, C3AF (r2 DELL AS09 1072009 INTL 20120913) (XEN) ACPI: FACS 772DDE80, 0040 (XEN) ACPI: APIC 76D52670, 0068 (r3 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FPDT 76D526D8, 0044 (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FIDT 76D52720, 009C (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: MCFG 76D527C0, 003C (r1 DELL AS09 1072009 MSFT 97) (XEN) ACPI: LPIT 76D52800, 0104 (r1 DELL AS093 VLV2 10D) (XEN) ACPI: HPET 76D52908, 0038 (r1 DELL AS09 1072009 AMI.5) (XEN) ACPI: SSDT 76D52940, 0763 (r1 DELL AS09 3000 INTL 20061109) (XEN) ACPI: SSDT 76D530A8, 0290 (r1 DELL AS09 3000 INTL 20061109) (XEN) ACPI: SSDT 76D53338, 017A (r1 DELL AS09 3000 INTL 20061109) (XEN) ACPI: UEFI 76D534B8, 0042 (r1 DELL AS090 0) (XEN) ACPI: CSRT 76D53500, 014C (r0 DELL AS095 INTL 20120624) (XEN) ACPI: TPM2 76D53650, 0034 (r3Tpm2Tabl1 AMI 0) (XEN) ACPI: SSDT 76D53688, 00C9 (r1 MSFT RHPROXY1 INTL 20120913) (XEN) No NUMA configuration found (XEN) Faking a node at -7900 (XEN)
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Mon, Nov 25, 2019 at 07:44:03PM -0800, Roman Shaposhnik wrote:
> On Sun, Nov 24, 2019 at 4:48 PM Marek Marczykowski-Górecki
> wrote:
> > Do you have by
> > a chance messages of that crash (without efi=no-rs, but with
> > EFI_SET_VIRTUAL_ADDRESS_MAP enabled)? Or even a photo if no serial output is
> > available?
>
> With my awesome soldering skills ;-) I managed to rig a serial console.
>
> Output is attached. Please let me know if you'd like me to run any
> other experiments.
Looks helpful, lets try to do something:
> Xen 4.13.0-rc
> (XEN) Xen version 4.13.0-rc (@) (gcc (Alpine 6.4.0) 6.4.0) debug=y Tue Nov
> 26 03:19:38 UTC 2019
> (XEN) Latest ChangeSet:
> (XEN) build-id: 07aa9f711fe09a91be2588ee7df10d93ebe34c80
> (XEN) Bootloader: GRUB 2.03
> (XEN) Command line: com1=115200,8n1 console=com1 loglvl=all noreboot
> dom0_mem=640M,max:640M dom0_max_vcpus=1 dom0_vcpus_pin smt=false
(...)
> (XEN) EFI memory map:
(...)
> (XEN) 077587000-0775f4fff type=5 attr=800f
This is code that crashes - runtime services code, so somewhere with
actual UEFI code.
(...)
> (XEN) 0ff90-0 type=11 attr=8000
> (XEN) Unknown cachability for MFNs 0xff900-0xf
The faulting address is in this range. And because of unknown
cachability, it isn't mapped. Try adding 'efi=attr=uc' to the Xen
cmdline.
(...)
> (XEN) Xen call trace:
> (XEN)[<775e0d21>] R 775e0d21
> (XEN)[<775ddb8e>] S 775ddb8e
> (XEN)[<>] F
> (XEN)[<7fff>] F 7fff
> (XEN)
> (XEN) Pagetable walk from ff920020:
> (XEN) L4[0x000] = 787c0063
> (XEN) L3[0x003] = 71298063
> (XEN) L2[0x1fc] =
> (XEN)
> (XEN)
> (XEN) Panic on CPU 0:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=]
> (XEN) Faulting linear address: ff920020
> (XEN)
> (XEN)
> (XEN) Manual reset required ('noreboot' specified)
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] UEFI support on Dell boxes (was: Re: Status of 4.13)
On Sun, Nov 24, 2019 at 4:48 PM Marek Marczykowski-Górecki wrote: > > On Fri, Nov 22, 2019 at 10:00:13PM -0800, Roman Shaposhnik wrote: > > 3. Bad news: Marek's suggestion didn't work on Dell product line (and yes > > I double checked that I built it correctly). > > > > So... when it comes to RC2 regression -- we're all good. > > > > But since we're here anyway -- I'm wondering if anyone would be > > interested in helping me figure out why Xen on those Dell boxes coredumps > > without efi=no-rs ? > > > > Marek, any chance I can interest you in helping me a bit here? ;-) > > Yes, I am interested in helping with UEFI state there. Thanks! That's very much appreciated! Btw, I'll keep CCing xen-devel in case anyone else is interested in this conversation. > Do you have by > a chance messages of that crash (without efi=no-rs, but with > EFI_SET_VIRTUAL_ADDRESS_MAP enabled)? Or even a photo if no serial output is > available? With my awesome soldering skills ;-) I managed to rig a serial console. Output is attached. Please let me know if you'd like me to run any other experiments. Thanks, Roman. Xen 4.13.0-rc (XEN) Xen version 4.13.0-rc (@) (gcc (Alpine 6.4.0) 6.4.0) debug=y Tue Nov 26 03:19:38 UTC 2019 (XEN) Latest ChangeSet: (XEN) build-id: 07aa9f711fe09a91be2588ee7df10d93ebe34c80 (XEN) Bootloader: GRUB 2.03 (XEN) Command line: com1=115200,8n1 console=com1 loglvl=all noreboot dom0_mem=640M,max:640M dom0_max_vcpus=1 dom0_vcpus_pin smt=false (XEN) Xen image load base address: 0x70e0 (XEN) Video information: (XEN) VGA is text mode 80x25, font 8x16 (XEN) Disc information: (XEN) Found 0 MBR signatures (XEN) Found 1 EDD information structures (XEN) EFI RAM map: (XEN) - 0003f000 (usable) (XEN) 0003f000 - 0004 (ACPI NVS) (XEN) 0004 - 000a (usable) (XEN) 0010 - 2000 (usable) (XEN) 2000 - 2010 (reserved) (XEN) 2010 - 76ccb000 (usable) (XEN) 76ccb000 - 76d43000 (reserved) (XEN) 76d43000 - 76d54000 (ACPI data) (XEN) 76d54000 - 772de000 (ACPI NVS) (XEN) 772de000 - 775f5000 (reserved) (XEN) 775f5000 - 775f6000 (usable) (XEN) 775f6000 - 77638000 (reserved) (XEN) 77638000 - 789e5000 (usable) (XEN) 789e5000 - 78ffa000 (reserved) (XEN) 78ffa000 - 7900 (usable) (XEN) e000 - f000 (reserved) (XEN) fec0 - fec01000 (reserved) (XEN) fed01000 - fed02000 (reserved) (XEN) fed03000 - fed04000 (reserved) (XEN) fed08000 - fed09000 (reserved) (XEN) fed0c000 - fed1 (reserved) (XEN) fed1c000 - fed1d000 (reserved) (XEN) fee0 - fee01000 (reserved) (XEN) fef0 - ff00 (reserved) (XEN) ff90 - 0001 (reserved) (XEN) System RAM: 1919MB (1965176kB) (XEN) ACPI: RSDP 76D46000, 0024 (r2 DELL) (XEN) ACPI: XSDT 76D46088, 0094 (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FACP 76D52560, 010C (r5 DELL AS09 1072009 AMI 10013) (XEN) ACPI: DSDT 76D461B0, C3AF (r2 DELL AS09 1072009 INTL 20120913) (XEN) ACPI: FACS 772DDE80, 0040 (XEN) ACPI: APIC 76D52670, 0068 (r3 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FPDT 76D526D8, 0044 (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: FIDT 76D52720, 009C (r1 DELL AS09 1072009 AMI 10013) (XEN) ACPI: MCFG 76D527C0, 003C (r1 DELL AS09 1072009 MSFT 97) (XEN) ACPI: LPIT 76D52800, 0104 (r1 DELL AS093 VLV2 10D) (XEN) ACPI: HPET 76D52908, 0038 (r1 DELL AS09 1072009 AMI.5) (XEN) ACPI: SSDT 76D52940, 0763 (r1 DELL AS09 3000 INTL 20061109) (XEN) ACPI: SSDT 76D530A8, 0290 (r1 DELL AS09 3000 INTL 20061109) (XEN) ACPI: SSDT 76D53338, 017A (r1 DELL AS09 3000 INTL 20061109) (XEN) ACPI: UEFI 76D534B8, 0042 (r1 DELL AS090 0) (XEN) ACPI: CSRT 76D53500, 014C (r0 DELL AS095 INTL 20120624) (XEN) ACPI: TPM2 76D53650, 0034 (r3Tpm2Tabl1 AMI 0) (XEN) ACPI: SSDT 76D53688, 00C9 (r1 MSFT RHPROXY1 INTL 20120913) (XEN) No NUMA configuration found (XEN) Faking a node at -7900 (XEN) Domain heap initialised (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 55 (0x37), Stepping 9 (raw 00030679) (XEN) SMBIOS 3.0 present. (XEN) DMI 3.0 present. (XEN) Using APIC driver default (XEN) ACPI: PM-Timer IO Port: 0x408 (32 bits) (XEN) ACPI: v5 SLEEP INFO: control[0:0], status[0:0] (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:404,1:0], pm1x_evt[1:400,1:0] (XEN) ACPI: 32/64X FACS address mismatch in FADT - 772dde80/, using 32 (XEN) ACPI: wakeup_vec[772dde8c], vec_size[20] (XEN) ACPI: Local APIC address
