Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Thu, Aug 16, 2018 at 10:43:54AM -0600, Tamas K Lengyel wrote: > I double checked and the option is set properly but I'm still getting > the same non-present entry faults as before. At the moment I don't > have serial access so not sure how to verify that the option took > effect. This is my boot info: > > [pvh] > options=loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M > dom0_max_vcpus=2 sched=null dom0=pvh iommu=required,debug > dom0-iommu=relaxed console=vga > rmrr=0x5f48385=0:0:2.0,0x216183=0:0:2.0,0x70b6d19=0:0:2.0 This is not the correct format. It's: rmrr=0x5f48385=0:0:2.0;0x216183=0:0:2.0;0x70b6d19=0:0:2.0 Note the ';' between rmrr entries. Roger. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
>>> On 16.08.18 at 18:43, wrote: > Booting just linux with both options works just fine, I grepped the > dmesg log for the device that causes the issues with Xen but there are > no errors in the log: Well, this as well as ... > I double checked and the option is set properly but I'm still getting > the same non-present entry faults as before. At the moment I don't > have serial access so not sure how to verify that the option took > effect. This is my boot info: > > [pvh] > options=loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M > dom0_max_vcpus=2 sched=null dom0=pvh iommu=required,debug > dom0-iommu=relaxed console=vga > rmrr=0x5f48385=0:0:2.0,0x216183=0:0:2.0,0x70b6d19=0:0:2.0 > kernel=vmlinuz-4.18.0-rc8 root=/dev/mapper/ubuntu--vg-root ro quiet > splash vt.handoff=1 > ramdisk=initrd.img-4.18.0-rc8 ... this (option looks fine to me) suggest that some debugging is going to be on order on that system, which I understand will be at least difficult without serial console. Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Wed, Aug 15, 2018 at 12:40 AM Jan Beulich wrote: > > >>> On 15.08.18 at 03:00, wrote: > > On Wed, Aug 8, 2018 at 3:54 AM Jan Beulich wrote: > >> > >> >>> On 08.08.18 at 10:25, wrote: > >> > On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote: > >> >> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel > >> >> wrote: > >> >> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > >> >> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > >> >> (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > >> >> 428f926000, iommu reg = 82c000a0c000 > >> >> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > >> >> (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 428f926 > >> >> (XEN) root_entry[00] = 43aaae001 > >> >> (XEN) context[10] = 2_43cf92001 > >> >> (XEN) l4[000] = 9c043cf91107 > >> >> (XEN) l3[10a] = 8000 > >> >> (XEN) l3[10a] not present > >> >> > >> >> The fault is repeated a million times per second and the system is > >> >> pretty much stalled. > >> > > >> > As Jan says, this page is outside of any range in the memory map. I > >> > wonder however what's in there. > >> > > >> > I think (also seeing the PV issues) you should bring this up with the > >> > driver maintainers, it might actually be a bug that the driver is > >> > trying to access such address. > >> > >> Right, especially considering that the address does not appear to be > >> invariant, I suspect the driver sets up some I/O from (presumably) > >> uninitialized data. This goes unnoticed until DMA translation comes > >> into play. Tamas - did you try enabling DMA translation in Linux > >> when not running on top of Xen? Depending on the > >> CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the > >> default. > > > > I checked and this kernel option is not enabled. Are you suggesting to > > try booting just Linux with this option enabled to see what happens? > > Well, you don't need to rebuild the kernel with the config option > enabled, you can use what you have and add "intel_iommu=on" > to the kernel command line. In case this works without any faults, > changing to "intel_iommu=on,igfx_off" may or may not provide > further insight. Booting just linux with both options works just fine, I grepped the dmesg log for the device that causes the issues with Xen but there are no errors in the log: # dmesg | grep "00\:02\.0" [0.165669] pci :00:02.0: [8086:5916] type 00 class 0x03 [0.165681] pci :00:02.0: reg 0x10: [mem 0xdb00-0xdbff 64bit] [0.165687] pci :00:02.0: reg 0x18: [mem 0x9000-0x9fff 64bit pref] [0.165692] pci :00:02.0: reg 0x20: [io 0xf000-0xf03f] [0.165707] pci :00:02.0: BAR 2: assigned to efifb [0.184223] pci :00:02.0: vgaarb: setting as boot VGA device [0.184223] pci :00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none [0.184223] pci :00:02.0: vgaarb: bridge control possible [0.248925] pci :00:02.0: Video device with shadowed ROM at [mem 0x000c-0x000d] [1.896026] i915 :00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem [1.903881] [drm] Initialized i915 1.6.0 20180514 for :00:02.0 on minor 0 [3.271519] i915 :00:02.0: fb0: inteldrmfb frame buffer device [ 12.519683] snd_hda_intel :00:1f.3: bound :00:02.0 (ops i915_audio_component_bind_ops [i915]) > >> > In the meantime, you can try to add to the command line: > >> > > >> > rmrr=0x428f926=0:0:2.0 > >> > > >> > In order to force an iommu mapping of this address. > >> > >> I suspect this won't help much. > >> > > > > The mfn is not always the same but seems to be at least on some > > occasion. > > I expect that's because many things early at boot go pretty > deterministically. > > > I managed to reboot with the right rmrr= option set but I'm > > still getting the same faults over and over for that mfn I set in the > > rmrr= option. > > Hmm, and the faults still show non-present entries for these? This > options is at risk of getting misspelled - did you check that the > option you've specified actually took effect? I double checked and the option is set properly but I'm still getting the same non-present entry faults as before. At the moment I don't have serial access so not sure how to verify that the option took effect. This is my boot info: [pvh] options=loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null dom0=pvh iommu=required,debug dom0-iommu=relaxed console=vga rmrr=0x5f48385=0:0:2.0,0x216183=0:0:2.0,0x70b6d19=0:0:2.0 kernel=vmlinuz-4.18.0-rc8 root=/dev/mapper/ubuntu--vg-root ro quiet splash vt.handoff=1 ramdisk=initrd.img-4.18.0-rc8 Tamas ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
>>> On 15.08.18 at 03:00, wrote: > On Wed, Aug 8, 2018 at 3:54 AM Jan Beulich wrote: >> >> >>> On 08.08.18 at 10:25, wrote: >> > On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote: >> >> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel >> >> wrote: >> >> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow >> >> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault >> >> (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr >> >> 428f926000, iommu reg = 82c000a0c000 >> >> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set >> >> (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 428f926 >> >> (XEN) root_entry[00] = 43aaae001 >> >> (XEN) context[10] = 2_43cf92001 >> >> (XEN) l4[000] = 9c043cf91107 >> >> (XEN) l3[10a] = 8000 >> >> (XEN) l3[10a] not present >> >> >> >> The fault is repeated a million times per second and the system is >> >> pretty much stalled. >> > >> > As Jan says, this page is outside of any range in the memory map. I >> > wonder however what's in there. >> > >> > I think (also seeing the PV issues) you should bring this up with the >> > driver maintainers, it might actually be a bug that the driver is >> > trying to access such address. >> >> Right, especially considering that the address does not appear to be >> invariant, I suspect the driver sets up some I/O from (presumably) >> uninitialized data. This goes unnoticed until DMA translation comes >> into play. Tamas - did you try enabling DMA translation in Linux >> when not running on top of Xen? Depending on the >> CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the >> default. > > I checked and this kernel option is not enabled. Are you suggesting to > try booting just Linux with this option enabled to see what happens? Well, you don't need to rebuild the kernel with the config option enabled, you can use what you have and add "intel_iommu=on" to the kernel command line. In case this works without any faults, changing to "intel_iommu=on,igfx_off" may or may not provide further insight. >> > In the meantime, you can try to add to the command line: >> > >> > rmrr=0x428f926=0:0:2.0 >> > >> > In order to force an iommu mapping of this address. >> >> I suspect this won't help much. >> > > The mfn is not always the same but seems to be at least on some > occasion. I expect that's because many things early at boot go pretty deterministically. > I managed to reboot with the right rmrr= option set but I'm > still getting the same faults over and over for that mfn I set in the > rmrr= option. Hmm, and the faults still show non-present entries for these? This options is at risk of getting misspelled - did you check that the option you've specified actually took effect? Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Wed, Aug 8, 2018 at 3:54 AM Jan Beulich wrote: > > >>> On 08.08.18 at 10:25, wrote: > > On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote: > >> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel > >> wrote: > >> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > >> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > >> (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > >> 428f926000, iommu reg = 82c000a0c000 > >> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > >> (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 428f926 > >> (XEN) root_entry[00] = 43aaae001 > >> (XEN) context[10] = 2_43cf92001 > >> (XEN) l4[000] = 9c043cf91107 > >> (XEN) l3[10a] = 8000 > >> (XEN) l3[10a] not present > >> > >> The fault is repeated a million times per second and the system is > >> pretty much stalled. > > > > As Jan says, this page is outside of any range in the memory map. I > > wonder however what's in there. > > > > I think (also seeing the PV issues) you should bring this up with the > > driver maintainers, it might actually be a bug that the driver is > > trying to access such address. > > Right, especially considering that the address does not appear to be > invariant, I suspect the driver sets up some I/O from (presumably) > uninitialized data. This goes unnoticed until DMA translation comes > into play. Tamas - did you try enabling DMA translation in Linux > when not running on top of Xen? Depending on the > CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the > default. I checked and this kernel option is not enabled. Are you suggesting to try booting just Linux with this option enabled to see what happens? > > > In the meantime, you can try to add to the command line: > > > > rmrr=0x428f926=0:0:2.0 > > > > In order to force an iommu mapping of this address. > > I suspect this won't help much. > The mfn is not always the same but seems to be at least on some occasion. I managed to reboot with the right rmrr= option set but I'm still getting the same faults over and over for that mfn I set in the rmrr= option. Tamas ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
>>> On 08.08.18 at 10:25, wrote: > On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote: >> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel >> wrote: >> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow >> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault >> (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr >> 428f926000, iommu reg = 82c000a0c000 >> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set >> (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 428f926 >> (XEN) root_entry[00] = 43aaae001 >> (XEN) context[10] = 2_43cf92001 >> (XEN) l4[000] = 9c043cf91107 >> (XEN) l3[10a] = 8000 >> (XEN) l3[10a] not present >> >> The fault is repeated a million times per second and the system is >> pretty much stalled. > > As Jan says, this page is outside of any range in the memory map. I > wonder however what's in there. > > I think (also seeing the PV issues) you should bring this up with the > driver maintainers, it might actually be a bug that the driver is > trying to access such address. Right, especially considering that the address does not appear to be invariant, I suspect the driver sets up some I/O from (presumably) uninitialized data. This goes unnoticed until DMA translation comes into play. Tamas - did you try enabling DMA translation in Linux when not running on top of Xen? Depending on the CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the default. > In the meantime, you can try to add to the command line: > > rmrr=0x428f926=0:0:2.0 > > In order to force an iommu mapping of this address. I suspect this won't help much. Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote: > On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel > wrote: > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > 428f926000, iommu reg = 82c000a0c000 > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 428f926 > (XEN) root_entry[00] = 43aaae001 > (XEN) context[10] = 2_43cf92001 > (XEN) l4[000] = 9c043cf91107 > (XEN) l3[10a] = 8000 > (XEN) l3[10a] not present > > The fault is repeated a million times per second and the system is > pretty much stalled. As Jan says, this page is outside of any range in the memory map. I wonder however what's in there. I think (also seeing the PV issues) you should bring this up with the driver maintainers, it might actually be a bug that the driver is trying to access such address. In the meantime, you can try to add to the command line: rmrr=0x428f926=0:0:2.0 In order to force an iommu mapping of this address. Thanks, Roger. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Tue, Aug 7, 2018 at 10:45 AM Tamas K Lengyel wrote: > > On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel > wrote: > > > > On Tue, Aug 7, 2018 at 9:09 AM Roger Pau Monné wrote: > > > > > > On Tue, Aug 07, 2018 at 08:45:07AM -0600, Tamas K Lengyel wrote: > > > > On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné > > > > wrote: > > > > > > > > > > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote: > > > > > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne > > > > > > wrote: > > > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > The following series implement a workaround for missing RMRR > > > > > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping > > > > > > > VTd > > > > > > > option. > > > > > > > > > > > > > > The PVH workaround identity maps all regions marked as reserved > > > > > > > in the > > > > > > > memory map. > > > > > > > > > > > > > > Note that this workaround is enabled by default on Intel > > > > > > > hardware. It's > > > > > > > also available to AMD hardware, although it's disabled by default > > > > > > > in > > > > > > > that case. > > > > > > > > > > > > > > The series can be found at: > > > > > > > > > > > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3 > > > > > > > > > > > > > > Thanks, Roger. > > > > > > > Roger Pau Monne (4): > > > > > > > iommu: introduce dom0-iommu option > > > > > > > iommu: make iommu_inclusive_mapping a suboption of dom0-iommu > > > > > > > dom0/pvh: change the order of the MMCFG initialization > > > > > > > x86/iommu: add reserved dom0-iommu option to map reserved memory > > > > > > > ranges > > > > > > > > > > > > > > docs/misc/xen-command-line.markdown | 47 +++ > > > > > > > xen/arch/x86/hvm/dom0_build.c | 9 ++- > > > > > > > xen/arch/x86/hvm/io.c | 5 ++ > > > > > > > xen/arch/x86/x86_64/mm.c| 3 +- > > > > > > > xen/drivers/passthrough/amd/iommu_init.c| 2 +- > > > > > > > xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++- > > > > > > > xen/drivers/passthrough/arm/iommu.c | 4 + > > > > > > > xen/drivers/passthrough/iommu.c | 62 +-- > > > > > > > xen/drivers/passthrough/vtd/extern.h| 2 - > > > > > > > xen/drivers/passthrough/vtd/iommu.c | 25 +++--- > > > > > > > xen/drivers/passthrough/vtd/x86/vtd.c | 58 +- > > > > > > > xen/drivers/passthrough/x86/iommu.c | 87 > > > > > > > + > > > > > > > xen/include/asm-x86/hvm/io.h| 3 + > > > > > > > xen/include/xen/iommu.h | 8 +- > > > > > > > 14 files changed, 240 insertions(+), 86 deletions(-) > > > > > > > > > > > > > > -- > > > > > > > > > > > > Hi Roger, > > > > > > I gave this branch a spin on a Dell XPS laptop booting UEFI with > > > > > > Linux > > > > > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical > > > > > > keyboard of the laptop stopped working, it works no problem with > > > > > > just > > > > > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After > > > > > > running for a minute or two the system starts to slow down to the > > > > > > point where it becomes unresponsive. The xl dmesg log is filled with > > > > > > this error: > > > > > > > > > > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > > > > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > > > > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > > > > > > 4625f3a000, iommu reg = 82c00181c000 > > > > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > > > > > > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a > > > > > > > > > > Is the gmfn always the same (0x4625f3a)? > > > > > > > > > > > (XEN) root_entry[00] = 273a18001 > > > > > > (XEN) context[10] = 2_27ba35001 > > > > > > (XEN) l4[000] = 9c027ba34107 > > > > > > (XEN) l3[118] = 8000 > > > > > > (XEN) l3[118] not present > > > > > > > > > > Can you also paste the full xl dmesg log? I'm specially interested in > > > > > the memory map of the machine which is printed quite early during Xen > > > > > boot. > > > > > > > > > > > > > Unfortunately I don't have serial access on this laptop and "xl dmesg" > > > > gets completely filled with that error so the beginning of the log is > > > > lost by the time I get a terminal in dom0. > > > > > > You can get the memory map while booting in PV mode, it's going to be > > > exactly the same regardless of whether Dom0 is PV or PVH. > > > > This is the PV dmesg: > > > > (XEN) Xen version 4.12-unstable (dr@) (gcc (Debian 7.3.0-19) 7.3.0) > > debug=y Mon Aug 6 13:42:42 MDT 2018 > > (XEN) Latest ChangeSet: Fri Aug 3 10:01:36 2018 +0200 git:ddba1c2b1f > > (XEN) Bootloader: EFI > > (XEN) Command line: loglvl=all guest_loglvl=all > > dom0_mem=4096M,max:4096M dom0_max_vcpus=2
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel wrote: > > On Tue, Aug 7, 2018 at 9:09 AM Roger Pau Monné wrote: > > > > On Tue, Aug 07, 2018 at 08:45:07AM -0600, Tamas K Lengyel wrote: > > > On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné > > > wrote: > > > > > > > > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote: > > > > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne > > > > > wrote: > > > > > > > > > > > > Hello, > > > > > > > > > > > > The following series implement a workaround for missing RMRR > > > > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping > > > > > > VTd > > > > > > option. > > > > > > > > > > > > The PVH workaround identity maps all regions marked as reserved in > > > > > > the > > > > > > memory map. > > > > > > > > > > > > Note that this workaround is enabled by default on Intel hardware. > > > > > > It's > > > > > > also available to AMD hardware, although it's disabled by default in > > > > > > that case. > > > > > > > > > > > > The series can be found at: > > > > > > > > > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3 > > > > > > > > > > > > Thanks, Roger. > > > > > > Roger Pau Monne (4): > > > > > > iommu: introduce dom0-iommu option > > > > > > iommu: make iommu_inclusive_mapping a suboption of dom0-iommu > > > > > > dom0/pvh: change the order of the MMCFG initialization > > > > > > x86/iommu: add reserved dom0-iommu option to map reserved memory > > > > > > ranges > > > > > > > > > > > > docs/misc/xen-command-line.markdown | 47 +++ > > > > > > xen/arch/x86/hvm/dom0_build.c | 9 ++- > > > > > > xen/arch/x86/hvm/io.c | 5 ++ > > > > > > xen/arch/x86/x86_64/mm.c| 3 +- > > > > > > xen/drivers/passthrough/amd/iommu_init.c| 2 +- > > > > > > xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++- > > > > > > xen/drivers/passthrough/arm/iommu.c | 4 + > > > > > > xen/drivers/passthrough/iommu.c | 62 +-- > > > > > > xen/drivers/passthrough/vtd/extern.h| 2 - > > > > > > xen/drivers/passthrough/vtd/iommu.c | 25 +++--- > > > > > > xen/drivers/passthrough/vtd/x86/vtd.c | 58 +- > > > > > > xen/drivers/passthrough/x86/iommu.c | 87 > > > > > > + > > > > > > xen/include/asm-x86/hvm/io.h| 3 + > > > > > > xen/include/xen/iommu.h | 8 +- > > > > > > 14 files changed, 240 insertions(+), 86 deletions(-) > > > > > > > > > > > > -- > > > > > > > > > > Hi Roger, > > > > > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux > > > > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical > > > > > keyboard of the laptop stopped working, it works no problem with just > > > > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After > > > > > running for a minute or two the system starts to slow down to the > > > > > point where it becomes unresponsive. The xl dmesg log is filled with > > > > > this error: > > > > > > > > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > > > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > > > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > > > > > 4625f3a000, iommu reg = 82c00181c000 > > > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > > > > > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a > > > > > > > > Is the gmfn always the same (0x4625f3a)? > > > > > > > > > (XEN) root_entry[00] = 273a18001 > > > > > (XEN) context[10] = 2_27ba35001 > > > > > (XEN) l4[000] = 9c027ba34107 > > > > > (XEN) l3[118] = 8000 > > > > > (XEN) l3[118] not present > > > > > > > > Can you also paste the full xl dmesg log? I'm specially interested in > > > > the memory map of the machine which is printed quite early during Xen > > > > boot. > > > > > > > > > > Unfortunately I don't have serial access on this laptop and "xl dmesg" > > > gets completely filled with that error so the beginning of the log is > > > lost by the time I get a terminal in dom0. > > > > You can get the memory map while booting in PV mode, it's going to be > > exactly the same regardless of whether Dom0 is PV or PVH. > > This is the PV dmesg: > > (XEN) Xen version 4.12-unstable (dr@) (gcc (Debian 7.3.0-19) 7.3.0) > debug=y Mon Aug 6 13:42:42 MDT 2018 > (XEN) Latest ChangeSet: Fri Aug 3 10:01:36 2018 +0200 git:ddba1c2b1f > (XEN) Bootloader: EFI > (XEN) Command line: loglvl=all guest_loglvl=all > dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null console=vga > (XEN) Xen image load base address: 0x5a20 > (XEN) Video information: > (XEN) VGA is graphics mode 3200x1800, 32 bpp > (XEN) Disc information: > (XEN) Found 0 MBR signatures > (XEN) Found 1 EDD information structures > (XEN) EFI RAM map: > (XEN) - 00058000
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
>>> On 07.08.18 at 18:04, wrote: >> > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow >> > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault >> > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr >> > > > 4625f3a000, iommu reg = 82c00181c000 >> > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set >> > > > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a So the address here is clearly outside of any RAM: > (XEN) EFI RAM map: > (XEN) - 00058000 (usable) > (XEN) 00058000 - 00059000 (reserved) > (XEN) 00059000 - 0009f000 (usable) > (XEN) 0009f000 - 0010 (reserved) > (XEN) 0010 - 5f14d000 (usable) > (XEN) 5f14d000 - 5f14e000 (ACPI NVS) > (XEN) 5f14e000 - 5f14f000 (reserved) > (XEN) 5f14f000 - 6ee89000 (usable) > (XEN) 6ee89000 - 6f214000 (reserved) > (XEN) 6f214000 - 6f258000 (ACPI data) > (XEN) 6f258000 - 6f8fd000 (ACPI NVS) > (XEN) 6f8fd000 - 6000 (reserved) > (XEN) 6000 - 7000 (usable) > (XEN) 7000 - 7800 (reserved) > (XEN) 7800 - 7860 (usable) > (XEN) 7860 - 7c80 (reserved) > (XEN) e000 - f000 (reserved) > (XEN) fe00 - fe011000 (reserved) > (XEN) fec0 - fec01000 (reserved) > (XEN) fee0 - fee01000 (reserved) > (XEN) ff00 - 0001 (reserved) > (XEN) 0001 - 00028180 (usable) And it's also outside of any of the GPU device's BARs. Makes me wonder what the graphics driver is doing. Jan ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Tue, Aug 7, 2018 at 9:09 AM Roger Pau Monné wrote: > > On Tue, Aug 07, 2018 at 08:45:07AM -0600, Tamas K Lengyel wrote: > > On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné wrote: > > > > > > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote: > > > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne > > > > wrote: > > > > > > > > > > Hello, > > > > > > > > > > The following series implement a workaround for missing RMRR > > > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd > > > > > option. > > > > > > > > > > The PVH workaround identity maps all regions marked as reserved in the > > > > > memory map. > > > > > > > > > > Note that this workaround is enabled by default on Intel hardware. > > > > > It's > > > > > also available to AMD hardware, although it's disabled by default in > > > > > that case. > > > > > > > > > > The series can be found at: > > > > > > > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3 > > > > > > > > > > Thanks, Roger. > > > > > Roger Pau Monne (4): > > > > > iommu: introduce dom0-iommu option > > > > > iommu: make iommu_inclusive_mapping a suboption of dom0-iommu > > > > > dom0/pvh: change the order of the MMCFG initialization > > > > > x86/iommu: add reserved dom0-iommu option to map reserved memory > > > > > ranges > > > > > > > > > > docs/misc/xen-command-line.markdown | 47 +++ > > > > > xen/arch/x86/hvm/dom0_build.c | 9 ++- > > > > > xen/arch/x86/hvm/io.c | 5 ++ > > > > > xen/arch/x86/x86_64/mm.c| 3 +- > > > > > xen/drivers/passthrough/amd/iommu_init.c| 2 +- > > > > > xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++- > > > > > xen/drivers/passthrough/arm/iommu.c | 4 + > > > > > xen/drivers/passthrough/iommu.c | 62 +-- > > > > > xen/drivers/passthrough/vtd/extern.h| 2 - > > > > > xen/drivers/passthrough/vtd/iommu.c | 25 +++--- > > > > > xen/drivers/passthrough/vtd/x86/vtd.c | 58 +- > > > > > xen/drivers/passthrough/x86/iommu.c | 87 > > > > > + > > > > > xen/include/asm-x86/hvm/io.h| 3 + > > > > > xen/include/xen/iommu.h | 8 +- > > > > > 14 files changed, 240 insertions(+), 86 deletions(-) > > > > > > > > > > -- > > > > > > > > Hi Roger, > > > > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux > > > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical > > > > keyboard of the laptop stopped working, it works no problem with just > > > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After > > > > running for a minute or two the system starts to slow down to the > > > > point where it becomes unresponsive. The xl dmesg log is filled with > > > > this error: > > > > > > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > > > > 4625f3a000, iommu reg = 82c00181c000 > > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > > > > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a > > > > > > Is the gmfn always the same (0x4625f3a)? > > > > > > > (XEN) root_entry[00] = 273a18001 > > > > (XEN) context[10] = 2_27ba35001 > > > > (XEN) l4[000] = 9c027ba34107 > > > > (XEN) l3[118] = 8000 > > > > (XEN) l3[118] not present > > > > > > Can you also paste the full xl dmesg log? I'm specially interested in > > > the memory map of the machine which is printed quite early during Xen > > > boot. > > > > > > > Unfortunately I don't have serial access on this laptop and "xl dmesg" > > gets completely filled with that error so the beginning of the log is > > lost by the time I get a terminal in dom0. > > You can get the memory map while booting in PV mode, it's going to be > exactly the same regardless of whether Dom0 is PV or PVH. This is the PV dmesg: (XEN) Xen version 4.12-unstable (dr@) (gcc (Debian 7.3.0-19) 7.3.0) debug=y Mon Aug 6 13:42:42 MDT 2018 (XEN) Latest ChangeSet: Fri Aug 3 10:01:36 2018 +0200 git:ddba1c2b1f (XEN) Bootloader: EFI (XEN) Command line: loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null console=vga (XEN) Xen image load base address: 0x5a20 (XEN) Video information: (XEN) VGA is graphics mode 3200x1800, 32 bpp (XEN) Disc information: (XEN) Found 0 MBR signatures (XEN) Found 1 EDD information structures (XEN) EFI RAM map: (XEN) - 00058000 (usable) (XEN) 00058000 - 00059000 (reserved) (XEN) 00059000 - 0009f000 (usable) (XEN) 0009f000 - 0010 (reserved) (XEN) 0010 - 5f14d000 (usable) (XEN) 5f14d000 - 5f14e000 (ACPI NVS) (XEN) 5f14e000 -
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On 07/08/18 15:45, Tamas K Lengyel wrote:
> On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné wrote:
>> On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote:
>>> On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne wrote:
Hello,
The following series implement a workaround for missing RMRR
entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
option.
The PVH workaround identity maps all regions marked as reserved in the
memory map.
Note that this workaround is enabled by default on Intel hardware. It's
also available to AMD hardware, although it's disabled by default in
that case.
The series can be found at:
git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
Thanks, Roger.
Roger Pau Monne (4):
iommu: introduce dom0-iommu option
iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
dom0/pvh: change the order of the MMCFG initialization
x86/iommu: add reserved dom0-iommu option to map reserved memory
ranges
docs/misc/xen-command-line.markdown | 47 +++
xen/arch/x86/hvm/dom0_build.c | 9 ++-
xen/arch/x86/hvm/io.c | 5 ++
xen/arch/x86/x86_64/mm.c| 3 +-
xen/drivers/passthrough/amd/iommu_init.c| 2 +-
xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
xen/drivers/passthrough/arm/iommu.c | 4 +
xen/drivers/passthrough/iommu.c | 62 +--
xen/drivers/passthrough/vtd/extern.h| 2 -
xen/drivers/passthrough/vtd/iommu.c | 25 +++---
xen/drivers/passthrough/vtd/x86/vtd.c | 58 +-
xen/drivers/passthrough/x86/iommu.c | 87 +
xen/include/asm-x86/hvm/io.h| 3 +
xen/include/xen/iommu.h | 8 +-
14 files changed, 240 insertions(+), 86 deletions(-)
--
>>> Hi Roger,
>>> I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
>>> 4.18-rc8. I was able to get dom0 to boot with PVH but the physical
>>> keyboard of the laptop stopped working, it works no problem with just
>>> Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
>>> running for a minute or two the system starts to slow down to the
>>> point where it becomes unresponsive. The xl dmesg log is filled with
>>> this error:
>>>
>>> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
>>> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
>>> (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr
>>> 4625f3a000, iommu reg = 82c00181c000
>>> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
>>> (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a
>> Is the gmfn always the same (0x4625f3a)?
>>
>>> (XEN) root_entry[00] = 273a18001
>>> (XEN) context[10] = 2_27ba35001
>>> (XEN) l4[000] = 9c027ba34107
>>> (XEN) l3[118] = 8000
>>> (XEN) l3[118] not present
>> Can you also paste the full xl dmesg log? I'm specially interested in
>> the memory map of the machine which is printed quite early during Xen
>> boot.
>>
> Unfortunately I don't have serial access on this laptop and "xl dmesg"
> gets completely filled with that error so the beginning of the log is
> lost by the time I get a terminal in dom0.
diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index a911958..9c1ee0a 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -81,7 +81,7 @@ custom_runtime_param("console_timestamps",
parse_console_timestamps);
static uint32_t __initdata opt_conring_size;
size_param("conring_size", opt_conring_size);
-#define _CONRING_SIZE 16384
+#define _CONRING_SIZE KB(512)
#define CONRING_IDX_MASK(i) ((i)&(conring_size-1))
static char __initdata _conring[_CONRING_SIZE];
static char *__read_mostly conring = _conring;
Sadly, the conring_size= option is almost completely useless, because
the smaller ring tends to truncate before it gets realloc()'d to be larger.
~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Tue, Aug 07, 2018 at 08:45:07AM -0600, Tamas K Lengyel wrote: > On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné wrote: > > > > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote: > > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne > > > wrote: > > > > > > > > Hello, > > > > > > > > The following series implement a workaround for missing RMRR > > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd > > > > option. > > > > > > > > The PVH workaround identity maps all regions marked as reserved in the > > > > memory map. > > > > > > > > Note that this workaround is enabled by default on Intel hardware. It's > > > > also available to AMD hardware, although it's disabled by default in > > > > that case. > > > > > > > > The series can be found at: > > > > > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3 > > > > > > > > Thanks, Roger. > > > > Roger Pau Monne (4): > > > > iommu: introduce dom0-iommu option > > > > iommu: make iommu_inclusive_mapping a suboption of dom0-iommu > > > > dom0/pvh: change the order of the MMCFG initialization > > > > x86/iommu: add reserved dom0-iommu option to map reserved memory > > > > ranges > > > > > > > > docs/misc/xen-command-line.markdown | 47 +++ > > > > xen/arch/x86/hvm/dom0_build.c | 9 ++- > > > > xen/arch/x86/hvm/io.c | 5 ++ > > > > xen/arch/x86/x86_64/mm.c| 3 +- > > > > xen/drivers/passthrough/amd/iommu_init.c| 2 +- > > > > xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++- > > > > xen/drivers/passthrough/arm/iommu.c | 4 + > > > > xen/drivers/passthrough/iommu.c | 62 +-- > > > > xen/drivers/passthrough/vtd/extern.h| 2 - > > > > xen/drivers/passthrough/vtd/iommu.c | 25 +++--- > > > > xen/drivers/passthrough/vtd/x86/vtd.c | 58 +- > > > > xen/drivers/passthrough/x86/iommu.c | 87 + > > > > xen/include/asm-x86/hvm/io.h| 3 + > > > > xen/include/xen/iommu.h | 8 +- > > > > 14 files changed, 240 insertions(+), 86 deletions(-) > > > > > > > > -- > > > > > > Hi Roger, > > > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux > > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical > > > keyboard of the laptop stopped working, it works no problem with just > > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After > > > running for a minute or two the system starts to slow down to the > > > point where it becomes unresponsive. The xl dmesg log is filled with > > > this error: > > > > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > > > 4625f3a000, iommu reg = 82c00181c000 > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > > > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a > > > > Is the gmfn always the same (0x4625f3a)? > > > > > (XEN) root_entry[00] = 273a18001 > > > (XEN) context[10] = 2_27ba35001 > > > (XEN) l4[000] = 9c027ba34107 > > > (XEN) l3[118] = 8000 > > > (XEN) l3[118] not present > > > > Can you also paste the full xl dmesg log? I'm specially interested in > > the memory map of the machine which is printed quite early during Xen > > boot. > > > > Unfortunately I don't have serial access on this laptop and "xl dmesg" > gets completely filled with that error so the beginning of the log is > lost by the time I get a terminal in dom0. You can get the memory map while booting in PV mode, it's going to be exactly the same regardless of whether Dom0 is PV or PVH. > I'll try it on another > laptop later today for which I have serial access. Across reboots the > gmfn changes: > > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > 23c251000, iommu reg = 82c00181c000 > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 23c251 > (XEN) root_entry[00] = 273a18001 > (XEN) context[10] = 2_27ba35001 > (XEN) l4[000] = 9c027ba34107 > (XEN) l3[008] = 8000 > (XEN) l3[008] not present > Thanks, Roger. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné wrote: > > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote: > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne wrote: > > > > > > Hello, > > > > > > The following series implement a workaround for missing RMRR > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd > > > option. > > > > > > The PVH workaround identity maps all regions marked as reserved in the > > > memory map. > > > > > > Note that this workaround is enabled by default on Intel hardware. It's > > > also available to AMD hardware, although it's disabled by default in > > > that case. > > > > > > The series can be found at: > > > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3 > > > > > > Thanks, Roger. > > > Roger Pau Monne (4): > > > iommu: introduce dom0-iommu option > > > iommu: make iommu_inclusive_mapping a suboption of dom0-iommu > > > dom0/pvh: change the order of the MMCFG initialization > > > x86/iommu: add reserved dom0-iommu option to map reserved memory > > > ranges > > > > > > docs/misc/xen-command-line.markdown | 47 +++ > > > xen/arch/x86/hvm/dom0_build.c | 9 ++- > > > xen/arch/x86/hvm/io.c | 5 ++ > > > xen/arch/x86/x86_64/mm.c| 3 +- > > > xen/drivers/passthrough/amd/iommu_init.c| 2 +- > > > xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++- > > > xen/drivers/passthrough/arm/iommu.c | 4 + > > > xen/drivers/passthrough/iommu.c | 62 +-- > > > xen/drivers/passthrough/vtd/extern.h| 2 - > > > xen/drivers/passthrough/vtd/iommu.c | 25 +++--- > > > xen/drivers/passthrough/vtd/x86/vtd.c | 58 +- > > > xen/drivers/passthrough/x86/iommu.c | 87 + > > > xen/include/asm-x86/hvm/io.h| 3 + > > > xen/include/xen/iommu.h | 8 +- > > > 14 files changed, 240 insertions(+), 86 deletions(-) > > > > > > -- > > > > Hi Roger, > > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical > > keyboard of the laptop stopped working, it works no problem with just > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After > > running for a minute or two the system starts to slow down to the > > point where it becomes unresponsive. The xl dmesg log is filled with > > this error: > > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > > 4625f3a000, iommu reg = 82c00181c000 > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a > > Is the gmfn always the same (0x4625f3a)? > > > (XEN) root_entry[00] = 273a18001 > > (XEN) context[10] = 2_27ba35001 > > (XEN) l4[000] = 9c027ba34107 > > (XEN) l3[118] = 8000 > > (XEN) l3[118] not present > > Can you also paste the full xl dmesg log? I'm specially interested in > the memory map of the machine which is printed quite early during Xen > boot. > Unfortunately I don't have serial access on this laptop and "xl dmesg" gets completely filled with that error so the beginning of the log is lost by the time I get a terminal in dom0. I'll try it on another laptop later today for which I have serial access. Across reboots the gmfn changes: (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr 23c251000, iommu reg = 82c00181c000 (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 23c251 (XEN) root_entry[00] = 273a18001 (XEN) context[10] = 2_27ba35001 (XEN) l4[000] = 9c027ba34107 (XEN) l3[008] = 8000 (XEN) l3[008] not present Tamas ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote: > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne wrote: > > > > Hello, > > > > The following series implement a workaround for missing RMRR > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd > > option. > > > > The PVH workaround identity maps all regions marked as reserved in the > > memory map. > > > > Note that this workaround is enabled by default on Intel hardware. It's > > also available to AMD hardware, although it's disabled by default in > > that case. > > > > The series can be found at: > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3 > > > > Thanks, Roger. > > Roger Pau Monne (4): > > iommu: introduce dom0-iommu option > > iommu: make iommu_inclusive_mapping a suboption of dom0-iommu > > dom0/pvh: change the order of the MMCFG initialization > > x86/iommu: add reserved dom0-iommu option to map reserved memory > > ranges > > > > docs/misc/xen-command-line.markdown | 47 +++ > > xen/arch/x86/hvm/dom0_build.c | 9 ++- > > xen/arch/x86/hvm/io.c | 5 ++ > > xen/arch/x86/x86_64/mm.c| 3 +- > > xen/drivers/passthrough/amd/iommu_init.c| 2 +- > > xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++- > > xen/drivers/passthrough/arm/iommu.c | 4 + > > xen/drivers/passthrough/iommu.c | 62 +-- > > xen/drivers/passthrough/vtd/extern.h| 2 - > > xen/drivers/passthrough/vtd/iommu.c | 25 +++--- > > xen/drivers/passthrough/vtd/x86/vtd.c | 58 +- > > xen/drivers/passthrough/x86/iommu.c | 87 + > > xen/include/asm-x86/hvm/io.h| 3 + > > xen/include/xen/iommu.h | 8 +- > > 14 files changed, 240 insertions(+), 86 deletions(-) > > > > -- > > Hi Roger, > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical > keyboard of the laptop stopped working, it works no problem with just > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After > running for a minute or two the system starts to slow down to the > point where it becomes unresponsive. The xl dmesg log is filled with > this error: > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault > (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr > 4625f3a000, iommu reg = 82c00181c000 > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set > (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a Is the gmfn always the same (0x4625f3a)? > (XEN) root_entry[00] = 273a18001 > (XEN) context[10] = 2_27ba35001 > (XEN) l4[000] = 9c027ba34107 > (XEN) l3[118] = 8000 > (XEN) l3[118] not present Can you also paste the full xl dmesg log? I'm specially interested in the memory map of the machine which is printed quite early during Xen boot. Thanks, Roger. ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne wrote: > > Hello, > > The following series implement a workaround for missing RMRR > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd > option. > > The PVH workaround identity maps all regions marked as reserved in the > memory map. > > Note that this workaround is enabled by default on Intel hardware. It's > also available to AMD hardware, although it's disabled by default in > that case. > > The series can be found at: > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3 > > Thanks, Roger. > Roger Pau Monne (4): > iommu: introduce dom0-iommu option > iommu: make iommu_inclusive_mapping a suboption of dom0-iommu > dom0/pvh: change the order of the MMCFG initialization > x86/iommu: add reserved dom0-iommu option to map reserved memory > ranges > > docs/misc/xen-command-line.markdown | 47 +++ > xen/arch/x86/hvm/dom0_build.c | 9 ++- > xen/arch/x86/hvm/io.c | 5 ++ > xen/arch/x86/x86_64/mm.c| 3 +- > xen/drivers/passthrough/amd/iommu_init.c| 2 +- > xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++- > xen/drivers/passthrough/arm/iommu.c | 4 + > xen/drivers/passthrough/iommu.c | 62 +-- > xen/drivers/passthrough/vtd/extern.h| 2 - > xen/drivers/passthrough/vtd/iommu.c | 25 +++--- > xen/drivers/passthrough/vtd/x86/vtd.c | 58 +- > xen/drivers/passthrough/x86/iommu.c | 87 + > xen/include/asm-x86/hvm/io.h| 3 + > xen/include/xen/iommu.h | 8 +- > 14 files changed, 240 insertions(+), 86 deletions(-) > > -- Hi Roger, I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux 4.18-rc8. I was able to get dom0 to boot with PVH but the physical keyboard of the laptop stopped working, it works no problem with just Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After running for a minute or two the system starts to slow down to the point where it becomes unresponsive. The xl dmesg log is filled with this error: (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault (XEN) [VT-D]DMAR:[DMA Read] Request device [:00:02.0] fault addr 4625f3a000, iommu reg = 82c00181c000 (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set (XEN) print_vtd_entries: iommu #0 dev :00:02.0 gmfn 4625f3a (XEN) root_entry[00] = 273a18001 (XEN) context[10] = 2_27ba35001 (XEN) l4[000] = 9c027ba34107 (XEN) l3[118] = 8000 (XEN) l3[118] not present The device in question is: 00:02.0 VGA compatible controller: Intel Corporation HD Graphics 620 (rev 02) (prog-if 00 [VGA controller]) Subsystem: Dell HD Graphics 620 Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- SERR- Capabilities: [70] Express (v2) Root Complex Integrated Endpoint, MSI 00 DevCap:MaxPayload 128 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl:Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop- MaxPayload 128 bytes, MaxReadReq 128 bytes DevSta:CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-, OBFF Not Supported DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable- 64bit- Address: fee0 Data: 4028 Capabilities: [d0] Power Management version 2 Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Process Address Space ID (PASID) PASIDCap: Exec- Priv-, Max PASID Width: 14 PASIDCtl: Enable- Exec- Priv- Capabilities: [200 v1] Address Translation Service (ATS) ATSCap:Invalidate Queue Depth: 00 ATSCtl:Enable-, Smallest Translation Unit: 00 Capabilities: [300 v1] Page Request Interface (PRI) PRICtl: Enable- Reset- PRISta: RF- UPRGI- Stopped+ Page Request Capacity: 8000, Page Request Allocation: Kernel driver in use: i915 Kernel modules: i915 The error is similar to one I reported in 2015 (https://lists.xenproject.org/archives/html/xen-devel/2015-08/msg02339.html) but that was due to AMT being enabled on the system, while this laptop doesn't have AMT enabled. The boot params I had for Xen: loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null dom0=pvh iommu=required,debug dom0-iommu=relaxed console=vga Let us know if we can provide any additional information. Thanks, Tamas
[Xen-devel] [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries
Hello, The following series implement a workaround for missing RMRR entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd option. The PVH workaround identity maps all regions marked as reserved in the memory map. Note that this workaround is enabled by default on Intel hardware. It's also available to AMD hardware, although it's disabled by default in that case. The series can be found at: git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3 Thanks, Roger. Roger Pau Monne (4): iommu: introduce dom0-iommu option iommu: make iommu_inclusive_mapping a suboption of dom0-iommu dom0/pvh: change the order of the MMCFG initialization x86/iommu: add reserved dom0-iommu option to map reserved memory ranges docs/misc/xen-command-line.markdown | 47 +++ xen/arch/x86/hvm/dom0_build.c | 9 ++- xen/arch/x86/hvm/io.c | 5 ++ xen/arch/x86/x86_64/mm.c| 3 +- xen/drivers/passthrough/amd/iommu_init.c| 2 +- xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++- xen/drivers/passthrough/arm/iommu.c | 4 + xen/drivers/passthrough/iommu.c | 62 +-- xen/drivers/passthrough/vtd/extern.h| 2 - xen/drivers/passthrough/vtd/iommu.c | 25 +++--- xen/drivers/passthrough/vtd/x86/vtd.c | 58 +- xen/drivers/passthrough/x86/iommu.c | 87 + xen/include/asm-x86/hvm/io.h| 3 + xen/include/xen/iommu.h | 8 +- 14 files changed, 240 insertions(+), 86 deletions(-) -- 2.18.0 ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
