Re: Seeing some another issue with mixed domains in the same iommu_group
Hi Jerry,
On 2020/2/7 7:16, Jerry Snitselaar wrote:
Hi Baolu,
Would something along these lines makes sense?
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 9dc37672bf89..40cc8f5a3ebb 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3614,6 +3614,20 @@ static bool iommu_need_mapping(struct device *dev)
}
dmar_remove_one_dev_info(dev);
get_private_domain_for_dev(dev);
+ } else {
+ if (dev->archdata.iommu == NULL) {
+ struct iommu_domain *domain;
+ struct iommu_group *group;
+ struct dmar_domain *dmar_domain, *tmp;
+
+ group = iommu_group_get_for_dev(dev);
+ domain = iommu_group_default_domain(group);
+ dmar_domain = to_dmar_domain(domain);
+ tmp = set_domain_for_dev(dev, dmar_domain);
+ }
}
dev_info(dev, "32bit DMA uses non-identity mapping\n");
Thanks for reporting.
Actually, I prefer to removing this domain switch as long as users are
able to make a 32-bit device use DMA domain while system default is
identity or it breaks anything. 32-bit devices (or normally devices
with limited addressing capability over the whole system memory) using
DMA domain helps by removing the swiotlb performance overhead, which is
the original motivation of this code.
Best regards,
baolu
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: Seeing some another issue with mixed domains in the same iommu_group
On Thu Feb 06 20, Jerry Snitselaar wrote:
...
The above cases seem to be avoided by:
9235cb13d7d1 | 2020-01-24 | iommu/vt-d: Allow devices with RMRRs to use
identity domain (Lu Baolu)
which results in the watchdog device no longer taking a dma domain and
switching the group default.
Without that patch though when it gets into the iommu_need_mapping code for
:01:00.4 after
the following:
dmar_remove_one_dev_info(dev);
ret = iommu_request_dma_domain_for_dev(dev);
ret is 0 and dev->archdata.iommu is NULL. Even with 9235cb13d7d1
device_def_domain_type can return
return dma, but I'm not sure how likely it is for there to be an iommu group
like that again where
the group default ends up dma, a device gets removed and added to the identity
domain, and then
ends up in that code in iommu_need_mapping.
Hi Baolu,
Would something along these lines makes sense?
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 9dc37672bf89..40cc8f5a3ebb 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3614,6 +3614,20 @@ static bool iommu_need_mapping(struct device *dev)
}
dmar_remove_one_dev_info(dev);
get_private_domain_for_dev(dev);
+ } else {
+ if (dev->archdata.iommu == NULL) {
+ struct iommu_domain *domain;
+ struct iommu_group *group;
+ struct dmar_domain *dmar_domain, *tmp;
+
+ group = iommu_group_get_for_dev(dev);
+ domain = iommu_group_default_domain(group);
+ dmar_domain = to_dmar_domain(domain);
+ tmp = set_domain_for_dev(dev, dmar_domain);
+ }
}
dev_info(dev, "32bit DMA uses non-identity mapping\n");
--
Obviously needs some checks added, but this was just an initial test I
was trying.
Regards,
Jerry
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: Seeing some another issue with mixed domains in the same iommu_group
On Thu Feb 06 20, Jerry Snitselaar wrote: On Thu Feb 06 20, Jerry Snitselaar wrote: Hi Baolu, I'm seeing another issue with the devices in the HP ilo when the system is booted with intel_iommu=on and iommu=pt (iommu=nopt does not run into problems). first system: 01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05) 01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200EH 01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05) 01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02) [ 21.208103] pci :01:00.0: Adding to iommu group 24 [ 21.210911] pci :01:00.0: Using iommu dma mapping [ 21.212635] pci :01:00.1: Adding to iommu group 24 [ 21.214326] pci :01:00.1: Device uses a private identity domain. [ 21.216507] pci :01:00.2: Adding to iommu group 24 [ 21.618173] pci :01:00.4: Adding to iommu group 24 [ 21.619839] pci :01:00.4: Device uses a private identity domain. [ 26.206832] uhci_hcd: USB Universal Host Controller Interface driver [ 26.209044] uhci_hcd :01:00.4: UHCI Host Controller [ 26.210897] uhci_hcd :01:00.4: new USB bus registered, assigned bus number 3 [ 26.213247] uhci_hcd :01:00.4: detected 8 ports [ 26.214810] uhci_hcd :01:00.4: port count misdetected? forcing to 2 ports [ 26.217153] uhci_hcd :01:00.4: irq 16, io base 0x3c00 [ 26.219171] uhci_hcd :01:00.4: 32bit DMA uses non-identity mapping [ 26.221261] uhci_hcd :01:00.4: unable to allocate consistent memory for frame list [ 26.223787] uhci_hcd :01:00.4: startup error -16 [ 26.225381] uhci_hcd :01:00.4: USB bus 3 deregistered [ 26.227378] uhci_hcd :01:00.4: init :01:00.4 fail, -16 [ 26.229296] uhci_hcd: probe of :01:00.4 failed with error -16 different system with similar issue: 01:00.0 System peripheral [0880]: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support [103c:3306] (rev 07) 01:00.1 VGA compatible controller [0300]: Matrox Electronics Systems Ltd. MGA G200eH3 [102b:0538] (rev 02) (prog-if 00 [VGA controller]) 01:00.2 System peripheral [0880]: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging [103c:3307] (rev 07) 01:00.4 USB controller [0c03]: Hewlett-Packard Company iLO5 Virtual USB Controller [103c:22f6] (prog-if 20 [EHCI]) [ 13.695663] pci :01:00.0: Adding to iommu group 10 [ 13.703667] pci :01:00.0: Using iommu dma mapping [ 13.708871] pci :01:00.1: Adding to iommu group 10 [ 13.714033] pci :01:00.1: DMAR: Device uses a private identity domain. [ 13.721033] pci :01:00.2: Adding to iommu group 10 [ 13.726290] pci :01:00.4: Adding to iommu group 10 [ 13.731453] pci :01:00.4: DMAR: Device uses a private identity domain. [ 17.157796] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 17.164348] ehci-pci: EHCI PCI platform driver [ 17.170061] ehci-pci :01:00.4: EHCI Host Controller [ 17.175457] ehci-pci :01:00.4: new USB bus registered, assigned bus number 1 [ 17.182912] ehci-pci :01:00.4: DMAR: 32bit DMA uses non-identity mapping [ 17.189988] ehci-pci :01:00.4: can't setup: -12 [ 17.194884] ehci-pci :01:00.4: USB bus 1 deregistered [ 17.200567] ehci-pci :01:00.4: init :01:00.4 fail, -12 [ 17.206508] ehci-pci: probe of :01:00.4 failed with error -12 I'm looking through the code and trying to debug it, but any thoughts on this? Regards, Jerry In iommu_need_mapping, in a case like the above does something like dmar_insert_one_dev_info need to happen to associate the device back with the group default domain? In intel_iommu_add_device it is going to get removed and added to the identity domain, and then in iommu_need_mapping it gets removed from the identity domain, and iommu_request_dma_domain_for_dev should return 0 because the group default domain at this point is the correct type. The above cases seem to be avoided by: 9235cb13d7d1 | 2020-01-24 | iommu/vt-d: Allow devices with RMRRs to use identity domain (Lu Baolu) which results in the watchdog device no longer taking a dma domain and switching the group default. Without that patch though when it gets into the iommu_need_mapping code for :01:00.4 after the following: dmar_remove_one_dev_info(dev); ret = iommu_request_dma_domain_for_dev(dev); ret is 0 and dev->archdata.iommu is NULL. Even with 9235cb13d7d1 device_def_domain_type can return return dma, but I'm not sure how likely it is for there to be an iommu group like that again where the group default ends up dma, a device gets removed and added to the identity domain, and then ends up in that code in iommu_need_mapping.
Re: Seeing some another issue with mixed domains in the same iommu_group
On Thu Feb 06 20, Jerry Snitselaar wrote: Hi Baolu, I'm seeing another issue with the devices in the HP ilo when the system is booted with intel_iommu=on and iommu=pt (iommu=nopt does not run into problems). first system: 01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05) 01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200EH 01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05) 01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02) [ 21.208103] pci :01:00.0: Adding to iommu group 24 [ 21.210911] pci :01:00.0: Using iommu dma mapping [ 21.212635] pci :01:00.1: Adding to iommu group 24 [ 21.214326] pci :01:00.1: Device uses a private identity domain. [ 21.216507] pci :01:00.2: Adding to iommu group 24 [ 21.618173] pci :01:00.4: Adding to iommu group 24 [ 21.619839] pci :01:00.4: Device uses a private identity domain. [ 26.206832] uhci_hcd: USB Universal Host Controller Interface driver [ 26.209044] uhci_hcd :01:00.4: UHCI Host Controller [ 26.210897] uhci_hcd :01:00.4: new USB bus registered, assigned bus number 3 [ 26.213247] uhci_hcd :01:00.4: detected 8 ports [ 26.214810] uhci_hcd :01:00.4: port count misdetected? forcing to 2 ports [ 26.217153] uhci_hcd :01:00.4: irq 16, io base 0x3c00 [ 26.219171] uhci_hcd :01:00.4: 32bit DMA uses non-identity mapping [ 26.221261] uhci_hcd :01:00.4: unable to allocate consistent memory for frame list [ 26.223787] uhci_hcd :01:00.4: startup error -16 [ 26.225381] uhci_hcd :01:00.4: USB bus 3 deregistered [ 26.227378] uhci_hcd :01:00.4: init :01:00.4 fail, -16 [ 26.229296] uhci_hcd: probe of :01:00.4 failed with error -16 different system with similar issue: 01:00.0 System peripheral [0880]: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support [103c:3306] (rev 07) 01:00.1 VGA compatible controller [0300]: Matrox Electronics Systems Ltd. MGA G200eH3 [102b:0538] (rev 02) (prog-if 00 [VGA controller]) 01:00.2 System peripheral [0880]: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging [103c:3307] (rev 07) 01:00.4 USB controller [0c03]: Hewlett-Packard Company iLO5 Virtual USB Controller [103c:22f6] (prog-if 20 [EHCI]) [ 13.695663] pci :01:00.0: Adding to iommu group 10 [ 13.703667] pci :01:00.0: Using iommu dma mapping [ 13.708871] pci :01:00.1: Adding to iommu group 10 [ 13.714033] pci :01:00.1: DMAR: Device uses a private identity domain. [ 13.721033] pci :01:00.2: Adding to iommu group 10 [ 13.726290] pci :01:00.4: Adding to iommu group 10 [ 13.731453] pci :01:00.4: DMAR: Device uses a private identity domain. [ 17.157796] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 17.164348] ehci-pci: EHCI PCI platform driver [ 17.170061] ehci-pci :01:00.4: EHCI Host Controller [ 17.175457] ehci-pci :01:00.4: new USB bus registered, assigned bus number 1 [ 17.182912] ehci-pci :01:00.4: DMAR: 32bit DMA uses non-identity mapping [ 17.189988] ehci-pci :01:00.4: can't setup: -12 [ 17.194884] ehci-pci :01:00.4: USB bus 1 deregistered [ 17.200567] ehci-pci :01:00.4: init :01:00.4 fail, -12 [ 17.206508] ehci-pci: probe of :01:00.4 failed with error -12 I'm looking through the code and trying to debug it, but any thoughts on this? Regards, Jerry In iommu_need_mapping, in a case like the above does something like dmar_insert_one_dev_info need to happen to associate the device back with the group default domain? In intel_iommu_add_device it is going to get removed and added to the identity domain, and then in iommu_need_mapping it gets removed from the identity domain, and iommu_request_dma_domain_for_dev should return 0 because the group default domain at this point is the correct type. ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Seeing some another issue with mixed domains in the same iommu_group
Hi Baolu, I'm seeing another issue with the devices in the HP ilo when the system is booted with intel_iommu=on and iommu=pt (iommu=nopt does not run into problems). first system: 01:00.0 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support (rev 05) 01:00.1 VGA compatible controller: Matrox Electronics Systems Ltd. MGA G200EH 01:00.2 System peripheral: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging (rev 05) 01:00.4 USB controller: Hewlett-Packard Company Integrated Lights-Out Standard Virtual USB Controller (rev 02) [ 21.208103] pci :01:00.0: Adding to iommu group 24 [ 21.210911] pci :01:00.0: Using iommu dma mapping [ 21.212635] pci :01:00.1: Adding to iommu group 24 [ 21.214326] pci :01:00.1: Device uses a private identity domain. [ 21.216507] pci :01:00.2: Adding to iommu group 24 [ 21.618173] pci :01:00.4: Adding to iommu group 24 [ 21.619839] pci :01:00.4: Device uses a private identity domain. [ 26.206832] uhci_hcd: USB Universal Host Controller Interface driver [ 26.209044] uhci_hcd :01:00.4: UHCI Host Controller [ 26.210897] uhci_hcd :01:00.4: new USB bus registered, assigned bus number 3 [ 26.213247] uhci_hcd :01:00.4: detected 8 ports [ 26.214810] uhci_hcd :01:00.4: port count misdetected? forcing to 2 ports [ 26.217153] uhci_hcd :01:00.4: irq 16, io base 0x3c00 [ 26.219171] uhci_hcd :01:00.4: 32bit DMA uses non-identity mapping [ 26.221261] uhci_hcd :01:00.4: unable to allocate consistent memory for frame list [ 26.223787] uhci_hcd :01:00.4: startup error -16 [ 26.225381] uhci_hcd :01:00.4: USB bus 3 deregistered [ 26.227378] uhci_hcd :01:00.4: init :01:00.4 fail, -16 [ 26.229296] uhci_hcd: probe of :01:00.4 failed with error -16 different system with similar issue: 01:00.0 System peripheral [0880]: Hewlett-Packard Company Integrated Lights-Out Standard Slave Instrumentation & System Support [103c:3306] (rev 07) 01:00.1 VGA compatible controller [0300]: Matrox Electronics Systems Ltd. MGA G200eH3 [102b:0538] (rev 02) (prog-if 00 [VGA controller]) 01:00.2 System peripheral [0880]: Hewlett-Packard Company Integrated Lights-Out Standard Management Processor Support and Messaging [103c:3307] (rev 07) 01:00.4 USB controller [0c03]: Hewlett-Packard Company iLO5 Virtual USB Controller [103c:22f6] (prog-if 20 [EHCI]) [ 13.695663] pci :01:00.0: Adding to iommu group 10 [ 13.703667] pci :01:00.0: Using iommu dma mapping [ 13.708871] pci :01:00.1: Adding to iommu group 10 [ 13.714033] pci :01:00.1: DMAR: Device uses a private identity domain. [ 13.721033] pci :01:00.2: Adding to iommu group 10 [ 13.726290] pci :01:00.4: Adding to iommu group 10 [ 13.731453] pci :01:00.4: DMAR: Device uses a private identity domain. [ 17.157796] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver [ 17.164348] ehci-pci: EHCI PCI platform driver [ 17.170061] ehci-pci :01:00.4: EHCI Host Controller [ 17.175457] ehci-pci :01:00.4: new USB bus registered, assigned bus number 1 [ 17.182912] ehci-pci :01:00.4: DMAR: 32bit DMA uses non-identity mapping [ 17.189988] ehci-pci :01:00.4: can't setup: -12 [ 17.194884] ehci-pci :01:00.4: USB bus 1 deregistered [ 17.200567] ehci-pci :01:00.4: init :01:00.4 fail, -12 [ 17.206508] ehci-pci: probe of :01:00.4 failed with error -12 I'm looking through the code and trying to debug it, but any thoughts on this? Regards, Jerry ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
