Re: [PATCH v3 00/35] iommu/amd: Add multiple PCI segments support
Hi Joerg, On 6/23/2022 1:24 PM, Joerg Roedel wrote: > Hi Vasant, > > On Wed, May 11, 2022 at 12:51:06PM +0530, Vasant Hegde wrote: >> .../admin-guide/kernel-parameters.txt | 34 +- >> drivers/iommu/amd/amd_iommu.h | 13 +- >> drivers/iommu/amd/amd_iommu_types.h | 133 +++- >> drivers/iommu/amd/init.c | 687 +++--- >> drivers/iommu/amd/iommu.c | 563 -- >> drivers/iommu/amd/iommu_v2.c | 67 +- >> drivers/iommu/amd/quirks.c| 4 +- >> 7 files changed, 904 insertions(+), 597 deletions(-) > > So this is applied now to the IOMMU tree, thanks for the work. Something > that bothered me while looking at this was the almost complete lack of > locking while accessing the global data structures. Some of them are > lock-less, so it is partially fine, and most of them are used read-only > during system runtime. But I would appreciate if you and/or Suravee > could look over that again and check again if there needs to be more > locking. Thanks. We did look into the code and we think it should be fine for now. > > The current situation will fire back at the point where you want to > implement IOMMU hotplug. Note that device hotplug is already possible > today, either with real devices or SR-IOV. You are right. We think IOMMU hotplug support needs lot more changes including locking existing global structures. We will look into it whenever we support IOMMU hotplug. -Vasant > > Regards, > > Joerg ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 00/35] iommu/amd: Add multiple PCI segments support
Hi Vasant, On Wed, May 11, 2022 at 12:51:06PM +0530, Vasant Hegde wrote: > .../admin-guide/kernel-parameters.txt | 34 +- > drivers/iommu/amd/amd_iommu.h | 13 +- > drivers/iommu/amd/amd_iommu_types.h | 133 +++- > drivers/iommu/amd/init.c | 687 +++--- > drivers/iommu/amd/iommu.c | 563 -- > drivers/iommu/amd/iommu_v2.c | 67 +- > drivers/iommu/amd/quirks.c| 4 +- > 7 files changed, 904 insertions(+), 597 deletions(-) So this is applied now to the IOMMU tree, thanks for the work. Something that bothered me while looking at this was the almost complete lack of locking while accessing the global data structures. Some of them are lock-less, so it is partially fine, and most of them are used read-only during system runtime. But I would appreciate if you and/or Suravee could look over that again and check again if there needs to be more locking. The current situation will fire back at the point where you want to implement IOMMU hotplug. Note that device hotplug is already possible today, either with real devices or SR-IOV. Regards, Joerg ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 00/35] iommu/amd: Add multiple PCI segments support
Hi Joerg, On 6/7/2022 4:17 PM, Vasant Hegde wrote: > Hello Joerg, > > > On 5/20/2022 5:42 PM, Vasant Hegde wrote: >> Joerg, >> >> >> On 5/20/2022 3:33 PM, Joerg Roedel wrote: >>> Hi Vasant, >>> >>> On Fri, May 20, 2022 at 03:25:38PM +0530, Vasant Hegde wrote: Ping. Did you get a chance to look into this series? >>> >>> Sorry, too late for this round. The changes are pretty invasive and >>> merging them at -rc7 stage would not give them enough testing before >>> being merged. Please send me a reminder after the next merge window. >> >> Sure. I will remind you after v5.19 merge window closes. > > Ping. Can you please take a look of this series? > Do you want me to rebase patchset on to of v5.19-rc1 -OR- latest iommu/next > branch? > Ping? -Vasant ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 00/35] iommu/amd: Add multiple PCI segments support
Hello Joerg, On 5/20/2022 5:42 PM, Vasant Hegde wrote: > Joerg, > > > On 5/20/2022 3:33 PM, Joerg Roedel wrote: >> Hi Vasant, >> >> On Fri, May 20, 2022 at 03:25:38PM +0530, Vasant Hegde wrote: >>> Ping. Did you get a chance to look into this series? >> >> Sorry, too late for this round. The changes are pretty invasive and >> merging them at -rc7 stage would not give them enough testing before >> being merged. Please send me a reminder after the next merge window. > > Sure. I will remind you after v5.19 merge window closes. Ping. Can you please take a look of this series? Do you want me to rebase patchset on to of v5.19-rc1 -OR- latest iommu/next branch? -Vasant ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 00/35] iommu/amd: Add multiple PCI segments support
Joerg, On 5/20/2022 3:33 PM, Joerg Roedel wrote: > Hi Vasant, > > On Fri, May 20, 2022 at 03:25:38PM +0530, Vasant Hegde wrote: >> Ping. Did you get a chance to look into this series? > > Sorry, too late for this round. The changes are pretty invasive and > merging them at -rc7 stage would not give them enough testing before > being merged. Please send me a reminder after the next merge window. Sure. I will remind you after v5.19 merge window closes. -Vasant ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 00/35] iommu/amd: Add multiple PCI segments support
Hi Vasant, On Fri, May 20, 2022 at 03:25:38PM +0530, Vasant Hegde wrote: > Ping. Did you get a chance to look into this series? Sorry, too late for this round. The changes are pretty invasive and merging them at -rc7 stage would not give them enough testing before being merged. Please send me a reminder after the next merge window. Regards, Joerg ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 00/35] iommu/amd: Add multiple PCI segments support
Hello Joerg, On 5/11/2022 12:51 PM, Vasant Hegde wrote: > Newer AMD systems can support multiple PCI segments, where each segment > contains one or more IOMMU instances. However, an IOMMU instance can only > support a single PCI segment. Ping. Did you get a chance to look into this series? -Vasant ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH v3 00/35] iommu/amd: Add multiple PCI segments support
Newer AMD systems can support multiple PCI segments, where each segment contains one or more IOMMU instances. However, an IOMMU instance can only support a single PCI segment. Current code assumes a system contains only one PCI segment (segment 0) and creates global data structures such as device table, rlookup table, etc. This series introduces per-PCI-segment data structure, which contains device table, alias table, etc. For each PCI segment, all IOMMUs share the same data structure. The series also makes necessary code adjustment and logging enhancements. Finally it removes global data structures like device table, alias table, etc. In case of system w/ single PCI segment (e.g. PCI segment ID is zero), IOMMU driver allocates one PCI segment data structure, which will be shared by all IOMMUs. Patch 1 updates struct iommu_dev_data definition. Patch 2 - 13 introduce new PCI segment structure and allocate per data structures, and introduce the amd_iommu.pci_seg pointer to point to the corresponded pci_segment structure. Also, we have introduced a helper function rlookup_amd_iommu() to reverse-lookup each iommu for a particular device. Patch 14 - 27 adopt to per PCI segment data structure and removes global data structure. Patch 28 fixes flushing logic to flush upto last_bdf. Patch 29 - 35 convert usages of 16-bit PCI device ID to include 16-bit segment ID. Changes from v2 -> v3: - Addressed Joerg's review comments - Fixed typo in patch 1 subject - Fixed few minor things in patch 2 - Merged patch 27 - 29 into one patch - Added new macros to get seg and devid from sbdf - Patch 32 : Extend devid to 32bit and added new macro. v2 patchset : https://lore.kernel.org/linux-iommu/20220425113415.24087-1-vasant.he...@amd.com/T/#t Changes from v1 -> v2: - Updated patch 1 to include dev_is_pci() check v1 patchset : https://lore.kernel.org/linux-iommu/20220404100023.324645-1-vasant.he...@amd.com/T/#t Changes from RFC -> v1: - Rebased patches on top of iommu/next tree. - Update struct iommu_dev_data definition - Updated few log message to print segment ID - Fix smatch warnings RFC patchset : https://lore.kernel.org/linux-iommu/20220311094854.31595-1-vasant.he...@amd.com/T/#t Regards, Vasant Suravee Suthikulpanit (20): iommu/amd: Introduce per PCI segment device table iommu/amd: Introduce per PCI segment rlookup table iommu/amd: Introduce per PCI segment old_dev_tbl_cpy iommu/amd: Introduce per PCI segment alias_table iommu/amd: Convert to use rlookup_amd_iommu helper function iommu/amd: Update irq_remapping_alloc to use IOMMU lookup helper function iommu/amd: Introduce struct amd_ir_data.iommu iommu/amd: Update amd_irte_ops functions iommu/amd: Update alloc_irq_table and alloc_irq_index iommu/amd: Update set_dte_entry and clear_dte_entry iommu/amd: Update iommu_ignore_device iommu/amd: Update dump_dte_entry iommu/amd: Update set_dte_irq_entry iommu/amd: Update (un)init_device_table_dma() iommu/amd: Update set_dev_entry_bit() and get_dev_entry_bit() iommu/amd: Remove global amd_iommu_[dev_table/alias_table/last_bdf] iommu/amd: Introduce get_device_sbdf_id() helper function iommu/amd: Include PCI segment ID when initialize IOMMU iommu/amd: Specify PCI segment ID when getting pci device iommu/amd: Add PCI segment support for ivrs_[ioapic/hpet/acpihid] commands Vasant Hegde (15): iommu/amd: Update struct iommu_dev_data definition iommu/amd: Introduce pci segment structure iommu/amd: Introduce per PCI segment irq_lookup_table iommu/amd: Introduce per PCI segment dev_data_list iommu/amd: Introduce per PCI segment unity map list iommu/amd: Introduce per PCI segment last_bdf iommu/amd: Introduce per PCI segment device table size iommu/amd: Introduce per PCI segment alias table size iommu/amd: Introduce per PCI segment rlookup table size iommu/amd: Convert to use per PCI segment irq_lookup_table iommu/amd: Convert to use per PCI segment rlookup_table iommu/amd: Flush upto last_bdf only iommu/amd: Print PCI segment ID in error log messages iommu/amd: Update device_state structure to include PCI seg ID iommu/amd: Update amd_iommu_fault structure to include PCI seg ID .../admin-guide/kernel-parameters.txt | 34 +- drivers/iommu/amd/amd_iommu.h | 13 +- drivers/iommu/amd/amd_iommu_types.h | 133 +++- drivers/iommu/amd/init.c | 687 +++--- drivers/iommu/amd/iommu.c | 563 -- drivers/iommu/amd/iommu_v2.c | 67 +- drivers/iommu/amd/quirks.c| 4 +- 7 files changed, 904 insertions(+), 597 deletions(-) -- 2.27.0 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
