Re: [PATCH RFC 0/9] Possible set of VT-d optimizations

2021-02-01 Thread Chuck Lever
> On Jan 28, 2021, at 9:52 AM, Chuck Lever wrote: > > > >> On Jan 28, 2021, at 8:59 AM, Robin Murphy wrote: >> >> On 2021-01-27 20:00, Chuck Lever wrote: >>> Hi- >>> This collection of patches seems to get the best throughtput results &g

Re: [PATCH RFC 0/9] Possible set of VT-d optimizations

2021-01-28 Thread Chuck Lever
> On Jan 28, 2021, at 8:59 AM, Robin Murphy wrote: > > On 2021-01-27 20:00, Chuck Lever wrote: >> Hi- >> This collection of patches seems to get the best throughtput results >> so far. The NFS WRITE result is fully restored, and the NFS READ >> resul

[PATCH RFC 9/9] iommu/vt-d: Introduce map_sg() for Intel IOMMUs

2021-01-27 Thread Chuck Lever
Attempt to reduce indirect call overhead when mapping a substantial scatter-gather list. Signed-off-by: Chuck Lever --- drivers/iommu/intel/iommu.c | 37 + 1 file changed, 37 insertions(+) diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel

[PATCH RFC 4/9] iommu/io-pgtable: Introduce map_sg() as a page table op

2021-01-27 Thread Chuck Lever
, reducing the number of indirect calls, and boosting overall iommu_map_sg() performance. Signed-off-by: Isaac J. Manjarres Tested-by: Sai Prakash Ranjan Signed-off-by: Chuck Lever --- include/linux/io-pgtable.h |6 ++ 1 file changed, 6 insertions(+) diff --git a/include/linux/io-pgtable.h b

[PATCH RFC 8/9] iommu/arm-smmu: Hook up map_sg()

2021-01-27 Thread Chuck Lever
-off-by: Isaac J. Manjarres Tested-by: Sai Prakash Ranjan Signed-off-by: Chuck Lever --- drivers/iommu/arm/arm-smmu/arm-smmu.c | 19 +++ 1 file changed, 19 insertions(+) diff --git a/drivers/iommu/arm/arm-smmu/arm-smmu.c b/drivers/iommu/arm/arm-smmu/arm-smmu.c index d8c6bfde6a61

[PATCH RFC 3/9] iommu/vt-d: Add iotlb_sync_map callback

2021-01-27 Thread Chuck Lever
ops") Reported-by: Chuck Lever Link: https://lore.kernel.org/linux-iommu/d81314ed-5673-44a6-b597-090e3cb83...@oracle.com/ Signed-off-by: Lu Baolu Cc: Robin Murphy [ cel: removed @first_pte, which is no longer used ] Signed-off-by: Chuck Lever --- drivers/iommu/

[PATCH RFC 7/9] iommu: Introduce map_sg() as an IOMMU op for IOMMU drivers

2021-01-27 Thread Chuck Lever
iotlb_sync_map call site ] Signed-off-by: Chuck Lever --- drivers/iommu/iommu.c | 12 include/linux/iommu.h |5 + 2 files changed, 17 insertions(+) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index ed879a4d7fac..bd7adbd0339b 100644 --- a/drivers/iommu/iommu.c

[PATCH RFC 5/9] iommu/io-pgtable-arm: Hook up map_sg()

2021-01-27 Thread Chuck Lever
From: Isaac J. Manjarres Implement the map_sg io-pgtable op for the ARM LPAE io-pgtable code, so that IOMMU drivers can call it when they need to map a scatter-gather list. Signed-off-by: Isaac J. Manjarres Tested-by: Sai Prakash Ranjan Signed-off-by: Chuck Lever --- drivers/iommu/io

[PATCH RFC 6/9] iommu/io-pgtable-arm-v7s: Hook up map_sg()

2021-01-27 Thread Chuck Lever
From: Isaac J. Manjarres Implement the map_sg io-pgtable op for the ARMv7s io-pgtable code, so that IOMMU drivers can call it when they need to map a scatter-gather list. Signed-off-by: Isaac J. Manjarres Tested-by: Sai Prakash Ranjan Signed-off-by: Chuck Lever --- drivers/iommu/io-pgtable

[PATCH RFC 2/9] iommu: Add iova and size as parameters in iotlb_sync_map

2021-01-27 Thread Chuck Lever
-by: Robin Murphy Signed-off-by: Chuck Lever --- drivers/iommu/iommu.c |4 ++-- drivers/iommu/tegra-gart.c |7 +-- include/linux/iommu.h |3 ++- 3 files changed, 9 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index c304a6a30d42

[PATCH RFC 1/9] iommu: Move iotlb_sync_map out from __iommu_map

2021-01-27 Thread Chuck Lever
-by: Yong Wu Reviewed-by: Robin Murphy Signed-off-by: Chuck Lever --- drivers/iommu/iommu.c | 23 ++- 1 file changed, 18 insertions(+), 5 deletions(-) diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index ffeebda8d6de..c304a6a30d42 100644 --- a/drivers/iommu/iommu.c

[PATCH RFC 0/9] Possible set of VT-d optimizations

2021-01-27 Thread Chuck Lever
implementation of a map_sg op for the Intel IOMMU. This is nothing more than a copy of the loop in __iommu_map_sg() with the call to __iommu_map() replaced with a call to intel_iommu_map(). --- Chuck Lever (1): iommu/vt-d: Introduce map_sg() for Intel IOMMUs Isaac J. Manjarres (5): iommu

Re: [RFT PATCH 0/3] Performance regression noted in v5.11-rc after c062db039f40

2021-01-26 Thread Chuck Lever
> On Jan 26, 2021, at 8:53 PM, Lu Baolu wrote: > > Hi Chuck, > > On 1/26/21 11:52 PM, Chuck Lever wrote: >>> On Jan 26, 2021, at 1:18 AM, Lu Baolu wrote: >>> >>> On 2021/1/26 3:31, Chuck Lever wrote: >>>>> On Jan 25, 202

Re: [RFT PATCH 0/3] Performance regression noted in v5.11-rc after c062db039f40

2021-01-26 Thread Chuck Lever
re/rw.c , and as far as I understand, the same mapping functions are used for both I/O directions. It's possible that the NIC is doing something different for RDMA Read and RDMA Write, but I don't have much visibility into that. Reads are very different f

Re: [RFT PATCH 0/3] Performance regression noted in v5.11-rc after c062db039f40

2021-01-26 Thread Chuck Lever
> On Jan 26, 2021, at 1:18 AM, Lu Baolu wrote: > > On 2021/1/26 3:31, Chuck Lever wrote: >>> On Jan 25, 2021, at 12:39 PM, Chuck Lever wrote: >>> >>> Hello Lu - >>> >>> Many thanks for your prototype. >>> >>> >>

Re: [RFT PATCH 0/3] Performance regression noted in v5.11-rc after c062db039f40

2021-01-25 Thread Chuck Lever
> On Jan 25, 2021, at 12:39 PM, Chuck Lever wrote: > > Hello Lu - > > Many thanks for your prototype. > > >> On Jan 24, 2021, at 9:38 PM, Lu Baolu wrote: >> >> This patch series is only for Request-For-Testing purpose. It aims to >>

Re: [RFT PATCH 0/3] Performance regression noted in v5.11-rc after c062db039f40

2021-01-25 Thread Chuck Lever
t have an explanation for this result. I applied your patches on the NFS server also without noting improvement. -- Chuck Lever ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-22 Thread Chuck Lever
> On Jan 22, 2021, at 12:38 PM, Robin Murphy wrote: > > On 2021-01-22 16:18, Chuck Lever wrote: >>> On Jan 21, 2021, at 10:00 PM, Lu Baolu wrote: >>> >>> +Isaac >>> >>> On 1/22/21 3:09 AM, Chuck Lever wrote: >>>>> On Jan

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-22 Thread Chuck Lever
> On Jan 21, 2021, at 10:00 PM, Lu Baolu wrote: > > +Isaac > > On 1/22/21 3:09 AM, Chuck Lever wrote: >>> On Jan 18, 2021, at 1:00 PM, Robin Murphy wrote: >>> >>> On 2021-01-18 16:18, Chuck Lever wrote: >>>>> On Jan 12, 2021, at 9:3

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-21 Thread Chuck Lever
> On Jan 18, 2021, at 1:00 PM, Robin Murphy wrote: > > On 2021-01-18 16:18, Chuck Lever wrote: >>> On Jan 12, 2021, at 9:38 AM, Will Deacon wrote: >>> >>> [Expanding cc list to include DMA-IOMMU and intel IOMMU folks] >>> >>> On Fr

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-20 Thread Chuck Lever
> On Jan 19, 2021, at 9:11 PM, Lu Baolu wrote: > > On 1/19/21 10:37 PM, Chuck Lever wrote: >>> On Jan 18, 2021, at 8:22 PM, Lu Baolu wrote: >>> >>> Do you mind posting the cap and ecap of the iommu used by your device? >>> >>> You ca

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-19 Thread Chuck Lever
p_sg(30) returns 1 kworker/u28:3-1269 [000] 336.055312: bprint: frwr_map: ib_dma_map_sg(30) returns 1 kworker/u28:3-1269 [000] 336.055407: bprint: frwr_map: ib_dma_map_sg(4) returns 1 -- Chuck Lever ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-18 Thread Chuck Lever
> On Jan 18, 2021, at 1:00 PM, Robin Murphy wrote: > > On 2021-01-18 16:18, Chuck Lever wrote: >>> On Jan 12, 2021, at 9:38 AM, Will Deacon wrote: >>> >>> [Expanding cc list to include DMA-IOMMU and intel IOMMU folks] >>> >>> On Fr

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-18 Thread Chuck Lever
> On Jan 12, 2021, at 9:38 AM, Will Deacon wrote: > > [Expanding cc list to include DMA-IOMMU and intel IOMMU folks] > > On Fri, Jan 08, 2021 at 04:18:36PM -0500, Chuck Lever wrote: >> Hi- >> >> [ Please cc: me on replies, I'm not currently subscribed to >

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-13 Thread Chuck Lever
> On Jan 13, 2021, at 9:07 AM, Chuck Lever wrote: > > > >> On Jan 12, 2021, at 9:25 PM, Lu Baolu wrote: >> >> Hi, >> >> On 1/12/21 10:38 PM, Will Deacon wrote: >>> [Expanding cc list to include DMA-IOMMU and intel IOMMU folks] >&g

Re: performance regression noted in v5.11-rc after c062db039f40

2021-01-13 Thread Chuck Lever
> On Jan 12, 2021, at 9:25 PM, Lu Baolu wrote: > > Hi, > > On 1/12/21 10:38 PM, Will Deacon wrote: >> [Expanding cc list to include DMA-IOMMU and intel IOMMU folks] >> On Fri, Jan 08, 2021 at 04:18:36PM -0500, Chuck Lever wrote: >>> Hi- >>> >

performance regression noted in v5.11-rc after c062db039f40

2021-01-08 Thread Chuck Lever
ization 65.74 % The regression appears to be 100% reproducible. -- Chuck Lever ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH v2] xprtrdma: Fix DMA scatter-gather list mapping imbalance

2020-02-12 Thread Chuck Lever
> On Feb 12, 2020, at 11:03 AM, Andre Tomt wrote: > > On 12.02.2020 14:48, Chuck Lever wrote: >>> On Feb 12, 2020, at 8:43 AM, Chuck Lever wrote: >>> >>> The @nents value that was passed to ib_dma_map_sg() has to be passed >>> to the matc

Re: [PATCH v2] xprtrdma: Fix DMA scatter-gather list mapping imbalance

2020-02-12 Thread Chuck Lever
> On Feb 12, 2020, at 8:43 AM, Chuck Lever wrote: > > The @nents value that was passed to ib_dma_map_sg() has to be passed > to the matching ib_dma_unmap_sg() call. If ib_dma_map_sg() choses to > concatenate sg entries, it will return a different nents value than > it was pa

[PATCH v2] xprtrdma: Fix DMA scatter-gather list mapping imbalance

2020-02-12 Thread Chuck Lever
sg entry concatenation. Looking all the way back to 4143f34e01e9 ("xprtrdma: Port to new memory registration API") and reviewing other kernel ULPs, it's not clear that the frwr_map() logic was ever correct for this case. Reported-by: Andre Tomt Suggested-by: Robin Murphy Signed-off

Re: [PATCH v1] xprtrdma: Fix DMA scatter-gather list mapping imbalance

2020-02-11 Thread Chuck Lever
Hi Andre, thanks for trying this out. > On Feb 11, 2020, at 3:50 PM, Andre Tomt wrote: > > On 11.02.2020 20:58, Chuck Lever wrote: >> The @nents value that was passed to ib_dma_map_sg() has to be passed >> to the matching ib_dma_unmap_sg() call. If ib_dma_map_sg() choses

[PATCH v1] xprtrdma: Fix DMA scatter-gather list mapping imbalance

2020-02-11 Thread Chuck Lever
-by: Andre Tomt Suggested-by: Robin Murphy Fixes: 1f541895dae9 ("xprtrdma: Don't defer MR recovery if ro_map fails") Signed-off-by: Chuck Lever --- net/sunrpc/xprtrdma/frwr_ops.c |5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) Hey Andre, please try this out. It just rever

Re: AMD IOMMU stops RDMA NFS from working since kernel 5.5 (bisected)

2020-02-11 Thread Chuck Lever
> On Feb 11, 2020, at 11:36 AM, Robin Murphy wrote: > > On 11/02/2020 4:03 pm, Chuck Lever wrote: >>> On Feb 11, 2020, at 10:32 AM, Robin Murphy wrote: >>> >>> On 11/02/2020 3:24 pm, Chuck Lever wrote: >>>>> On Feb 11, 2020, at 10:12 AM, R

Re: AMD IOMMU stops RDMA NFS from working since kernel 5.5 (bisected)

2020-02-11 Thread Chuck Lever
> On Feb 11, 2020, at 10:32 AM, Robin Murphy wrote: > > On 11/02/2020 3:24 pm, Chuck Lever wrote: >>> On Feb 11, 2020, at 10:12 AM, Robin Murphy wrote: >>> >>> On 11/02/2020 1:48 pm, Chuck Lever wrote: >>>> Andre- >>>> Thank you

Re: AMD IOMMU stops RDMA NFS from working since kernel 5.5 (bisected)

2020-02-11 Thread Chuck Lever
> On Feb 11, 2020, at 10:12 AM, Robin Murphy wrote: > > On 11/02/2020 1:48 pm, Chuck Lever wrote: >> Andre- >> Thank you for the detailed report! >> Tom- >> There is a rich set of trace points available in the RPC/RDMA implementation >> in 5.4/5.5, fwiw

Re: AMD IOMMU stops RDMA NFS from working since kernel 5.5 (bisected)

2020-02-11 Thread Chuck Lever
ltiple PCI DMA aliases in IRQ Remapping >>> git bisect bad 3c124435e8dd516df4b2fc983f4415386fd6edae >>> # bad: [be62dbf554c5b50718a54a359372c148cd9975c7] iommu/amd: Convert AMD >>> iommu driver to the dma-iommu api >>> git bisect bad be62dbf554c5b50718a54a359372c148cd9975c7 >>>

Re: [PATCH v4 7/8] linux/log2.h: Fix 64bit calculations in roundup/down_pow_two()

2019-12-03 Thread Chuck Lever
-1094,7 +1094,8 @@ static int __init crash_notes_memory_init(void) >* crash_notes is allocated inside one physical page. >*/ > size = sizeof(note_buf_t); > - align = min(roundup_pow_of_two(sizeof(note_buf_t)), PAGE_SIZE); > + align = min(roundup