Re: [RESEND PATCH v3 0/4] iommu/iova: Solve longterm IOVA issue
+ linux-scsi (see https://lore.kernel.org/linux-iommu/1607538189-237944-4-git-send-email-john.ga...@huawei.com/) On 17/11/2020 10:25, John Garry wrote: This series contains a patch to solve the longterm IOVA issue which leizhen originally tried to address at [0]. A sieved kernel log is at the following, showing periodic dumps of IOVA sizes, per CPU and per depot bin, per IOVA size granule: https://raw.githubusercontent.com/hisilicon/kernel-dev/topic-iommu-5.10-iova-debug-v3/aging_test Notice, for example, the following logs: [13175.355584] print_iova1 cpu_total=40135 depot_total=3866 total=44001 [83483.457858] print_iova1 cpu_total=62532 depot_total=24476 total=87008 Where total IOVA rcache size has grown from 44K->87K over a long time. JFYI, I am able to reproduce this aging issue on another storage card, an LSI SAS 3008, so now it's harder to say it's an issue specific to a (buggy) single driver. A log of the IOVA size dumps is here: https://raw.githubusercontent.com/hisilicon/kernel-dev/064c4dc8869b3f2ad07edffceafde0b129f276b0/lsi3008_dmesg Notice again how the total IOVA size goes up over time, like: [ 68.176914] print_iova1 cpu_total=23663 depot_total=256 total=23919 [ 2337.008194] print_iova1 cpu_total=67361 depot_total=9088 total=76449 [17141.860078] print_iova1 cpu_total=73397 depot_total=10368 total=83765 [27087.850830] print_iova1 cpu_total=73386 depot_total=10624 total=84010 [10434.042877] print_iova1 cpu_total=90652 depot_total=12928 total=103580 I had to change some settings for that storage card to reproduce, though [0]. Could explain why no other reports. So please consider this issue again... Thanks, john [0] https://lore.kernel.org/linux-scsi/dd8e6fdc-397d-b6ad-3371-0b65d1932...@huawei.com/T/#m953d21446a5756981412c92d0924ca65c8d2f3a5 Along with this patch, I included the following: - A smaller helper to clear all IOVAs for a domain - Change polarity of the IOVA magazine helpers - Small optimisation from Cong Wang included, which was never applied [1]. There was some debate of the other patches in that series, but this one is quite straightforward. Differnces to v2: - Update commit message for patch 3/4 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RESEND PATCH v3 0/4] iommu/iova: Solve longterm IOVA issue
On 03/12/2020 06:04, Dmitry Safonov wrote:
On Tue, 1 Dec 2020 at 21:50, Will Deacon wrote:
On Tue, 17 Nov 2020 18:25:30 +0800, John Garry wrote:
This series contains a patch to solve the longterm IOVA issue which
leizhen originally tried to address at [0].
A sieved kernel log is at the following, showing periodic dumps of IOVA
sizes, per CPU and per depot bin, per IOVA size granule:
https://raw.githubusercontent.com/hisilicon/kernel-dev/topic-iommu-5.10-iova-debug-v3/aging_test
[...]
Applied the final patch to arm64 (for-next/iommu/iova), thanks!
[4/4] iommu: avoid taking iova_rbtree_lock twice
https://git.kernel.org/arm64/c/3a651b3a27a1
Glad it made in next, 2 years ago I couldn't convince iommu maintainer
it's worth it (but with a different justification):
https://lore.kernel.org/linux-iommu/20180621180823.805-3-d...@arista.com/
Hi Dmitry,
I was unaware of your series, and it’s unfortunate that your
optimization never made it. However I was having a quick look there,
and, in case you did not notice, that the code which you were proposing
changing in patch #1 for intel-iommu.c was removed in e70b081c6f37
("iommu/vt-d: Remove IOVA handling code from the non-dma_ops path").
BTW, split_and_remove_iova() has no in-tree users anymore, so I can send
a patch to delete if nobody else wants to.
BTW2, there's some more patches in my series which could use a review if
you're feeling very helpful :)
Cheers,
John
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RESEND PATCH v3 0/4] iommu/iova: Solve longterm IOVA issue
On Tue, 1 Dec 2020 at 21:50, Will Deacon wrote: > > On Tue, 17 Nov 2020 18:25:30 +0800, John Garry wrote: > > This series contains a patch to solve the longterm IOVA issue which > > leizhen originally tried to address at [0]. > > > > A sieved kernel log is at the following, showing periodic dumps of IOVA > > sizes, per CPU and per depot bin, per IOVA size granule: > > https://raw.githubusercontent.com/hisilicon/kernel-dev/topic-iommu-5.10-iova-debug-v3/aging_test > > > > [...] > > Applied the final patch to arm64 (for-next/iommu/iova), thanks! > > [4/4] iommu: avoid taking iova_rbtree_lock twice > https://git.kernel.org/arm64/c/3a651b3a27a1 Glad it made in next, 2 years ago I couldn't convince iommu maintainer it's worth it (but with a different justification): https://lore.kernel.org/linux-iommu/20180621180823.805-3-d...@arista.com/ Thanks, Dmitry ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RESEND PATCH v3 0/4] iommu/iova: Solve longterm IOVA issue
On 01/12/2020 21:02, Will Deacon wrote: cc'ing some more people who have touched iova code recently On Tue, Dec 01, 2020 at 03:35:02PM +, John Garry wrote: On 17/11/2020 10:25, John Garry wrote: Is there any chance that we can get these picked up for 5.11? We've seen this issue solved here for a long time. Or, @Robin, let me know if not happy with this since v1. BTW, patch #4 has been on the go for ~1 year now, and is a nice small optimisation from Cong, which I picked up and already had a RB tag. I can pick the last patch up, but I'd really like some reviewed/tested-bys on the others. ok, fair enough. Considering the extremes required to unearth the main problem, it'll be hard to get testers, but, fwiw, I can provide a tested-by from the reporter: Tested-by: Xiang Chen @Robin, You originally had some interest in this topic - are you now satisfied with the changes I am proposing? Please let me know. Thanks, John ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RESEND PATCH v3 0/4] iommu/iova: Solve longterm IOVA issue
On Tue, 17 Nov 2020 18:25:30 +0800, John Garry wrote: > This series contains a patch to solve the longterm IOVA issue which > leizhen originally tried to address at [0]. > > A sieved kernel log is at the following, showing periodic dumps of IOVA > sizes, per CPU and per depot bin, per IOVA size granule: > https://raw.githubusercontent.com/hisilicon/kernel-dev/topic-iommu-5.10-iova-debug-v3/aging_test > > [...] Applied the final patch to arm64 (for-next/iommu/iova), thanks! [4/4] iommu: avoid taking iova_rbtree_lock twice https://git.kernel.org/arm64/c/3a651b3a27a1 Cheers, -- Will https://fixes.arm64.dev https://next.arm64.dev https://will.arm64.dev ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RESEND PATCH v3 0/4] iommu/iova: Solve longterm IOVA issue
On Tue, Dec 01, 2020 at 03:35:02PM +, John Garry wrote: > On 17/11/2020 10:25, John Garry wrote: > Is there any chance that we can get these picked up for 5.11? We've seen > this issue solved here for a long time. > > Or, @Robin, let me know if not happy with this since v1. > > BTW, patch #4 has been on the go for ~1 year now, and is a nice small > optimisation from Cong, which I picked up and already had a RB tag. I can pick the last patch up, but I'd really like some reviewed/tested-bys on the others. Will ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [RESEND PATCH v3 0/4] iommu/iova: Solve longterm IOVA issue
On 17/11/2020 10:25, John Garry wrote: Hi Will, Is there any chance that we can get these picked up for 5.11? We've seen this issue solved here for a long time. Or, @Robin, let me know if not happy with this since v1. BTW, patch #4 has been on the go for ~1 year now, and is a nice small optimisation from Cong, which I picked up and already had a RB tag. Thanks, John This series contains a patch to solve the longterm IOVA issue which leizhen originally tried to address at [0]. A sieved kernel log is at the following, showing periodic dumps of IOVA sizes, per CPU and per depot bin, per IOVA size granule: https://raw.githubusercontent.com/hisilicon/kernel-dev/topic-iommu-5.10-iova-debug-v3/aging_test Notice, for example, the following logs: [13175.355584] print_iova1 cpu_total=40135 depot_total=3866 total=44001 [83483.457858] print_iova1 cpu_total=62532 depot_total=24476 total=87008 Where total IOVA rcache size has grown from 44K->87K over a long time. Along with this patch, I included the following: - A smaller helper to clear all IOVAs for a domain - Change polarity of the IOVA magazine helpers - Small optimisation from Cong Wang included, which was never applied [1]. There was some debate of the other patches in that series, but this one is quite straightforward. Differnces to v2: - Update commit message for patch 3/4 Differences to v1: - Add IOVA clearing helper - Add patch to change polarity of mag helpers - Avoid logically-redundant extra variable in __iova_rcache_insert() [0] https://lore.kernel.org/linux-iommu/20190815121104.29140-3-thunder.leiz...@huawei.com/ [1] https://lore.kernel.org/linux-iommu/4b74d40a-22d1-af53-fcb6-5d7018370...@huawei.com/ Cong Wang (1): iommu: avoid taking iova_rbtree_lock twice John Garry (3): iommu/iova: Add free_all_cpu_cached_iovas() iommu/iova: Avoid double-negatives in magazine helpers iommu/iova: Flush CPU rcache for when a depot fills drivers/iommu/iova.c | 66 +--- 1 file changed, 38 insertions(+), 28 deletions(-) ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[RESEND PATCH v3 0/4] iommu/iova: Solve longterm IOVA issue
This series contains a patch to solve the longterm IOVA issue which leizhen originally tried to address at [0]. A sieved kernel log is at the following, showing periodic dumps of IOVA sizes, per CPU and per depot bin, per IOVA size granule: https://raw.githubusercontent.com/hisilicon/kernel-dev/topic-iommu-5.10-iova-debug-v3/aging_test Notice, for example, the following logs: [13175.355584] print_iova1 cpu_total=40135 depot_total=3866 total=44001 [83483.457858] print_iova1 cpu_total=62532 depot_total=24476 total=87008 Where total IOVA rcache size has grown from 44K->87K over a long time. Along with this patch, I included the following: - A smaller helper to clear all IOVAs for a domain - Change polarity of the IOVA magazine helpers - Small optimisation from Cong Wang included, which was never applied [1]. There was some debate of the other patches in that series, but this one is quite straightforward. Differnces to v2: - Update commit message for patch 3/4 Differences to v1: - Add IOVA clearing helper - Add patch to change polarity of mag helpers - Avoid logically-redundant extra variable in __iova_rcache_insert() [0] https://lore.kernel.org/linux-iommu/20190815121104.29140-3-thunder.leiz...@huawei.com/ [1] https://lore.kernel.org/linux-iommu/4b74d40a-22d1-af53-fcb6-5d7018370...@huawei.com/ Cong Wang (1): iommu: avoid taking iova_rbtree_lock twice John Garry (3): iommu/iova: Add free_all_cpu_cached_iovas() iommu/iova: Avoid double-negatives in magazine helpers iommu/iova: Flush CPU rcache for when a depot fills drivers/iommu/iova.c | 66 +--- 1 file changed, 38 insertions(+), 28 deletions(-) -- 2.26.2 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
