Re: [Intel-gfx] [PATCH v3 0/6] Convert the intel iommu driver to the dma-iommu api
On 2020-09-22 3:51 a.m., Robin Murphy wrote:
> On 2020-09-18 21:47, Logan Gunthorpe wrote:
>> Hi Lu,
>>
>> On 2020-09-11 9:21 p.m., Lu Baolu wrote:
>>> Tom Murphy has almost done all the work. His latest patch series was
>>> posted here.
>>>
>>> https://lore.kernel.org/linux-iommu/20200903201839.7327-1-murph...@tcd.ie/
>>>
>>> Thanks a lot!
>>>
>>> This series is a follow-up with below changes:
>>>
>>> 1. Add a quirk for the i915 driver issue described in Tom's cover
>>> letter.
>>> 2. Fix several bugs in patch "iommu: Allow the dma-iommu api to use
>>> bounce buffers" to make the bounce buffer work for untrusted devices.
>>> 3. Several cleanups in iommu/vt-d driver after the conversion.
>>>
>>
>> I'm trying to test this on an old Sandy Bridge, but found that I get
>> spammed with warnings on boot. I've put a sample of a few of them below.
>> They all seem to be related to ioat.
>>
>> I had the same issue with Tom's v2 but never saw this on his v1.
>
> I think this might have more to do with ioat being totally broken -
> AFAICS it appears to allocate descriptors with a size of 2MB, but free
> them with a size of 512KB. Try throwing CONFIG_DMA_API_DEBUG at it to
> confirm.
Ah, yes, nice catch. Looks like it was broken recently by the following
commit, but nobody noticed and the dma-iommu patch set added a warning
which caught it.
a02254f8a676 ("dmaengine: ioat: Decreasing allocation chunk size 2M->512K")
Reverting that fixes the issue. I'll try to send patch or two for this.
Logan
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Intel-gfx] [PATCH v3 0/6] Convert the intel iommu driver to the dma-iommu api
On 2020-09-21 6:24 p.m., Lu Baolu wrote: I'm trying to test this on an old Sandy Bridge, but found that I get spammed with warnings on boot. I've put a sample of a few of them below. They all seem to be related to ioat. I had the same issue with Tom's v2 but never saw this on his v1. >>> >>> Have you verified whether this could be reproduced with the lasted >>> upstream kernel (without this patch series)? >> >> Yes. > > I am sorry. Just want to make sure I understand you correctly. :-) When > you say "yes", do you mean you could reproduce this with pure upstream > kernel (5.9-rc6)? I mean I've verified the bug does *not* exist without this patch set. > >> Also, it's hitting a warning in the dma-iommu code which would not >> be hit without this patch set. > > Without this series, DMA APIs don't go through dma-iommu. Do you mind > posting the warning message? It was on my original email but here it is again: [ 44.057877] [ cut here ] [ 44.063167] WARNING: CPU: 4 PID: 84 at drivers/iommu/dma-iommu.c:496 __iommu_dma_unmap+0x115/0x130 [ 44.073351] Modules linked in: [ 44.076882] CPU: 4 PID: 84 Comm: kworker/4:1 Tainted: GW 5.9.0-rc4-6-g110da1e703a2 #216 [ 44.087935] Hardware name: Supermicro SYS-7047GR-TRF/X9DRG-QF, BIOS 3.0a 12/05/2013 [ 44.096650] Workqueue: events work_for_cpu_fn [ 44.101644] RIP: 0010:__iommu_dma_unmap+0x115/0x130 [ 44.107225] Code: 48 8b 0c 24 48 c7 44 24 10 00 00 00 00 48 c7 44 24 18 00 00 00 00 48 c7 44 24 20 00 00 00 00 48 c7 44 24 08 ff ff ff ff eb 8d <0f> 0b e9 74 ff ff ff e8 1f 36 66 00 66 66 2e 0f 1f 84 00 00 00 00 [ 44.128443] RSP: :c90002397c38 EFLAGS: 00010206 [ 44.134413] RAX: 0020 RBX: 0008 RCX: [ 44.144487] RDX: 0403 RSI: 82a7fb20 RDI: 888477804f30 [ 44.152613] RBP: fec0 R08: 000fec00 R09: 000fedff [ 44.160733] R10: 0002 R11: 0004 R12: 888270c39000 [ 44.168861] R13: 888472d85ee8 R14: 0008 R15: [ 44.176980] FS: () GS:88847980() knlGS: [ 44.186198] CS: 0010 DS: ES: CR0: 80050033 [ 44.192761] CR2: CR3: 02a20001 CR4: 000606e0 [ 44.200871] Call Trace: [ 44.203716] ? lockdep_hardirqs_on_prepare+0xad/0x180 [ 44.209509] iommu_dma_free+0x18/0x30 [ 44.213734] ioat_free_chan_resources+0x19e/0x300 [ 44.219133] ioat_dma_self_test+0x2a0/0x3d0 [ 44.223964] ioat_pci_probe+0x60e/0x903 [ 44.228387] local_pci_probe+0x42/0x80 [ 44.232721] work_for_cpu_fn+0x16/0x20 [ 44.237037] process_one_work+0x292/0x630 [ 44.241644] ? __schedule+0x344/0x970 [ 44.245868] worker_thread+0x227/0x3e0 [ 44.250185] ? process_one_work+0x630/0x630 [ 44.254989] kthread+0x136/0x150 [ 44.258718] ? kthread_use_mm+0x100/0x100 [ 44.263334] ret_from_fork+0x22/0x30 [ 44.267474] irq event stamp: 1881262 [ 44.271602] hardirqs last enabled at (1881272): [] console_unlock+0x435/0x570 [ 44.281593] hardirqs last disabled at (1881281): [] console_unlock+0x42b/0x570 [ 44.291588] softirqs last enabled at (1747140): [] ioat_cleanup+0x65/0x470 [ 44.301285] softirqs last disabled at (1747144): [] ioat_free_chan_resources+0x6a/0x300 [ 44.312153] ---[ end trace ed0f88fd959a5a39 ]--- ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Intel-gfx] [PATCH v3 0/6] Convert the intel iommu driver to the dma-iommu api
On 2020-09-18 21:47, Logan Gunthorpe wrote: Hi Lu, On 2020-09-11 9:21 p.m., Lu Baolu wrote: Tom Murphy has almost done all the work. His latest patch series was posted here. https://lore.kernel.org/linux-iommu/20200903201839.7327-1-murph...@tcd.ie/ Thanks a lot! This series is a follow-up with below changes: 1. Add a quirk for the i915 driver issue described in Tom's cover letter. 2. Fix several bugs in patch "iommu: Allow the dma-iommu api to use bounce buffers" to make the bounce buffer work for untrusted devices. 3. Several cleanups in iommu/vt-d driver after the conversion. I'm trying to test this on an old Sandy Bridge, but found that I get spammed with warnings on boot. I've put a sample of a few of them below. They all seem to be related to ioat. I had the same issue with Tom's v2 but never saw this on his v1. I think this might have more to do with ioat being totally broken - AFAICS it appears to allocate descriptors with a size of 2MB, but free them with a size of 512KB. Try throwing CONFIG_DMA_API_DEBUG at it to confirm. Robin. Thanks, Logan [ 44.057877] [ cut here ] [ 44.063167] WARNING: CPU: 4 PID: 84 at drivers/iommu/dma-iommu.c:496 __iommu_dma_unmap+0x115/0x130 [ 44.073351] Modules linked in: [ 44.076882] CPU: 4 PID: 84 Comm: kworker/4:1 Tainted: GW 5.9.0-rc4-6-g110da1e703a2 #216 [ 44.087935] Hardware name: Supermicro SYS-7047GR-TRF/X9DRG-QF, BIOS 3.0a 12/05/2013 [ 44.096650] Workqueue: events work_for_cpu_fn [ 44.101644] RIP: 0010:__iommu_dma_unmap+0x115/0x130 [ 44.107225] Code: 48 8b 0c 24 48 c7 44 24 10 00 00 00 00 48 c7 44 24 18 00 00 00 00 48 c7 44 24 20 00 00 00 00 48 c7 44 24 08 ff ff ff ff eb 8d <0f> 0b e9 74 ff ff ff e8 1f 36 66 00 66 66 2e 0f 1f 84 00 00 00 00 [ 44.128443] RSP: :c90002397c38 EFLAGS: 00010206 [ 44.134413] RAX: 0020 RBX: 0008 RCX: [ 44.144487] RDX: 0403 RSI: 82a7fb20 RDI: 888477804f30 [ 44.152613] RBP: fec0 R08: 000fec00 R09: 000fedff [ 44.160733] R10: 0002 R11: 0004 R12: 888270c39000 [ 44.168861] R13: 888472d85ee8 R14: 0008 R15: [ 44.176980] FS: () GS:88847980() knlGS: [ 44.186198] CS: 0010 DS: ES: CR0: 80050033 [ 44.192761] CR2: CR3: 02a20001 CR4: 000606e0 [ 44.200871] Call Trace: [ 44.203716] ? lockdep_hardirqs_on_prepare+0xad/0x180 [ 44.209509] iommu_dma_free+0x18/0x30 [ 44.213734] ioat_free_chan_resources+0x19e/0x300 [ 44.219133] ioat_dma_self_test+0x2a0/0x3d0 [ 44.223964] ioat_pci_probe+0x60e/0x903 [ 44.228387] local_pci_probe+0x42/0x80 [ 44.232721] work_for_cpu_fn+0x16/0x20 [ 44.237037] process_one_work+0x292/0x630 [ 44.241644] ? __schedule+0x344/0x970 [ 44.245868] worker_thread+0x227/0x3e0 [ 44.250185] ? process_one_work+0x630/0x630 [ 44.254989] kthread+0x136/0x150 [ 44.258718] ? kthread_use_mm+0x100/0x100 [ 44.263334] ret_from_fork+0x22/0x30 [ 44.267474] irq event stamp: 1881262 [ 44.271602] hardirqs last enabled at (1881272): [] console_unlock+0x435/0x570 [ 44.281593] hardirqs last disabled at (1881281): [] console_unlock+0x42b/0x570 [ 44.291588] softirqs last enabled at (1747140): [] ioat_cleanup+0x65/0x470 [ 44.301285] softirqs last disabled at (1747144): [] ioat_free_chan_resources+0x6a/0x300 [ 44.312153] ---[ end trace ed0f88fd959a5a39 ]--- [ 44.353963] [ cut here ] [ 44.359291] WARNING: CPU: 4 PID: 84 at drivers/iommu/dma-iommu.c:496 __iommu_dma_unmap+0x115/0x130 [ 44.369482] Modules linked in: [ 44.373030] CPU: 4 PID: 84 Comm: kworker/4:1 Tainted: GW 5.9.0-rc4-6-g110da1e703a2 #216 [ 44.384097] Hardware name: Supermicro SYS-7047GR-TRF/X9DRG-QF, BIOS 3.0a 12/05/2013 [ 44.392825] Workqueue: events work_for_cpu_fn [ 44.397831] RIP: 0010:__iommu_dma_unmap+0x115/0x130 [ 44.403421] Code: 48 8b 0c 24 48 c7 44 24 10 00 00 00 00 48 c7 44 24 18 00 00 00 00 48 c7 44 24 20 00 00 00 00 48 c7 44 24 08 ff ff ff ff eb 8d <0f> 0b e9 74 ff ff ff e8 1f 36 66 00 66 66 2e 0f 1f 84 00 00 00 00 [ 44.424644] RSP: :c90002397c38 EFLAGS: 00010206 [ 44.430627] RAX: 0020 RBX: 0008 RCX: [ 44.438770] RDX: 0403 RSI: 82a7fb20 RDI: 888477804f30 [ 44.446885] RBP: ffc0 R08: 000ffc00 R09: 000ffdff [ 44.455000] R10: 0002 R11: 0004 R12: 888270c55000 [ 44.463119] R13: 888472dc7268 R14: 0008 R15: [ 44.471235] FS: () GS:88847980() knlGS: [ 44.480440] CS: 0010 DS: ES: CR0: 80050033 [ 44.487004] CR2: CR3: 02a20001 CR4: 000606e0 [
Re: [Intel-gfx] [PATCH v3 0/6] Convert the intel iommu driver to the dma-iommu api
Hi Logan, On 9/21/20 11:48 PM, Logan Gunthorpe wrote: On 2020-09-20 12:36 a.m., Lu Baolu wrote: Hi Logan, On 2020/9/19 4:47, Logan Gunthorpe wrote: Hi Lu, On 2020-09-11 9:21 p.m., Lu Baolu wrote: Tom Murphy has almost done all the work. His latest patch series was posted here. https://lore.kernel.org/linux-iommu/20200903201839.7327-1-murph...@tcd.ie/ Thanks a lot! This series is a follow-up with below changes: 1. Add a quirk for the i915 driver issue described in Tom's cover letter. 2. Fix several bugs in patch "iommu: Allow the dma-iommu api to use bounce buffers" to make the bounce buffer work for untrusted devices. 3. Several cleanups in iommu/vt-d driver after the conversion. I'm trying to test this on an old Sandy Bridge, but found that I get spammed with warnings on boot. I've put a sample of a few of them below. They all seem to be related to ioat. I had the same issue with Tom's v2 but never saw this on his v1. Have you verified whether this could be reproduced with the lasted upstream kernel (without this patch series)? Yes. I am sorry. Just want to make sure I understand you correctly. :-) When you say "yes", do you mean you could reproduce this with pure upstream kernel (5.9-rc6)? Also, it's hitting a warning in the dma-iommu code which would not be hit without this patch set. Without this series, DMA APIs don't go through dma-iommu. Do you mind posting the warning message? Best regards, baolu ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Intel-gfx] [PATCH v3 0/6] Convert the intel iommu driver to the dma-iommu api
On 2020-09-20 12:36 a.m., Lu Baolu wrote: > Hi Logan, > > On 2020/9/19 4:47, Logan Gunthorpe wrote: >> Hi Lu, >> >> On 2020-09-11 9:21 p.m., Lu Baolu wrote: >>> Tom Murphy has almost done all the work. His latest patch series was >>> posted here. >>> >>> https://lore.kernel.org/linux-iommu/20200903201839.7327-1-murph...@tcd.ie/ >>> >>> Thanks a lot! >>> >>> This series is a follow-up with below changes: >>> >>> 1. Add a quirk for the i915 driver issue described in Tom's cover >>> letter. >>> 2. Fix several bugs in patch "iommu: Allow the dma-iommu api to use >>> bounce buffers" to make the bounce buffer work for untrusted devices. >>> 3. Several cleanups in iommu/vt-d driver after the conversion. >>> >> >> I'm trying to test this on an old Sandy Bridge, but found that I get >> spammed with warnings on boot. I've put a sample of a few of them below. >> They all seem to be related to ioat. >> >> I had the same issue with Tom's v2 but never saw this on his v1. > > Have you verified whether this could be reproduced with the lasted > upstream kernel (without this patch series)? Yes. Also, it's hitting a warning in the dma-iommu code which would not be hit without this patch set. Thanks, Logan ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Intel-gfx] [PATCH v3 0/6] Convert the intel iommu driver to the dma-iommu api
Hi Logan, On 2020/9/19 4:47, Logan Gunthorpe wrote: Hi Lu, On 2020-09-11 9:21 p.m., Lu Baolu wrote: Tom Murphy has almost done all the work. His latest patch series was posted here. https://lore.kernel.org/linux-iommu/20200903201839.7327-1-murph...@tcd.ie/ Thanks a lot! This series is a follow-up with below changes: 1. Add a quirk for the i915 driver issue described in Tom's cover letter. 2. Fix several bugs in patch "iommu: Allow the dma-iommu api to use bounce buffers" to make the bounce buffer work for untrusted devices. 3. Several cleanups in iommu/vt-d driver after the conversion. I'm trying to test this on an old Sandy Bridge, but found that I get spammed with warnings on boot. I've put a sample of a few of them below. They all seem to be related to ioat. I had the same issue with Tom's v2 but never saw this on his v1. Have you verified whether this could be reproduced with the lasted upstream kernel (without this patch series)? Best regards, baolu ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [Intel-gfx] [PATCH v3 0/6] Convert the intel iommu driver to the dma-iommu api
Hi Lu, On 2020-09-11 9:21 p.m., Lu Baolu wrote: > Tom Murphy has almost done all the work. His latest patch series was > posted here. > > https://lore.kernel.org/linux-iommu/20200903201839.7327-1-murph...@tcd.ie/ > > Thanks a lot! > > This series is a follow-up with below changes: > > 1. Add a quirk for the i915 driver issue described in Tom's cover > letter. > 2. Fix several bugs in patch "iommu: Allow the dma-iommu api to use > bounce buffers" to make the bounce buffer work for untrusted devices. > 3. Several cleanups in iommu/vt-d driver after the conversion. > I'm trying to test this on an old Sandy Bridge, but found that I get spammed with warnings on boot. I've put a sample of a few of them below. They all seem to be related to ioat. I had the same issue with Tom's v2 but never saw this on his v1. Thanks, Logan [ 44.057877] [ cut here ] [ 44.063167] WARNING: CPU: 4 PID: 84 at drivers/iommu/dma-iommu.c:496 __iommu_dma_unmap+0x115/0x130 [ 44.073351] Modules linked in: [ 44.076882] CPU: 4 PID: 84 Comm: kworker/4:1 Tainted: GW 5.9.0-rc4-6-g110da1e703a2 #216 [ 44.087935] Hardware name: Supermicro SYS-7047GR-TRF/X9DRG-QF, BIOS 3.0a 12/05/2013 [ 44.096650] Workqueue: events work_for_cpu_fn [ 44.101644] RIP: 0010:__iommu_dma_unmap+0x115/0x130 [ 44.107225] Code: 48 8b 0c 24 48 c7 44 24 10 00 00 00 00 48 c7 44 24 18 00 00 00 00 48 c7 44 24 20 00 00 00 00 48 c7 44 24 08 ff ff ff ff eb 8d <0f> 0b e9 74 ff ff ff e8 1f 36 66 00 66 66 2e 0f 1f 84 00 00 00 00 [ 44.128443] RSP: :c90002397c38 EFLAGS: 00010206 [ 44.134413] RAX: 0020 RBX: 0008 RCX: [ 44.144487] RDX: 0403 RSI: 82a7fb20 RDI: 888477804f30 [ 44.152613] RBP: fec0 R08: 000fec00 R09: 000fedff [ 44.160733] R10: 0002 R11: 0004 R12: 888270c39000 [ 44.168861] R13: 888472d85ee8 R14: 0008 R15: [ 44.176980] FS: () GS:88847980() knlGS: [ 44.186198] CS: 0010 DS: ES: CR0: 80050033 [ 44.192761] CR2: CR3: 02a20001 CR4: 000606e0 [ 44.200871] Call Trace: [ 44.203716] ? lockdep_hardirqs_on_prepare+0xad/0x180 [ 44.209509] iommu_dma_free+0x18/0x30 [ 44.213734] ioat_free_chan_resources+0x19e/0x300 [ 44.219133] ioat_dma_self_test+0x2a0/0x3d0 [ 44.223964] ioat_pci_probe+0x60e/0x903 [ 44.228387] local_pci_probe+0x42/0x80 [ 44.232721] work_for_cpu_fn+0x16/0x20 [ 44.237037] process_one_work+0x292/0x630 [ 44.241644] ? __schedule+0x344/0x970 [ 44.245868] worker_thread+0x227/0x3e0 [ 44.250185] ? process_one_work+0x630/0x630 [ 44.254989] kthread+0x136/0x150 [ 44.258718] ? kthread_use_mm+0x100/0x100 [ 44.263334] ret_from_fork+0x22/0x30 [ 44.267474] irq event stamp: 1881262 [ 44.271602] hardirqs last enabled at (1881272): [] console_unlock+0x435/0x570 [ 44.281593] hardirqs last disabled at (1881281): [] console_unlock+0x42b/0x570 [ 44.291588] softirqs last enabled at (1747140): [] ioat_cleanup+0x65/0x470 [ 44.301285] softirqs last disabled at (1747144): [] ioat_free_chan_resources+0x6a/0x300 [ 44.312153] ---[ end trace ed0f88fd959a5a39 ]--- [ 44.353963] [ cut here ] [ 44.359291] WARNING: CPU: 4 PID: 84 at drivers/iommu/dma-iommu.c:496 __iommu_dma_unmap+0x115/0x130 [ 44.369482] Modules linked in: [ 44.373030] CPU: 4 PID: 84 Comm: kworker/4:1 Tainted: GW 5.9.0-rc4-6-g110da1e703a2 #216 [ 44.384097] Hardware name: Supermicro SYS-7047GR-TRF/X9DRG-QF, BIOS 3.0a 12/05/2013 [ 44.392825] Workqueue: events work_for_cpu_fn [ 44.397831] RIP: 0010:__iommu_dma_unmap+0x115/0x130 [ 44.403421] Code: 48 8b 0c 24 48 c7 44 24 10 00 00 00 00 48 c7 44 24 18 00 00 00 00 48 c7 44 24 20 00 00 00 00 48 c7 44 24 08 ff ff ff ff eb 8d <0f> 0b e9 74 ff ff ff e8 1f 36 66 00 66 66 2e 0f 1f 84 00 00 00 00 [ 44.424644] RSP: :c90002397c38 EFLAGS: 00010206 [ 44.430627] RAX: 0020 RBX: 0008 RCX: [ 44.438770] RDX: 0403 RSI: 82a7fb20 RDI: 888477804f30 [ 44.446885] RBP: ffc0 R08: 000ffc00 R09: 000ffdff [ 44.455000] R10: 0002 R11: 0004 R12: 888270c55000 [ 44.463119] R13: 888472dc7268 R14: 0008 R15: [ 44.471235] FS: () GS:88847980() knlGS: [ 44.480440] CS: 0010 DS: ES: CR0: 80050033 [ 44.487004] CR2: CR3: 02a20001 CR4: 000606e0 [ 44.495128] Call Trace: [ 44.498008] iommu_dma_free+0x18/0x30 [ 44.502230] ioat_free_chan_resources+0x19e/0x300 [ 44.507631] ioat_dma_self_test+0x2a0/0x3d0 [ 44.512466] ioat_pci_probe+0x60e/0x903 [ 44.516889] local_pci_probe+0x42/0x80 [
