RE: [PATCH 5/6] dma-mapping/iommu: Add dma_set_max_opt_size()
(+) correction below, sorry for the typo in earlier post.
> From: iommu [mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of
> Robin Murphy
> Sent: Friday, March 19, 2021 5:00 PM
> To: John Garry ; j...@8bytes.org; w...@kernel.org;
> j...@linux.ibm.com; martin.peter...@oracle.com; h...@lst.de;
> m.szyprow...@samsung.com
> Cc: iommu@lists.linux-foundation.org; linux-ker...@vger.kernel.org;
> linux-s...@vger.kernel.org; Linuxarm
> Subject: Re: [PATCH 5/6] dma-mapping/iommu: Add dma_set_max_opt_size()
>
> On 2021-03-19 13:25, John Garry wrote:
> > Add a function to allow the max size which we want to optimise DMA mappings
> > for.
>
> It seems neat in theory - particularly for packet-based interfaces that
> might have a known fixed size of data unit that they're working on at
> any given time - but aren't there going to be many cases where the
> driver has no idea because it depends on whatever size(s) of request
> userspace happens to throw at it? Even if it does know the absolute
> maximum size of thing it could ever transfer, that could be
> impractically large in areas like video/AI/etc., so it could still be
> hard to make a reasonable decision.
This is also the case for networking workloads where we have MTU set but
actual packet sizes might vary.
>
> Being largely workload-dependent is why I still think this should be a
> command-line or sysfs tuneable - we could set the default based on how
> much total memory is available, but ultimately it's the end user who
> knows what the workload is going to be and what they care about
> optimising for.
>
> Another thought (which I'm almost reluctant to share) is that I would
> *love* to try implementing a self-tuning strategy that can detect high
> contention on particular allocation sizes and adjust the caches on the
> fly, but I can easily imagine that having enough inherent overhead to
> end up being an impractical (but fun) waste of time.
This might be particularly useful for the NICs where packet sizes vary
from 64B to 9K. But without optimal strategy this can affect the
performance of networking workloads.
>
> Robin.
>
> > Signed-off-by: John Garry
> > ---
> > drivers/iommu/dma-iommu.c | 2 +-
> > include/linux/dma-map-ops.h | 1 +
> > include/linux/dma-mapping.h | 5 +
> > kernel/dma/mapping.c| 11 +++
> > 4 files changed, 18 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index a5dfbd6c0496..d35881fcfb9c 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -447,7 +447,6 @@ static dma_addr_t iommu_dma_alloc_iova(struct
> > iommu_domain
> *domain,
> > return (dma_addr_t)iova << shift;
> > }
> >
> > -__maybe_unused
> > static void iommu_dma_set_opt_size(struct device *dev, size_t size)
> > {
> > struct iommu_domain *domain = iommu_get_dma_domain(dev);
> > @@ -1278,6 +1277,7 @@ static const struct dma_map_ops iommu_dma_ops = {
> > .map_resource = iommu_dma_map_resource,
> > .unmap_resource = iommu_dma_unmap_resource,
> > .get_merge_boundary = iommu_dma_get_merge_boundary,
> > + .set_max_opt_size = iommu_dma_set_opt_size,
> > };
> >
> > /*
> > diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
> > index 51872e736e7b..fed7a183b3b9 100644
> > --- a/include/linux/dma-map-ops.h
> > +++ b/include/linux/dma-map-ops.h
> > @@ -64,6 +64,7 @@ struct dma_map_ops {
> > u64 (*get_required_mask)(struct device *dev);
> > size_t (*max_mapping_size)(struct device *dev);
> > unsigned long (*get_merge_boundary)(struct device *dev);
> > + void (*set_max_opt_size)(struct device *dev, size_t size);
> > };
> >
> > #ifdef CONFIG_DMA_OPS
> > diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> > index 2a984cb4d1e0..91fe770145d4 100644
> > --- a/include/linux/dma-mapping.h
> > +++ b/include/linux/dma-mapping.h
> > @@ -144,6 +144,7 @@ u64 dma_get_required_mask(struct device *dev);
> > size_t dma_max_mapping_size(struct device *dev);
> > bool dma_need_sync(struct device *dev, dma_addr_t dma_addr);
> > unsigned long dma_get_merge_boundary(struct device *dev);
> > +void dma_set_max_opt_size(struct device *dev, size_t size);
> > #else /* CONFIG_HAS_DMA */
> > static inline dma_addr_t dma_map_page_attrs(struct device *dev,
> > struct page *page, size_t offset, size_t size,
> > @@ -257,6 +258,10 @@ static inline unsigned long
> > dma_get_merge_boundary(struct
> device *dev)
> > {
> > return 0;
> > }
> > +static inline void dma_set_max_opt_size(struct device *dev, size_t size)
> > +{
> > +}
> > +
> > #endif /* CONFIG_HAS_DMA */
> >
> > struct page *dma_alloc_pages(struct device *dev, size_t size,
> > diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
> > index b6a633679933..59e6acb1c471 100644
> > --- a/kernel/dma/mapping.c
> > +++ b/kernel/dma/mapping.c
> > @@ -608,3 +608,14 @@
RE: [PATCH 5/6] dma-mapping/iommu: Add dma_set_max_opt_size()
> From: iommu [mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of
> Robin Murphy
> Sent: Friday, March 19, 2021 5:00 PM
> To: John Garry ; j...@8bytes.org; w...@kernel.org;
> j...@linux.ibm.com; martin.peter...@oracle.com; h...@lst.de;
> m.szyprow...@samsung.com
> Cc: iommu@lists.linux-foundation.org; linux-ker...@vger.kernel.org;
> linux-s...@vger.kernel.org; Linuxarm
> Subject: Re: [PATCH 5/6] dma-mapping/iommu: Add dma_set_max_opt_size()
>
> On 2021-03-19 13:25, John Garry wrote:
> > Add a function to allow the max size which we want to optimise DMA mappings
> > for.
>
> It seems neat in theory - particularly for packet-based interfaces that
> might have a known fixed size of data unit that they're working on at
> any given time - but aren't there going to be many cases where the
> driver has no idea because it depends on whatever size(s) of request
> userspace happens to throw at it? Even if it does know the absolute
> maximum size of thing it could ever transfer, that could be
> impractically large in areas like video/AI/etc., so it could still be
> hard to make a reasonable decision.
This is also the case in networking workloads where we have MTU set but
actual packet sizes might vary.
>
> Being largely workload-dependent is why I still think this should be a
> command-line or sysfs tuneable - we could set the default based on how
> much total memory is available, but ultimately it's the end user who
> knows what the workload is going to be and what they care about
> optimising for.
>
> Another thought (which I'm almost reluctant to share) is that I would
> *love* to try implementing a self-tuning strategy that can detect high
> contention on particular allocation sizes and adjust the caches on the
> fly, but I can easily imagine that having enough inherent overhead to
> end up being an impractical (but fun) waste of time.
This might be particularly useful for the NICs where packet sizes vary
from 64K to 9K. Hence, without optimal strategy this can affect the
performance of networking workloads.
>
> Robin.
>
> > Signed-off-by: John Garry
> > ---
> > drivers/iommu/dma-iommu.c | 2 +-
> > include/linux/dma-map-ops.h | 1 +
> > include/linux/dma-mapping.h | 5 +
> > kernel/dma/mapping.c| 11 +++
> > 4 files changed, 18 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/iommu/dma-iommu.c b/drivers/iommu/dma-iommu.c
> > index a5dfbd6c0496..d35881fcfb9c 100644
> > --- a/drivers/iommu/dma-iommu.c
> > +++ b/drivers/iommu/dma-iommu.c
> > @@ -447,7 +447,6 @@ static dma_addr_t iommu_dma_alloc_iova(struct
> > iommu_domain
> *domain,
> > return (dma_addr_t)iova << shift;
> > }
> >
> > -__maybe_unused
> > static void iommu_dma_set_opt_size(struct device *dev, size_t size)
> > {
> > struct iommu_domain *domain = iommu_get_dma_domain(dev);
> > @@ -1278,6 +1277,7 @@ static const struct dma_map_ops iommu_dma_ops = {
> > .map_resource = iommu_dma_map_resource,
> > .unmap_resource = iommu_dma_unmap_resource,
> > .get_merge_boundary = iommu_dma_get_merge_boundary,
> > + .set_max_opt_size = iommu_dma_set_opt_size,
> > };
> >
> > /*
> > diff --git a/include/linux/dma-map-ops.h b/include/linux/dma-map-ops.h
> > index 51872e736e7b..fed7a183b3b9 100644
> > --- a/include/linux/dma-map-ops.h
> > +++ b/include/linux/dma-map-ops.h
> > @@ -64,6 +64,7 @@ struct dma_map_ops {
> > u64 (*get_required_mask)(struct device *dev);
> > size_t (*max_mapping_size)(struct device *dev);
> > unsigned long (*get_merge_boundary)(struct device *dev);
> > + void (*set_max_opt_size)(struct device *dev, size_t size);
> > };
> >
> > #ifdef CONFIG_DMA_OPS
> > diff --git a/include/linux/dma-mapping.h b/include/linux/dma-mapping.h
> > index 2a984cb4d1e0..91fe770145d4 100644
> > --- a/include/linux/dma-mapping.h
> > +++ b/include/linux/dma-mapping.h
> > @@ -144,6 +144,7 @@ u64 dma_get_required_mask(struct device *dev);
> > size_t dma_max_mapping_size(struct device *dev);
> > bool dma_need_sync(struct device *dev, dma_addr_t dma_addr);
> > unsigned long dma_get_merge_boundary(struct device *dev);
> > +void dma_set_max_opt_size(struct device *dev, size_t size);
> > #else /* CONFIG_HAS_DMA */
> > static inline dma_addr_t dma_map_page_attrs(struct device *dev,
> > struct page *page, size_t offset, size_t size,
> > @@ -257,6 +258,10 @@ static inline unsigned long
> > dma_get_merge_boundary(struct
> device *dev)
> > {
> > return 0;
> > }
> > +static inline void dma_set_max_opt_size(struct device *dev, size_t size)
> > +{
> > +}
> > +
> > #endif /* CONFIG_HAS_DMA */
> >
> > struct page *dma_alloc_pages(struct device *dev, size_t size,
> > diff --git a/kernel/dma/mapping.c b/kernel/dma/mapping.c
> > index b6a633679933..59e6acb1c471 100644
> > --- a/kernel/dma/mapping.c
> > +++ b/kernel/dma/mapping.c
> > @@ -608,3 +608,14 @@ unsigned long dma_get_merge_boundary(struct device
> > *dev)
>
RE: iommu_iova slab eats too much memory
> From: Robin Murphy [mailto:robin.mur...@arm.com] > Sent: Wednesday, April 29, 2020 3:00 PM > To: Salil Mehta ; Bin > > On 2020-04-29 2:37 pm, Salil Mehta wrote: > > Hi Bin, > > > >> From: Bin [mailto:anole1...@gmail.com] > >> Sent: Wednesday, April 29, 2020 5:14 AM > >> To: Salil Mehta > >> Hi Shlil: > >> > >> Thank you for your attention, and these are my answers: > >> > >> 1. I don't really understand what you're saying. What's the difference > >> between > DMA buffer and DMA mapping? > >> It's like a memory block pool and a memory block or something like that? > > > > > > DMA Mapping: Mapping are translations/associations [IOVA<->HPA OR > > IOVA<->GPA(further translated > > to HPA by Stage-2)] which are created by the NIC driver. IOMMU hardware > > responsible for NIC > > IOVA translations is populated with the mappings by the driver before > > submitting the DMA buffer > > to the hardware for TX/RX. > > > > DMA buffers: Actual Memory allocated by the driver where data could be > > DMA'ed (RX'ed or TX'ed) > > > > > > I think you have missed the important point I mentioned earlier: > > If there is a leak of IOVA mapping due to dma_unmap_* not being called > > somewhere then at > > certain point the throughput will drastically fall and will almost become > > equal to zero. > > This is due to the exhaustion of available IOVA mapping space in the IOMMU > > hardware. > > With 64-bit address spaces, you're still likely to run out of memory for > the IOVA structures and pagetables before you run out of the actual > address space that they represent. I see. Good point and it was non-obvious. > The slowdown comes from having to > walk the whole the rbtree to search for free space or free a PFN, but > depending on how the allocation pattern interacts with the caching > mechanism that may never happen to a significant degree. So assuming, due to above limitation of the algorithm allocation of such free mapping space gets delayed, this should only help in more availability of the system memory in general unless this also affects the release of the mappings - perhaps I am missing something here? > > Above condition is very much different than a *memory leak* of the DMA > > buffer > itself which > > will eventually lead to OOM. > > > > > > Salil. > > > >> FYI: > >> I found an interesting phenomenon that it's just a small part of the > >> running > hosts has this issue, even though they all > >> have the same kernel, configuration and hardwares, I don't know if this > >> really > mean something. > > Another thought for a debugging sanity check is to look at the > intel-iommu tracepoints on a misbehaving system and see whether maps vs. > unmaps look significantly out of balance. You could probably do > something clever with ftrace to look for that kind of pattern in teh DMA > API calls, too. > > Robin. > > >> > >> > >> Salil Mehta 于2020年4月28日周二 下午5:17写道： > >> Hi Bin, > >> > >> Few questions: > >> > >> 1. If there is a leak of IOVA due to dma_unmap_* not being called somewhere > then > >> at certain point the throughput will drastically fall and will almost > >> become > >> equal > >> to zero. This should be due to unavailability of the mapping anymore. But > in > >> your > >> case VM is getting killed so this could be actual DMA buffer leak not DMA > mapping > >> leak. I doubt VM will get killed due to exhaustion of the DMA mappings in > the > >> IOMMU > >> Layer for a transient reason or even due to mapping/unmapping leak. > >> > >> 2. Could you check if you have TSO offload enabled on Intel 82599? It will > help > >> in reducing the number of mappings and will take off IOVA mapping pressure > from > >> the IOMMU/VT-d? Though I am not sure it will help in reducing the amount of > memory > >> required for the buffers. > >> > >> 3. Also, have you checked the cpu-usage while your experiment is going on? > >> > >> Thanks > >> Salil. > >> > >>> -Original Message- > >>> From: iommu [mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of > >>> Robin Murphy > >>> Sent: Friday, April 24, 2020 5:31 PM > >>> To: Bin > >>> Cc: iommu@lists.linux-foundation.org > >>> Subject: Re: iommu_iova slab eats too much memory > >
RE: iommu_iova slab eats too much memory
Hi Bin, > From: Bin [mailto:anole1...@gmail.com] > Sent: Wednesday, April 29, 2020 5:14 AM > To: Salil Mehta > Hi Shlil: > > Thank you for your attention, and these are my answers: > > 1. I don't really understand what you're saying. What's the difference > between DMA buffer and DMA mapping? > It's like a memory block pool and a memory block or something like that? DMA Mapping: Mapping are translations/associations [IOVA<->HPA OR IOVA<->GPA(further translated to HPA by Stage-2)] which are created by the NIC driver. IOMMU hardware responsible for NIC IOVA translations is populated with the mappings by the driver before submitting the DMA buffer to the hardware for TX/RX. DMA buffers: Actual Memory allocated by the driver where data could be DMA'ed (RX'ed or TX'ed) I think you have missed the important point I mentioned earlier: If there is a leak of IOVA mapping due to dma_unmap_* not being called somewhere then at certain point the throughput will drastically fall and will almost become equal to zero. This is due to the exhaustion of available IOVA mapping space in the IOMMU hardware. Above condition is very much different than a *memory leak* of the DMA buffer itself which will eventually lead to OOM. Salil. > FYI: > I found an interesting phenomenon that it's just a small part of the running > hosts has this issue, even though they all > have the same kernel, configuration and hardwares, I don't know if this > really mean something. > > > Salil Mehta 于2020年4月28日周二 下午5:17写道： > Hi Bin, > > Few questions: > > 1. If there is a leak of IOVA due to dma_unmap_* not being called somewhere > then > at certain point the throughput will drastically fall and will almost become > equal > to zero. This should be due to unavailability of the mapping anymore. But in > your > case VM is getting killed so this could be actual DMA buffer leak not DMA > mapping > leak. I doubt VM will get killed due to exhaustion of the DMA mappings in the > IOMMU > Layer for a transient reason or even due to mapping/unmapping leak. > > 2. Could you check if you have TSO offload enabled on Intel 82599? It will > help > in reducing the number of mappings and will take off IOVA mapping pressure > from > the IOMMU/VT-d? Though I am not sure it will help in reducing the amount of > memory > required for the buffers. > > 3. Also, have you checked the cpu-usage while your experiment is going on? > > Thanks > Salil. > > > -Original Message- > > From: iommu [mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of > > Robin Murphy > > Sent: Friday, April 24, 2020 5:31 PM > > To: Bin > > Cc: iommu@lists.linux-foundation.org > > Subject: Re: iommu_iova slab eats too much memory > > > > On 2020-04-24 2:20 pm, Bin wrote: > > > Dear Robin: > > > Thank you for your explanation. Now, I understand that this could be > > > NIC driver's fault, but how could I confirm it? Do I have to debug the > > > driver myself? > > > > I'd start with CONFIG_DMA_API_DEBUG - of course it will chew through > > memory about an order of magnitude faster than the IOVAs alone, but it > > should shed some light on whether DMA API usage looks suspicious, and > > dumping the mappings should help track down the responsible driver(s). > > Although the debugfs code doesn't show the stacktrace of where each > > mapping was made, I guess it would be fairly simple to tweak that for a > > quick way to narrow down where to start looking in an offending driver. > > > > Robin. > > > > > Robin Murphy 于2020年4月24日周五 下午8:15写道： > > > > > >> On 2020-04-24 1:06 pm, Bin wrote: > > >>> I'm not familiar with the mmu stuff, so what you mean by "some driver > > >>> leaking DMA mappings", is it possible that some other kernel module like > > >>> KVM or NIC driver leads to the leaking problem instead of the iommu > > >> module > > >>> itself? > > >> > > >> Yes - I doubt that intel-iommu itself is failing to free IOVAs when it > > >> should, since I'd expect a lot of people to have noticed that. It's far > > >> more likely that some driver is failing to call dma_unmap_* when it's > > >> finished with a buffer - with the IOMMU disabled that would be a no-op > > >> on x86 with a modern 64-bit-capable device, so such a latent bug could > > >> have been easily overlooked. > > >> > > >> Robin. > > >> > > >>> Bin 于 2020年4月24日周五 20:00写道： > > >>> > > >>>
RE: iommu_iova slab eats too much memory
Hi Bin, Few questions: 1. If there is a leak of IOVA due to dma_unmap_* not being called somewhere then at certain point the throughput will drastically fall and will almost become equal to zero. This should be due to unavailability of the mapping anymore. But in your case VM is getting killed so this could be actual DMA buffer leak not DMA mapping leak. I doubt VM will get killed due to exhaustion of the DMA mappings in the IOMMU Layer for a transient reason or even due to mapping/unmapping leak. 2. Could you check if you have TSO offload enabled on Intel 82599? It will help in reducing the number of mappings and will take off IOVA mapping pressure from the IOMMU/VT-d? Though I am not sure it will help in reducing the amount of memory required for the buffers. 3. Also, have you checked the cpu-usage while your experiment is going on? Thanks Salil. > -Original Message- > From: iommu [mailto:iommu-boun...@lists.linux-foundation.org] On Behalf Of > Robin Murphy > Sent: Friday, April 24, 2020 5:31 PM > To: Bin > Cc: iommu@lists.linux-foundation.org > Subject: Re: iommu_iova slab eats too much memory > > On 2020-04-24 2:20 pm, Bin wrote: > > Dear Robin: > > Thank you for your explanation. Now, I understand that this could be > > NIC driver's fault, but how could I confirm it? Do I have to debug the > > driver myself? > > I'd start with CONFIG_DMA_API_DEBUG - of course it will chew through > memory about an order of magnitude faster than the IOVAs alone, but it > should shed some light on whether DMA API usage looks suspicious, and > dumping the mappings should help track down the responsible driver(s). > Although the debugfs code doesn't show the stacktrace of where each > mapping was made, I guess it would be fairly simple to tweak that for a > quick way to narrow down where to start looking in an offending driver. > > Robin. > > > Robin Murphy 于2020年4月24日周五 下午8:15写道： > > > >> On 2020-04-24 1:06 pm, Bin wrote: > >>> I'm not familiar with the mmu stuff, so what you mean by "some driver > >>> leaking DMA mappings", is it possible that some other kernel module like > >>> KVM or NIC driver leads to the leaking problem instead of the iommu > >> module > >>> itself? > >> > >> Yes - I doubt that intel-iommu itself is failing to free IOVAs when it > >> should, since I'd expect a lot of people to have noticed that. It's far > >> more likely that some driver is failing to call dma_unmap_* when it's > >> finished with a buffer - with the IOMMU disabled that would be a no-op > >> on x86 with a modern 64-bit-capable device, so such a latent bug could > >> have been easily overlooked. > >> > >> Robin. > >> > >>> Bin 于 2020年4月24日周五 20:00写道： > >>> > Well, that's the problem! I'm assuming the iommu kernel module is > >> leaking > memory. But I don't know why and how. > > Do you have any idea about it? Or any further information is needed? > > Robin Murphy 于 2020年4月24日周五 19:20写道： > > > On 2020-04-24 1:40 am, Bin wrote: > >> Hello? anyone there? > >> > >> Bin 于2020年4月23日周四 下午5:14写道： > >> > >>> Forget to mention, I've already disabled the slab merge, so this is > > what > >>> it is. > >>> > >>> Bin 于2020年4月23日周四 下午5:11写道： > >>> > Hey, guys: > > I'm running a batch of CoreOS boxes, the lsb_release is: > > ``` > # cat /etc/lsb-release > DISTRIB_ID="Container Linux by CoreOS" > DISTRIB_RELEASE=2303.3.0 > DISTRIB_CODENAME="Rhyolite" > DISTRIB_DESCRIPTION="Container Linux by CoreOS 2303.3.0 (Rhyolite)" > ``` > > ``` > # uname -a > Linux cloud-worker-25 4.19.86-coreos #1 SMP Mon Dec 2 20:13:38 -00 > > 2019 > x86_64 Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz GenuineIntel > > GNU/Linux > ``` > Recently, I found my vms constently being killed due to OOM, and > >> after > digging into the problem, I finally realized that the kernel is > > leaking > memory. > > Here's my slabinfo: > > Active / Total Objects (% used): 83818306 / 84191607 (99.6%) > Active / Total Slabs (% used) : 1336293 / 1336293 (100.0%) > Active / Total Caches (% used) : 152 / 217 (70.0%) > Active / Total Size (% used) : 5828768.08K / 5996848.72K > > (97.2%) > Minimum / Average / Maximum Object : 0.01K / 0.07K / 23.25K > > OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME > > 80253888 80253888 100%0.06K 1253967 64 5015868K > >> iommu_iova > > > > Do you really have a peak demand of ~80 million simultaneous DMA > > buffers, or is some driver leaking DMA mappings? > > > > Robin. > > > 489472 489123 99%0.03K 3824 128 15296K kmalloc-32 > >
