RE: Re: dma_alloc_coherent not allocating memory from CMA Reserved
Hi Robin, My question is during boot up the CMA was allocated at [ 0.014538] *cma: Reserved 400 MiB at 0x000205c0* But when we do dma_alloac_coherent in the driver the address is different f800 The address f800 doesnt lie in the range of CMA allocated at 0x000205c0. Is this still vald? I have one more question regarding CMA. If the system is Intel VTD enabled, then the dma_alloc_coherent for the VF fails. Regards, Sathya - Original Message - Sender : Robin Murphy Date : 2020-09-23 19:23 (GMT+5:30) Title : Re: dma_alloc_coherent not allocating memory from CMA Reserved To : Sathyavathi M, null On 2020-09-23 08:43, Sathyavathi M wrote: > Hi All, > > I am trying to allocate coherent memory for 33 MB in kerenl driver. and for that > i have reserved CMA of 1024 MB, but from dmesg, i can see that address reserved > for cma is different and what i get with dma_alloc_coherent is different. My pc > is intel x86 machine and tried in different motherboard, but this issue is > occuring in some specific PCs. please help me in debugging the actual issue, > below are kernel bootup logs > > > [ 0.014362] No NUMA configuration found > [ 0.014363] Faking a node at [mem 0x-0x00021edf] > [ 0.014374] NODE_DATA(0) allocated [mem 0x21edd5000-0x21edf] > [ 0.014538] *cma: Reserved 400 MiB at 0x000205c0* > [ 0.014541] Reserving 640MB of memory at 2512MB for crashkernel (System RAM: 8046MB) > [ 0.014553] Zone ranges: > [ 0.014554] DMA [mem 0x1000-0x00ff] > [ 0.014554] DMA32 [mem 0x0100-0x] > [ 0.014555] Normal [mem 0x0001-0x00021edf] > > > *at dma_alloc_coherent call > [ 27.816062] dev->dma_33M_addr is f800* I'm confused - if you got a DMA address back, then the allocation must have succeeded, so what exactly is the issue there? If the allocator managed to find a suitable amount of memory that your device can address, does it really matter exactly where it came from? Also bear in mind that a DMA address is not necessarily the same as a physical address anyway, if for instance you have an IOMMU or other forms of interconnect translation. In fact the default behaviour of the intel-iommu driver for a PCI device is to allocate DMA addresses in naturally-aligned regions downwards from 4GB - if the first request in an empty address space was for 33MB, it would get rounded up to 64MB, wouldn't fit at 4GB - 64MB (0xfc00) because that would clash with the IOAPIC region, so would end up at 4GB - 128MB (0xf800). Now where have I seen that before?... Robin. > --- > and below are the logs in working case, at driver dma_alloc_coherent api call we > have address which is in range of what reserved for cma. > at boot i get.. > Faking a node at [mem 0x-0x00019fdf] > NODE_DATA(0) allocated [mem 0x19fdd3000-0x19fdfdfff] > *cma: Reserved 800 MiB at 0x00016dc0* > Reserving 640MB of memory at 1792MB for crashkernel (System RAM: 6016MB) > Zone ranges: > DMA [mem 0x1000-0x00ff] > DMA32 [mem 0x0100-0x] > Normal [mem 0x0001-0x00019fdf] > > *at dma_alloc_coherent call > dev->dma_33M_addr is 16e20* > > Please help me in solving this iissue, or can suggest any alternative way to > allocate big coherent memory. > > Regards, > > Sathya > > > ___ > iommu mailing list > iommu@lists.linux-foundation.org > https://lists.linuxfoundation.org/mailman/listinfo/iommu > Regards, Sathya ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: dma_alloc_coherent not allocating memory from CMA Reserved
On 2020-09-23 08:43, Sathyavathi M wrote: Hi All, I am trying to allocate coherent memory for 33 MB in kerenl driver. and for that i have reserved CMA of 1024 MB, but from dmesg, i can see that address reserved for cma is different and what i get with dma_alloc_coherent is different. My pc is intel x86 machine and tried in different motherboard, but this issue is occuring in some specific PCs. please help me in debugging the actual issue, below are kernel bootup logs [ 0.014362] No NUMA configuration found [ 0.014363] Faking a node at [mem 0x-0x00021edf] [ 0.014374] NODE_DATA(0) allocated [mem 0x21edd5000-0x21edf] [ 0.014538] *cma: Reserved 400 MiB at 0x000205c0* [ 0.014541] Reserving 640MB of memory at 2512MB for crashkernel (System RAM: 8046MB) [ 0.014553] Zone ranges: [ 0.014554] DMA [mem 0x1000-0x00ff] [ 0.014554] DMA32 [mem 0x0100-0x] [ 0.014555] Normal [mem 0x0001-0x00021edf] *at dma_alloc_coherent call [ 27.816062] dev->dma_33M_addr is f800* I'm confused - if you got a DMA address back, then the allocation must have succeeded, so what exactly is the issue there? If the allocator managed to find a suitable amount of memory that your device can address, does it really matter exactly where it came from? Also bear in mind that a DMA address is not necessarily the same as a physical address anyway, if for instance you have an IOMMU or other forms of interconnect translation. In fact the default behaviour of the intel-iommu driver for a PCI device is to allocate DMA addresses in naturally-aligned regions downwards from 4GB - if the first request in an empty address space was for 33MB, it would get rounded up to 64MB, wouldn't fit at 4GB - 64MB (0xfc00) because that would clash with the IOAPIC region, so would end up at 4GB - 128MB (0xf800). Now where have I seen that before?... Robin. --- and below are the logs in working case, at driver dma_alloc_coherent api call we have address which is in range of what reserved for cma. at boot i get.. Faking a node at [mem 0x-0x00019fdf] NODE_DATA(0) allocated [mem 0x19fdd3000-0x19fdfdfff] *cma: Reserved 800 MiB at 0x00016dc0* Reserving 640MB of memory at 1792MB for crashkernel (System RAM: 6016MB) Zone ranges: DMA [mem 0x1000-0x00ff] DMA32 [mem 0x0100-0x] Normal [mem 0x0001-0x00019fdf] *at dma_alloc_coherent call dev->dma_33M_addr is 16e20* Please help me in solving this iissue, or can suggest any alternative way to allocate big coherent memory. Regards, Sathya ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
dma_alloc_coherent not allocating memory from CMA Reserved
Hi All, I am trying to allocate coherent memory for 33 MB in kerenl driver. and for that i have reserved CMA of 1024 MB, but from dmesg, i can see that address reserved for cma is different and what i get with dma_alloc_coherent is different. My pc is intel x86 machine and tried in different motherboard, but this issue is occuring in some specific PCs. please help me in debugging the actual issue, below are kernel bootup logs [ 0.014362] No NUMA configuration found[ 0.014363] Faking a node at [mem 0x-0x00021edf][ 0.014374] NODE_DATA(0) allocated [mem 0x21edd5000-0x21edf][ 0.014538] cma: Reserved 400 MiB at 0x000205c0[ 0.014541] Reserving 640MB of memory at 2512MB for crashkernel (System RAM: 8046MB)[ 0.014553] Zone ranges:[ 0.014554] DMA [mem 0x1000-0x00ff][ 0.014554] DMA32 [mem 0x0100-0x][ 0.014555] Normal [mem 0x0001-0x00021edf] at dma_alloc_coherent call[ 27.816062] dev->dma_33M_addr is f800---and below are the logs in working case, at driver dma_alloc_coherent api call we have address which is in range of what reserved for cma.at boot i get..Faking a node at [mem 0x-0x00019fdf]NODE_DATA(0) allocated [mem 0x19fdd3000-0x19fdfdfff]cma: Reserved 800 MiB at 0x00016dc0Reserving 640MB of memory at 1792MB for crashkernel (System RAM: 6016MB)Zone ranges:DMA [mem 0x1000-0x00ff]DMA32 [mem 0x0100-0x]Normal [mem 0x0001-0x00019fdf]at dma_alloc_coherent calldev->dma_33M_addr is 16e20 Please help me in solving this iissue, or can suggest any alternative way to allocate big coherent memory. Regards, Sathya ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
dma_alloc_coherent not allocating memory from CMA Reserved.
Hi All, I am trying to allocate coherent memory for 33 MB in kerenl driver. and for that i have reserved CMA of 200 MB, but from dmesg, i can see that address reserved for cma is different and what i get with dma_alloc_coherent is different. My pc is x86 machine and tried in different motherboard, but this issue is occuring in some specific PCs. please help me in debugging the actual issue, below are kernel logs..at init i boot... [ 0.014362] No NUMA configuration found[ 0.014363] Faking a node at [mem 0x-0x00021edf][ 0.014374] NODE_DATA(0) allocated [mem 0x21edd5000-0x21edf][ 0.014538] cma: Reserved 400 MiB at 0x000205c0[ 0.014541] Reserving 640MB of memory at 2512MB for crashkernel (System RAM: 8046MB)[ 0.014553] Zone ranges:[ 0.014554] DMA [mem 0x1000-0x00ff][ 0.014554] DMA32 [mem 0x0100-0x][ 0.014555] Normal [mem 0x0001-0x00021edf] at dma_alloc_coherent call[ 27.816062] dev->dma_33M_addr is f800---and below are the logs in working case, at driver dma_alloc_coherent api call we have address which is in range of what reserved for cma.at boot i get..Faking a node at [mem 0x-0x00019fdf]NODE_DATA(0) allocated [mem 0x19fdd3000-0x19fdfdfff]cma: Reserved 800 MiB at 0x00016dc0Reserving 640MB of memory at 1792MB for crashkernel (System RAM: 6016MB)Zone ranges:DMA [mem 0x1000-0x00ff]DMA32 [mem 0x0100-0x]Normal [mem 0x0001-0x00019fdf]at dma_alloc_coherent calldev->dma_33M_addr is 16e20 Please help me in solving this iissue, or can suggest any alternative way to allocate big coherent memory. Thanks and Regards, Ankit Soni ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
