RE: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA
> -Original Message- > From: Will Deacon [mailto:w...@kernel.org] > Sent: Friday, August 21, 2020 9:27 PM > To: Song Bao Hua (Barry Song) > Cc: h...@lst.de; m.szyprow...@samsung.com; robin.mur...@arm.com; > ganapatrao.kulka...@cavium.com; catalin.mari...@arm.com; > iommu@lists.linux-foundation.org; Linuxarm ; > linux-arm-ker...@lists.infradead.org; linux-ker...@vger.kernel.org; > huangdaode ; Jonathan Cameron > ; Nicolas Saenz Julienne > ; Steve Capper ; Andrew > Morton ; Mike Rapoport > Subject: Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve > per-numa CMA > > On Fri, Aug 21, 2020 at 09:13:39AM +, Song Bao Hua (Barry Song) wrote: > > > > > > > -Original Message- > > > From: Will Deacon [mailto:w...@kernel.org] > > > Sent: Friday, August 21, 2020 8:47 PM > > > To: Song Bao Hua (Barry Song) > > > Cc: h...@lst.de; m.szyprow...@samsung.com; robin.mur...@arm.com; > > > ganapatrao.kulka...@cavium.com; catalin.mari...@arm.com; > > > iommu@lists.linux-foundation.org; Linuxarm ; > > > linux-arm-ker...@lists.infradead.org; linux-ker...@vger.kernel.org; > > > huangdaode ; Jonathan Cameron > > > ; Nicolas Saenz Julienne > > > ; Steve Capper ; > > > Andrew Morton ; Mike Rapoport > > > > > > Subject: Re: [PATCH v6 1/2] dma-contiguous: provide the ability to > > > reserve per-numa CMA > > > > > > On Fri, Aug 21, 2020 at 02:26:14PM +1200, Barry Song wrote: > > > > diff --git a/Documentation/admin-guide/kernel-parameters.txt > > > b/Documentation/admin-guide/kernel-parameters.txt > > > > index bdc1f33fd3d1..3f33b89aeab5 100644 > > > > --- a/Documentation/admin-guide/kernel-parameters.txt > > > > +++ b/Documentation/admin-guide/kernel-parameters.txt > > > > @@ -599,6 +599,15 @@ > > > > altogether. For more information, see > > > > include/linux/dma-contiguous.h > > > > > > > > + pernuma_cma=nn[MG] > > > > + [ARM64,KNL] > > > > + Sets the size of kernel per-numa memory area for > > > > + contiguous memory allocations. A value of 0 > > > > disables > > > > + per-numa CMA altogether. DMA users on node nid > > > > will > > > > + first try to allocate buffer from the pernuma > > > > area > > > > + which is located in node nid, if the allocation > > > > fails, > > > > + they will fallback to the global default memory > > > > area. > > > > > > What is the default behaviour if this option is not specified? Seems > > > like that should be mentioned here. > > Just wanted to make sure you didn't miss this ^^ If it is not specified, the default size is 0 that means pernuma_cma is disabled. Will put some words for this. > > > > > > > > diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig index > > > > 847a9d1fa634..db7a37ed35eb 100644 > > > > --- a/kernel/dma/Kconfig > > > > +++ b/kernel/dma/Kconfig > > > > @@ -118,6 +118,16 @@ config DMA_CMA > > > > If unsure, say "n". > > > > > > > > if DMA_CMA > > > > + > > > > +config DMA_PERNUMA_CMA > > > > + bool "Enable separate DMA Contiguous Memory Area for each > NUMA > > > Node" > > > > > > I don't understand the need for this config option. If you have > > > DMA_DMA and you have NUMA, why wouldn't you want this enabled? > > > > Christoph preferred this in previous patchset in order to be able to > > remove all of the code in the text if users don't use pernuma CMA. > > Ok, I defer to Christoph here, but maybe a "default NUMA" might work? maybe "default NUMA && ARM64"? Though I believe it will benefit x86, but I don't have a x86 server hardware and real scenario to test. So I haven't put the dma_pernuma_cma_reserve() code in arch/x86. Hopefully some x86 guys will bring it up and remove the "&& ARM64". > > > > > + help > > > > + Enable this option to get pernuma CMA areas so that devices > > > > like > > > > + ARM64 SMMU can get local memory by DMA coherent APIs. > > > > + > > > > + You can set the size of pernuma CMA by specifying
Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA
On Fri, Aug 21, 2020 at 09:13:39AM +, Song Bao Hua (Barry Song) wrote:
>
>
> > -Original Message-
> > From: Will Deacon [mailto:w...@kernel.org]
> > Sent: Friday, August 21, 2020 8:47 PM
> > To: Song Bao Hua (Barry Song)
> > Cc: h...@lst.de; m.szyprow...@samsung.com; robin.mur...@arm.com;
> > ganapatrao.kulka...@cavium.com; catalin.mari...@arm.com;
> > iommu@lists.linux-foundation.org; Linuxarm ;
> > linux-arm-ker...@lists.infradead.org; linux-ker...@vger.kernel.org;
> > huangdaode ; Jonathan Cameron
> > ; Nicolas Saenz Julienne
> > ; Steve Capper ; Andrew
> > Morton ; Mike Rapoport
> > Subject: Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve
> > per-numa CMA
> >
> > On Fri, Aug 21, 2020 at 02:26:14PM +1200, Barry Song wrote:
> > > diff --git a/Documentation/admin-guide/kernel-parameters.txt
> > b/Documentation/admin-guide/kernel-parameters.txt
> > > index bdc1f33fd3d1..3f33b89aeab5 100644
> > > --- a/Documentation/admin-guide/kernel-parameters.txt
> > > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > > @@ -599,6 +599,15 @@
> > > altogether. For more information, see
> > > include/linux/dma-contiguous.h
> > >
> > > + pernuma_cma=nn[MG]
> > > + [ARM64,KNL]
> > > + Sets the size of kernel per-numa memory area for
> > > + contiguous memory allocations. A value of 0 disables
> > > + per-numa CMA altogether. DMA users on node nid will
> > > + first try to allocate buffer from the pernuma area
> > > + which is located in node nid, if the allocation fails,
> > > + they will fallback to the global default memory area.
> >
> > What is the default behaviour if this option is not specified? Seems like
> > that should be mentioned here.
Just wanted to make sure you didn't miss this ^^
> >
> > > diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig
> > > index 847a9d1fa634..db7a37ed35eb 100644
> > > --- a/kernel/dma/Kconfig
> > > +++ b/kernel/dma/Kconfig
> > > @@ -118,6 +118,16 @@ config DMA_CMA
> > > If unsure, say "n".
> > >
> > > if DMA_CMA
> > > +
> > > +config DMA_PERNUMA_CMA
> > > + bool "Enable separate DMA Contiguous Memory Area for each NUMA
> > Node"
> >
> > I don't understand the need for this config option. If you have DMA_DMA and
> > you have NUMA, why wouldn't you want this enabled?
>
> Christoph preferred this in previous patchset in order to be able to remove
> all of the code
> in the text if users don't use pernuma CMA.
Ok, I defer to Christoph here, but maybe a "default NUMA" might work?
> > > + help
> > > + Enable this option to get pernuma CMA areas so that devices like
> > > + ARM64 SMMU can get local memory by DMA coherent APIs.
> > > +
> > > + You can set the size of pernuma CMA by specifying
> > "pernuma_cma=size"
> > > + on the kernel's command line.
> > > +
> > > comment "Default contiguous memory area size:"
> > >
> > > config CMA_SIZE_MBYTES
> > > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
> > > index cff7e60968b9..89b95f10e56d 100644
> > > --- a/kernel/dma/contiguous.c
> > > +++ b/kernel/dma/contiguous.c
> > > @@ -69,6 +69,19 @@ static int __init early_cma(char *p)
> > > }
> > > early_param("cma", early_cma);
> > >
> > > +#ifdef CONFIG_DMA_PERNUMA_CMA
> > > +
> > > +static struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES];
> > > +static phys_addr_t pernuma_size_bytes __initdata;
> > > +
> > > +static int __init early_pernuma_cma(char *p)
> > > +{
> > > + pernuma_size_bytes = memparse(p, );
> > > + return 0;
> > > +}
> > > +early_param("pernuma_cma", early_pernuma_cma);
> > > +#endif
> > > +
> > > #ifdef CONFIG_CMA_SIZE_PERCENTAGE
> > >
> > > static phys_addr_t __init __maybe_unused
> > cma_early_percent_memory(void)
> > > @@ -96,6 +109,34 @@ static inline __maybe_unused phys_addr_t
> > cma_early_percent_memory(void)
> > >
> > > #endif
> > >
> > > +#ifdef CONFIG_DMA_PERNUMA_CMA
> > > +void __init dma_pernuma_cma_reserve(void)
> > > +{
> > > + int nid;
> >
RE: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA
> -Original Message-
> From: Will Deacon [mailto:w...@kernel.org]
> Sent: Friday, August 21, 2020 8:47 PM
> To: Song Bao Hua (Barry Song)
> Cc: h...@lst.de; m.szyprow...@samsung.com; robin.mur...@arm.com;
> ganapatrao.kulka...@cavium.com; catalin.mari...@arm.com;
> iommu@lists.linux-foundation.org; Linuxarm ;
> linux-arm-ker...@lists.infradead.org; linux-ker...@vger.kernel.org;
> huangdaode ; Jonathan Cameron
> ; Nicolas Saenz Julienne
> ; Steve Capper ; Andrew
> Morton ; Mike Rapoport
> Subject: Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve
> per-numa CMA
>
> On Fri, Aug 21, 2020 at 02:26:14PM +1200, Barry Song wrote:
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt
> b/Documentation/admin-guide/kernel-parameters.txt
> > index bdc1f33fd3d1..3f33b89aeab5 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -599,6 +599,15 @@
> > altogether. For more information, see
> > include/linux/dma-contiguous.h
> >
> > + pernuma_cma=nn[MG]
> > + [ARM64,KNL]
> > + Sets the size of kernel per-numa memory area for
> > + contiguous memory allocations. A value of 0 disables
> > + per-numa CMA altogether. DMA users on node nid will
> > + first try to allocate buffer from the pernuma area
> > + which is located in node nid, if the allocation fails,
> > + they will fallback to the global default memory area.
>
> What is the default behaviour if this option is not specified? Seems like
> that should be mentioned here.
>
> > diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig
> > index 847a9d1fa634..db7a37ed35eb 100644
> > --- a/kernel/dma/Kconfig
> > +++ b/kernel/dma/Kconfig
> > @@ -118,6 +118,16 @@ config DMA_CMA
> > If unsure, say "n".
> >
> > if DMA_CMA
> > +
> > +config DMA_PERNUMA_CMA
> > + bool "Enable separate DMA Contiguous Memory Area for each NUMA
> Node"
>
> I don't understand the need for this config option. If you have DMA_DMA and
> you have NUMA, why wouldn't you want this enabled?
Christoph preferred this in previous patchset in order to be able to remove all
of the code
in the text if users don't use pernuma CMA.
>
> > + help
> > + Enable this option to get pernuma CMA areas so that devices like
> > + ARM64 SMMU can get local memory by DMA coherent APIs.
> > +
> > + You can set the size of pernuma CMA by specifying
> "pernuma_cma=size"
> > + on the kernel's command line.
> > +
> > comment "Default contiguous memory area size:"
> >
> > config CMA_SIZE_MBYTES
> > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
> > index cff7e60968b9..89b95f10e56d 100644
> > --- a/kernel/dma/contiguous.c
> > +++ b/kernel/dma/contiguous.c
> > @@ -69,6 +69,19 @@ static int __init early_cma(char *p)
> > }
> > early_param("cma", early_cma);
> >
> > +#ifdef CONFIG_DMA_PERNUMA_CMA
> > +
> > +static struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES];
> > +static phys_addr_t pernuma_size_bytes __initdata;
> > +
> > +static int __init early_pernuma_cma(char *p)
> > +{
> > + pernuma_size_bytes = memparse(p, );
> > + return 0;
> > +}
> > +early_param("pernuma_cma", early_pernuma_cma);
> > +#endif
> > +
> > #ifdef CONFIG_CMA_SIZE_PERCENTAGE
> >
> > static phys_addr_t __init __maybe_unused
> cma_early_percent_memory(void)
> > @@ -96,6 +109,34 @@ static inline __maybe_unused phys_addr_t
> cma_early_percent_memory(void)
> >
> > #endif
> >
> > +#ifdef CONFIG_DMA_PERNUMA_CMA
> > +void __init dma_pernuma_cma_reserve(void)
> > +{
> > + int nid;
> > +
> > + if (!pernuma_size_bytes)
> > + return;
>
> If this is useful (I assume it is), then I think we should have a non-zero
> default value, a bit like normal CMA does via CMA_SIZE_MBYTES.
The patchet used to have a CONFIG_PERNUMA_CMA_SIZE in kernel/dma/Kconfig, but
Christoph was not comfortable
with it:
https://lore.kernel.org/linux-iommu/20200728115231.ga...@lst.de/
Would you mind to hardcode the value in CONFIG_CMDLINE in arch/arm64/Kconfig as
Christoph mentioned:
config CMDLINE
default "pernuma_cma=16M"
If you also don't like the change in arch/arm64/Kconfig CMDLINE, I guess I have
to depend on users' setting in
cmdline just
Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA
On Fri, Aug 21, 2020 at 02:26:14PM +1200, Barry Song wrote:
> diff --git a/Documentation/admin-guide/kernel-parameters.txt
> b/Documentation/admin-guide/kernel-parameters.txt
> index bdc1f33fd3d1..3f33b89aeab5 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -599,6 +599,15 @@
> altogether. For more information, see
> include/linux/dma-contiguous.h
>
> + pernuma_cma=nn[MG]
> + [ARM64,KNL]
> + Sets the size of kernel per-numa memory area for
> + contiguous memory allocations. A value of 0 disables
> + per-numa CMA altogether. DMA users on node nid will
> + first try to allocate buffer from the pernuma area
> + which is located in node nid, if the allocation fails,
> + they will fallback to the global default memory area.
What is the default behaviour if this option is not specified? Seems like
that should be mentioned here.
> diff --git a/kernel/dma/Kconfig b/kernel/dma/Kconfig
> index 847a9d1fa634..db7a37ed35eb 100644
> --- a/kernel/dma/Kconfig
> +++ b/kernel/dma/Kconfig
> @@ -118,6 +118,16 @@ config DMA_CMA
> If unsure, say "n".
>
> if DMA_CMA
> +
> +config DMA_PERNUMA_CMA
> + bool "Enable separate DMA Contiguous Memory Area for each NUMA Node"
I don't understand the need for this config option. If you have DMA_DMA and
you have NUMA, why wouldn't you want this enabled?
> + help
> + Enable this option to get pernuma CMA areas so that devices like
> + ARM64 SMMU can get local memory by DMA coherent APIs.
> +
> + You can set the size of pernuma CMA by specifying "pernuma_cma=size"
> + on the kernel's command line.
> +
> comment "Default contiguous memory area size:"
>
> config CMA_SIZE_MBYTES
> diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
> index cff7e60968b9..89b95f10e56d 100644
> --- a/kernel/dma/contiguous.c
> +++ b/kernel/dma/contiguous.c
> @@ -69,6 +69,19 @@ static int __init early_cma(char *p)
> }
> early_param("cma", early_cma);
>
> +#ifdef CONFIG_DMA_PERNUMA_CMA
> +
> +static struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES];
> +static phys_addr_t pernuma_size_bytes __initdata;
> +
> +static int __init early_pernuma_cma(char *p)
> +{
> + pernuma_size_bytes = memparse(p, );
> + return 0;
> +}
> +early_param("pernuma_cma", early_pernuma_cma);
> +#endif
> +
> #ifdef CONFIG_CMA_SIZE_PERCENTAGE
>
> static phys_addr_t __init __maybe_unused cma_early_percent_memory(void)
> @@ -96,6 +109,34 @@ static inline __maybe_unused phys_addr_t
> cma_early_percent_memory(void)
>
> #endif
>
> +#ifdef CONFIG_DMA_PERNUMA_CMA
> +void __init dma_pernuma_cma_reserve(void)
> +{
> + int nid;
> +
> + if (!pernuma_size_bytes)
> + return;
If this is useful (I assume it is), then I think we should have a non-zero
default value, a bit like normal CMA does via CMA_SIZE_MBYTES.
> + for_each_node_state(nid, N_ONLINE) {
for_each_online_node() {
> + int ret;
> + char name[20];
20?
Ah, wait, this is copy-pasta from hugetlb_cma_reserve(). Can you factor out
the common parts at all?
Will
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
RE: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA
> -Original Message-
> From: linux-kernel-ow...@vger.kernel.org
> [mailto:linux-kernel-ow...@vger.kernel.org] On Behalf Of Randy Dunlap
> Sent: Friday, August 21, 2020 2:50 PM
> To: Song Bao Hua (Barry Song) ; h...@lst.de;
> m.szyprow...@samsung.com; robin.mur...@arm.com; w...@kernel.org;
> ganapatrao.kulka...@cavium.com; catalin.mari...@arm.com
> Cc: iommu@lists.linux-foundation.org; Linuxarm ;
> linux-arm-ker...@lists.infradead.org; linux-ker...@vger.kernel.org;
> huangdaode ; Jonathan Cameron
> ; Nicolas Saenz Julienne
> ; Steve Capper ; Andrew
> Morton ; Mike Rapoport
> Subject: Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve
> per-numa CMA
>
> On 8/20/20 7:26 PM, Barry Song wrote:
> >
> >
> > Cc: Jonathan Cameron
> > Cc: Christoph Hellwig
> > Cc: Marek Szyprowski
> > Cc: Will Deacon
> > Cc: Robin Murphy
> > Cc: Ganapatrao Kulkarni
> > Cc: Catalin Marinas
> > Cc: Nicolas Saenz Julienne
> > Cc: Steve Capper
> > Cc: Andrew Morton
> > Cc: Mike Rapoport
> > Signed-off-by: Barry Song
> > ---
> > v6: rebase on top of 5.9-rc1;
> > doc cleanup
> >
> > .../admin-guide/kernel-parameters.txt | 9 ++
> > include/linux/dma-contiguous.h| 6 ++
> > kernel/dma/Kconfig| 10 ++
> > kernel/dma/contiguous.c | 100
> --
> > 4 files changed, 115 insertions(+), 10 deletions(-)
> >
> > diff --git a/Documentation/admin-guide/kernel-parameters.txt
> b/Documentation/admin-guide/kernel-parameters.txt
> > index bdc1f33fd3d1..3f33b89aeab5 100644
> > --- a/Documentation/admin-guide/kernel-parameters.txt
> > +++ b/Documentation/admin-guide/kernel-parameters.txt
> > @@ -599,6 +599,15 @@
> > altogether. For more information, see
> > include/linux/dma-contiguous.h
> >
> > + pernuma_cma=nn[MG]
>
> memparse() allows any one of these suffixes: K, M, G, T, P, E
> and nothing in the option parsing function cares what suffix is used...
Hello Randy,
Thanks for your comments.
Actually I am following the suffix of default cma:
cma=nn[MG]@[start[MG][-end[MG]]]
[ARM,X86,KNL]
Sets the size of kernel global memory area for
contiguous memory allocations and optionally the
placement constraint by the physical address range of
memory allocations. A value of 0 disables CMA
altogether. For more information, see
include/linux/dma-contiguous.h
I suggest users should set the size in either MB or GB as they set cma.
>
> > + [ARM64,KNL]
> > + Sets the size of kernel per-numa memory area for
> > + contiguous memory allocations. A value of 0 disables
> > + per-numa CMA altogether. DMA users on node nid will
> > + first try to allocate buffer from the pernuma area
> > + which is located in node nid, if the allocation fails,
> > + they will fallback to the global default memory area.
> > +
> > cmo_free_hint= [PPC] Format: { yes | no }
> > Specify whether pages are marked as being inactive
> > when they are freed. This is used in CMO environments
>
> > diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
> > index cff7e60968b9..89b95f10e56d 100644
> > --- a/kernel/dma/contiguous.c
> > +++ b/kernel/dma/contiguous.c
> > @@ -69,6 +69,19 @@ static int __init early_cma(char *p)
> > }
> > early_param("cma", early_cma);
> >
> > +#ifdef CONFIG_DMA_PERNUMA_CMA
> > +
> > +static struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES];
> > +static phys_addr_t pernuma_size_bytes __initdata;
>
> why phys_addr_t? couldn't it just be unsigned long long?
>
Mainly because of following the programming habit in kernel/dma/contiguous.c:
I think the original code probably meant the size should not be larger than the
MAXIMUM
value of phys_addr_t:
/*
* Default global CMA area size can be defined in kernel's .config.
* This is useful mainly for distro maintainers to create a kernel
* that works correctly for most supported systems.
* The size can be set in bytes or as a percentage of the total memory
* in the system.
*
* Users, who want to set the size of global CMA area for their system
* should use cma= kernel parameter.
*/
static const phys_addr_t size_bytes __initconst =
(phys_addr_t)CMA_SIZE_MBYT
Re: [PATCH v6 1/2] dma-contiguous: provide the ability to reserve per-numa CMA
On 8/20/20 7:26 PM, Barry Song wrote:
>
>
> Cc: Jonathan Cameron
> Cc: Christoph Hellwig
> Cc: Marek Szyprowski
> Cc: Will Deacon
> Cc: Robin Murphy
> Cc: Ganapatrao Kulkarni
> Cc: Catalin Marinas
> Cc: Nicolas Saenz Julienne
> Cc: Steve Capper
> Cc: Andrew Morton
> Cc: Mike Rapoport
> Signed-off-by: Barry Song
> ---
> v6: rebase on top of 5.9-rc1;
> doc cleanup
>
> .../admin-guide/kernel-parameters.txt | 9 ++
> include/linux/dma-contiguous.h| 6 ++
> kernel/dma/Kconfig| 10 ++
> kernel/dma/contiguous.c | 100 --
> 4 files changed, 115 insertions(+), 10 deletions(-)
>
> diff --git a/Documentation/admin-guide/kernel-parameters.txt
> b/Documentation/admin-guide/kernel-parameters.txt
> index bdc1f33fd3d1..3f33b89aeab5 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -599,6 +599,15 @@
> altogether. For more information, see
> include/linux/dma-contiguous.h
>
> + pernuma_cma=nn[MG]
memparse() allows any one of these suffixes: K, M, G, T, P, E
and nothing in the option parsing function cares what suffix is used...
> + [ARM64,KNL]
> + Sets the size of kernel per-numa memory area for
> + contiguous memory allocations. A value of 0 disables
> + per-numa CMA altogether. DMA users on node nid will
> + first try to allocate buffer from the pernuma area
> + which is located in node nid, if the allocation fails,
> + they will fallback to the global default memory area.
> +
> cmo_free_hint= [PPC] Format: { yes | no }
> Specify whether pages are marked as being inactive
> when they are freed. This is used in CMO environments
> diff --git a/kernel/dma/contiguous.c b/kernel/dma/contiguous.c
> index cff7e60968b9..89b95f10e56d 100644
> --- a/kernel/dma/contiguous.c
> +++ b/kernel/dma/contiguous.c
> @@ -69,6 +69,19 @@ static int __init early_cma(char *p)
> }
> early_param("cma", early_cma);
>
> +#ifdef CONFIG_DMA_PERNUMA_CMA
> +
> +static struct cma *dma_contiguous_pernuma_area[MAX_NUMNODES];
> +static phys_addr_t pernuma_size_bytes __initdata;
why phys_addr_t? couldn't it just be unsigned long long?
OK, so cma_declare_contiguous_nid() uses phys_addr_t. Fine.
> +
> +static int __init early_pernuma_cma(char *p)
> +{
> + pernuma_size_bytes = memparse(p, );
> + return 0;
> +}
> +early_param("pernuma_cma", early_pernuma_cma);
> +#endif
> +
> #ifdef CONFIG_CMA_SIZE_PERCENTAGE
>
> static phys_addr_t __init __maybe_unused cma_early_percent_memory(void)
> @@ -96,6 +109,34 @@ static inline __maybe_unused phys_addr_t
> cma_early_percent_memory(void)
>
> #endif
>
> +#ifdef CONFIG_DMA_PERNUMA_CMA
> +void __init dma_pernuma_cma_reserve(void)
> +{
> + int nid;
> +
> + if (!pernuma_size_bytes)
> + return;
> +
> + for_each_node_state(nid, N_ONLINE) {
> + int ret;
> + char name[20];
> + struct cma **cma = _contiguous_pernuma_area[nid];
> +
> + snprintf(name, sizeof(name), "pernuma%d", nid);
> + ret = cma_declare_contiguous_nid(0, pernuma_size_bytes, 0, 0,
> + 0, false, name, cma, nid);
> + if (ret) {
> + pr_warn("%s: reservation failed: err %d, node %d",
> __func__,
> + ret, nid);
> + continue;
> + }
> +
> + pr_debug("%s: reserved %llu MiB on node %d\n", __func__,
> + (unsigned long long)pernuma_size_bytes / SZ_1M, nid);
Conversely, if you want to leave pernuma_size_bytes as phys_addr_t,
you should use %pa (or %pap) to print it.
> + }
> +}
> +#endif
--
~Randy
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
