Re: [RFC PATCH 1/1] mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()

[Tejun Heo]

Hello,

On Fri, Sep 30, 2016 at 01:38:35AM +0800, zijun_hu wrote:
> 1) the simpler way don't work because it maybe free many memory block twice

Right, the punched holes.  Forgot about them.  Yeah, that's why the
later failure just leaks memory.

> 2) as we seen, pcpu_setup_first_chunk() doesn't cause a failure, it  return 0
>always or panic by BUG_ON(), even if it fails, we can conclude the 
> allocated
>memory based on information recorded by it, such as pcpu_base_addr and 
> many of
>static variable, we can complete the free operations; but we can't if we
>fail in the case pointed by this patch

So, being strictly correct doesn't matter that much here.  These
things failing indicates that something is very wrong with either the
code or configuration and we might as well trigger BUG.  That said,
yeah, it's nicer to recover without leaking anything.

> 3) my test way is simple, i force "if (max_distance > VMALLOC_TOTAL * 3 / 4)"
>to if (1) and print which memory i allocate before the jumping, then print 
> which memory
>i free after the jumping and before returning, then check whether i free 
> the memory i 
>allocate in this function, the result is okay

Can you please include what has been discussed into the patch
description?

Thanks.

-- 
tejun

Re: [RFC PATCH 1/1] mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()

[Tejun Heo]

Hello,

On Fri, Sep 30, 2016 at 01:38:35AM +0800, zijun_hu wrote:
> 1) the simpler way don't work because it maybe free many memory block twice

Right, the punched holes.  Forgot about them.  Yeah, that's why the
later failure just leaks memory.

> 2) as we seen, pcpu_setup_first_chunk() doesn't cause a failure, it  return 0
>always or panic by BUG_ON(), even if it fails, we can conclude the 
> allocated
>memory based on information recorded by it, such as pcpu_base_addr and 
> many of
>static variable, we can complete the free operations; but we can't if we
>fail in the case pointed by this patch

So, being strictly correct doesn't matter that much here.  These
things failing indicates that something is very wrong with either the
code or configuration and we might as well trigger BUG.  That said,
yeah, it's nicer to recover without leaking anything.

> 3) my test way is simple, i force "if (max_distance > VMALLOC_TOTAL * 3 / 4)"
>to if (1) and print which memory i allocate before the jumping, then print 
> which memory
>i free after the jumping and before returning, then check whether i free 
> the memory i 
>allocate in this function, the result is okay

Can you please include what has been discussed into the patch
description?

Thanks.

-- 
tejun

Re: [RFC PATCH 1/1] mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()

[zijun_hu]

On 2016/9/30 0:44, Tejun Heo wrote:
> Hello,
> 
> On Fri, Sep 30, 2016 at 12:03:20AM +0800, zijun_hu wrote:
>> From: zijun_hu 
>>
>> it will cause memory leakage for pcpu_embed_first_chunk() to go to
>> label @out_free if the chunk spans over 3/4 VMALLOC area. all memory
>> are allocated and recorded into array @areas for each CPU group, but
>> the memory allocated aren't be freed before returning after going to
>> label @out_free
>>
>> in order to fix this bug, we check chunk spanned area immediately
>> after completing memory allocation for all CPU group, we go to label
>> @out_free_areas other than @out_free to free all memory allocated if
>> the checking is failed.
>>
>> Signed-off-by: zijun_hu 
> ...
>> @@ -2000,6 +2001,21 @@ int __init pcpu_embed_first_chunk(size_t 
>> reserved_size, size_t dyn_size,
>>  areas[group] = ptr;
>>  
>>  base = min(ptr, base);
>> +if (ptr > areas[j])
>> +j = group;
>> +}
>> +max_distance = areas[j] - base;
>> +max_distance += ai->unit_size * ai->groups[j].nr_units;
>> +
>> +/* warn if maximum distance is further than 75% of vmalloc space */
>> +if (max_distance > VMALLOC_TOTAL * 3 / 4) {
>> +pr_warn("max_distance=0x%lx too large for vmalloc space 
>> 0x%lx\n",
>> +max_distance, VMALLOC_TOTAL);
>> +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
>> +/* and fail if we have fallback */
>> +rc = -EINVAL;
>> +goto out_free_areas;
>> +#endif
> 
> Isn't it way simpler to make the error path jump to out_free_areas?
> There's another similar case after pcpu_setup_first_chunk() failure
> too.  Also, can you please explain how you tested the changes?
> 
> Thanks.
> 
1) the simpler way don't work because it maybe free many memory block twice

let us take a CPU group as a example, after we allocate All memory
needed by a CPU group, we maybe free a unit memory block which
don't map to a available CPU, we maybe free a part of unit memory which 
we don't used too, you can refer to following code segments for detailed
info.
for (group = 0; group < ai->nr_groups; group++) {
struct pcpu_group_info *gi = >groups[group];
void *ptr = areas[group];

for (i = 0; i < gi->nr_units; i++, ptr += ai->unit_size) {
if (gi->cpu_map[i] == NR_CPUS) {
/* unused unit, free whole */
free_fn(ptr, ai->unit_size);
continue;
}
/* copy and return the unused part */
memcpy(ptr, __per_cpu_load, ai->static_size);
free_fn(ptr + size_sum, ai->unit_size - size_sum);
}
}

2) as we seen, pcpu_setup_first_chunk() doesn't cause a failure, it  return 0
   always or panic by BUG_ON(), even if it fails, we can conclude the allocated
   memory based on information recorded by it, such as pcpu_base_addr and many 
of
   static variable, we can complete the free operations; but we can't if we
   fail in the case pointed by this patch

3) my test way is simple, i force "if (max_distance > VMALLOC_TOTAL * 3 / 4)"
   to if (1) and print which memory i allocate before the jumping, then print 
which memory
   i free after the jumping and before returning, then check whether i free the 
memory i 
   allocate in this function, the result is okay

Re: [RFC PATCH 1/1] mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()

[zijun_hu]

On 2016/9/30 0:44, Tejun Heo wrote:
> Hello,
> 
> On Fri, Sep 30, 2016 at 12:03:20AM +0800, zijun_hu wrote:
>> From: zijun_hu 
>>
>> it will cause memory leakage for pcpu_embed_first_chunk() to go to
>> label @out_free if the chunk spans over 3/4 VMALLOC area. all memory
>> are allocated and recorded into array @areas for each CPU group, but
>> the memory allocated aren't be freed before returning after going to
>> label @out_free
>>
>> in order to fix this bug, we check chunk spanned area immediately
>> after completing memory allocation for all CPU group, we go to label
>> @out_free_areas other than @out_free to free all memory allocated if
>> the checking is failed.
>>
>> Signed-off-by: zijun_hu 
> ...
>> @@ -2000,6 +2001,21 @@ int __init pcpu_embed_first_chunk(size_t 
>> reserved_size, size_t dyn_size,
>>  areas[group] = ptr;
>>  
>>  base = min(ptr, base);
>> +if (ptr > areas[j])
>> +j = group;
>> +}
>> +max_distance = areas[j] - base;
>> +max_distance += ai->unit_size * ai->groups[j].nr_units;
>> +
>> +/* warn if maximum distance is further than 75% of vmalloc space */
>> +if (max_distance > VMALLOC_TOTAL * 3 / 4) {
>> +pr_warn("max_distance=0x%lx too large for vmalloc space 
>> 0x%lx\n",
>> +max_distance, VMALLOC_TOTAL);
>> +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
>> +/* and fail if we have fallback */
>> +rc = -EINVAL;
>> +goto out_free_areas;
>> +#endif
> 
> Isn't it way simpler to make the error path jump to out_free_areas?
> There's another similar case after pcpu_setup_first_chunk() failure
> too.  Also, can you please explain how you tested the changes?
> 
> Thanks.
> 
1) the simpler way don't work because it maybe free many memory block twice

let us take a CPU group as a example, after we allocate All memory
needed by a CPU group, we maybe free a unit memory block which
don't map to a available CPU, we maybe free a part of unit memory which 
we don't used too, you can refer to following code segments for detailed
info.
for (group = 0; group < ai->nr_groups; group++) {
struct pcpu_group_info *gi = >groups[group];
void *ptr = areas[group];

for (i = 0; i < gi->nr_units; i++, ptr += ai->unit_size) {
if (gi->cpu_map[i] == NR_CPUS) {
/* unused unit, free whole */
free_fn(ptr, ai->unit_size);
continue;
}
/* copy and return the unused part */
memcpy(ptr, __per_cpu_load, ai->static_size);
free_fn(ptr + size_sum, ai->unit_size - size_sum);
}
}

2) as we seen, pcpu_setup_first_chunk() doesn't cause a failure, it  return 0
   always or panic by BUG_ON(), even if it fails, we can conclude the allocated
   memory based on information recorded by it, such as pcpu_base_addr and many 
of
   static variable, we can complete the free operations; but we can't if we
   fail in the case pointed by this patch

3) my test way is simple, i force "if (max_distance > VMALLOC_TOTAL * 3 / 4)"
   to if (1) and print which memory i allocate before the jumping, then print 
which memory
   i free after the jumping and before returning, then check whether i free the 
memory i 
   allocate in this function, the result is okay

Re: [RFC PATCH 1/1] mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()

[Tejun Heo]

Hello,

On Fri, Sep 30, 2016 at 12:03:20AM +0800, zijun_hu wrote:
> From: zijun_hu 
> 
> it will cause memory leakage for pcpu_embed_first_chunk() to go to
> label @out_free if the chunk spans over 3/4 VMALLOC area. all memory
> are allocated and recorded into array @areas for each CPU group, but
> the memory allocated aren't be freed before returning after going to
> label @out_free
> 
> in order to fix this bug, we check chunk spanned area immediately
> after completing memory allocation for all CPU group, we go to label
> @out_free_areas other than @out_free to free all memory allocated if
> the checking is failed.
> 
> Signed-off-by: zijun_hu 
...
> @@ -2000,6 +2001,21 @@ int __init pcpu_embed_first_chunk(size_t 
> reserved_size, size_t dyn_size,
>   areas[group] = ptr;
>  
>   base = min(ptr, base);
> + if (ptr > areas[j])
> + j = group;
> + }
> + max_distance = areas[j] - base;
> + max_distance += ai->unit_size * ai->groups[j].nr_units;
> +
> + /* warn if maximum distance is further than 75% of vmalloc space */
> + if (max_distance > VMALLOC_TOTAL * 3 / 4) {
> + pr_warn("max_distance=0x%lx too large for vmalloc space 
> 0x%lx\n",
> + max_distance, VMALLOC_TOTAL);
> +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
> + /* and fail if we have fallback */
> + rc = -EINVAL;
> + goto out_free_areas;
> +#endif

Isn't it way simpler to make the error path jump to out_free_areas?
There's another similar case after pcpu_setup_first_chunk() failure
too.  Also, can you please explain how you tested the changes?

Thanks.

-- 
tejun

Re: [RFC PATCH 1/1] mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()

[Tejun Heo]

Hello,

On Fri, Sep 30, 2016 at 12:03:20AM +0800, zijun_hu wrote:
> From: zijun_hu 
> 
> it will cause memory leakage for pcpu_embed_first_chunk() to go to
> label @out_free if the chunk spans over 3/4 VMALLOC area. all memory
> are allocated and recorded into array @areas for each CPU group, but
> the memory allocated aren't be freed before returning after going to
> label @out_free
> 
> in order to fix this bug, we check chunk spanned area immediately
> after completing memory allocation for all CPU group, we go to label
> @out_free_areas other than @out_free to free all memory allocated if
> the checking is failed.
> 
> Signed-off-by: zijun_hu 
...
> @@ -2000,6 +2001,21 @@ int __init pcpu_embed_first_chunk(size_t 
> reserved_size, size_t dyn_size,
>   areas[group] = ptr;
>  
>   base = min(ptr, base);
> + if (ptr > areas[j])
> + j = group;
> + }
> + max_distance = areas[j] - base;
> + max_distance += ai->unit_size * ai->groups[j].nr_units;
> +
> + /* warn if maximum distance is further than 75% of vmalloc space */
> + if (max_distance > VMALLOC_TOTAL * 3 / 4) {
> + pr_warn("max_distance=0x%lx too large for vmalloc space 
> 0x%lx\n",
> + max_distance, VMALLOC_TOTAL);
> +#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
> + /* and fail if we have fallback */
> + rc = -EINVAL;
> + goto out_free_areas;
> +#endif

Isn't it way simpler to make the error path jump to out_free_areas?
There's another similar case after pcpu_setup_first_chunk() failure
too.  Also, can you please explain how you tested the changes?

Thanks.

-- 
tejun

[RFC PATCH 1/1] mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()

[zijun_hu]

From: zijun_hu 

it will cause memory leakage for pcpu_embed_first_chunk() to go to
label @out_free if the chunk spans over 3/4 VMALLOC area. all memory
are allocated and recorded into array @areas for each CPU group, but
the memory allocated aren't be freed before returning after going to
label @out_free

in order to fix this bug, we check chunk spanned area immediately
after completing memory allocation for all CPU group, we go to label
@out_free_areas other than @out_free to free all memory allocated if
the checking is failed.

Signed-off-by: zijun_hu 
---
 Hi Andrew,
  i am sorry to forget to prefix title with "PATCH" keyword in previous
  mail, so i resend it with correction
  this patch is based on mmotm/linux-next branch so can be
  applied directly

 mm/percpu.c | 36 ++--
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/mm/percpu.c b/mm/percpu.c
index 41d9d0b35801..7a5dae185ce1 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -1963,7 +1963,7 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, 
size_t dyn_size,
struct pcpu_alloc_info *ai;
size_t size_sum, areas_size;
unsigned long max_distance;
-   int group, i, rc;
+   int group, i, j, rc;
 
ai = pcpu_build_alloc_info(reserved_size, dyn_size, atom_size,
   cpu_distance_fn);
@@ -1979,7 +1979,8 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, 
size_t dyn_size,
goto out_free;
}
 
-   /* allocate, copy and determine base address */
+   /* allocate, copy and determine base address & max_distance */
+   j = 0;
for (group = 0; group < ai->nr_groups; group++) {
struct pcpu_group_info *gi = >groups[group];
unsigned int cpu = NR_CPUS;
@@ -2000,6 +2001,21 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, 
size_t dyn_size,
areas[group] = ptr;
 
base = min(ptr, base);
+   if (ptr > areas[j])
+   j = group;
+   }
+   max_distance = areas[j] - base;
+   max_distance += ai->unit_size * ai->groups[j].nr_units;
+
+   /* warn if maximum distance is further than 75% of vmalloc space */
+   if (max_distance > VMALLOC_TOTAL * 3 / 4) {
+   pr_warn("max_distance=0x%lx too large for vmalloc space 
0x%lx\n",
+   max_distance, VMALLOC_TOTAL);
+#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
+   /* and fail if we have fallback */
+   rc = -EINVAL;
+   goto out_free_areas;
+#endif
}
 
/*
@@ -2024,24 +2040,8 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, 
size_t dyn_size,
}
 
/* base address is now known, determine group base offsets */
-   i = 0;
for (group = 0; group < ai->nr_groups; group++) {
ai->groups[group].base_offset = areas[group] - base;
-   if (areas[group] > areas[i])
-   i = group;
-   }
-   max_distance = ai->groups[i].base_offset +
-   (unsigned long)ai->unit_size * ai->groups[i].nr_units;
-
-   /* warn if maximum distance is further than 75% of vmalloc space */
-   if (max_distance > VMALLOC_TOTAL * 3 / 4) {
-   pr_warn("max_distance=0x%lx too large for vmalloc space 
0x%lx\n",
-   max_distance, VMALLOC_TOTAL);
-#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
-   /* and fail if we have fallback */
-   rc = -EINVAL;
-   goto out_free;
-#endif
}
 
pr_info("Embedded %zu pages/cpu @%p s%zu r%zu d%zu u%zu\n",
-- 
1.9.1

[RFC PATCH 1/1] mm/percpu.c: fix potential memory leakage for pcpu_embed_first_chunk()

[zijun_hu]

From: zijun_hu 

it will cause memory leakage for pcpu_embed_first_chunk() to go to
label @out_free if the chunk spans over 3/4 VMALLOC area. all memory
are allocated and recorded into array @areas for each CPU group, but
the memory allocated aren't be freed before returning after going to
label @out_free

in order to fix this bug, we check chunk spanned area immediately
after completing memory allocation for all CPU group, we go to label
@out_free_areas other than @out_free to free all memory allocated if
the checking is failed.

Signed-off-by: zijun_hu 
---
 Hi Andrew,
  i am sorry to forget to prefix title with "PATCH" keyword in previous
  mail, so i resend it with correction
  this patch is based on mmotm/linux-next branch so can be
  applied directly

 mm/percpu.c | 36 ++--
 1 file changed, 18 insertions(+), 18 deletions(-)

diff --git a/mm/percpu.c b/mm/percpu.c
index 41d9d0b35801..7a5dae185ce1 100644
--- a/mm/percpu.c
+++ b/mm/percpu.c
@@ -1963,7 +1963,7 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, 
size_t dyn_size,
struct pcpu_alloc_info *ai;
size_t size_sum, areas_size;
unsigned long max_distance;
-   int group, i, rc;
+   int group, i, j, rc;
 
ai = pcpu_build_alloc_info(reserved_size, dyn_size, atom_size,
   cpu_distance_fn);
@@ -1979,7 +1979,8 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, 
size_t dyn_size,
goto out_free;
}
 
-   /* allocate, copy and determine base address */
+   /* allocate, copy and determine base address & max_distance */
+   j = 0;
for (group = 0; group < ai->nr_groups; group++) {
struct pcpu_group_info *gi = >groups[group];
unsigned int cpu = NR_CPUS;
@@ -2000,6 +2001,21 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, 
size_t dyn_size,
areas[group] = ptr;
 
base = min(ptr, base);
+   if (ptr > areas[j])
+   j = group;
+   }
+   max_distance = areas[j] - base;
+   max_distance += ai->unit_size * ai->groups[j].nr_units;
+
+   /* warn if maximum distance is further than 75% of vmalloc space */
+   if (max_distance > VMALLOC_TOTAL * 3 / 4) {
+   pr_warn("max_distance=0x%lx too large for vmalloc space 
0x%lx\n",
+   max_distance, VMALLOC_TOTAL);
+#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
+   /* and fail if we have fallback */
+   rc = -EINVAL;
+   goto out_free_areas;
+#endif
}
 
/*
@@ -2024,24 +2040,8 @@ int __init pcpu_embed_first_chunk(size_t reserved_size, 
size_t dyn_size,
}
 
/* base address is now known, determine group base offsets */
-   i = 0;
for (group = 0; group < ai->nr_groups; group++) {
ai->groups[group].base_offset = areas[group] - base;
-   if (areas[group] > areas[i])
-   i = group;
-   }
-   max_distance = ai->groups[i].base_offset +
-   (unsigned long)ai->unit_size * ai->groups[i].nr_units;
-
-   /* warn if maximum distance is further than 75% of vmalloc space */
-   if (max_distance > VMALLOC_TOTAL * 3 / 4) {
-   pr_warn("max_distance=0x%lx too large for vmalloc space 
0x%lx\n",
-   max_distance, VMALLOC_TOTAL);
-#ifdef CONFIG_NEED_PER_CPU_PAGE_FIRST_CHUNK
-   /* and fail if we have fallback */
-   rc = -EINVAL;
-   goto out_free;
-#endif
}
 
pr_info("Embedded %zu pages/cpu @%p s%zu r%zu d%zu u%zu\n",
-- 
1.9.1