[PATCH 1/3] x86/mmap: properly account for stack randomization in mmap_base

[riel]

From: Rik van Riel 

When RLIMIT_STACK is, for example, 256MB, the current code results in
a gap between the top of the task and mmap_base of 256MB, failing to
take into account the amount by which the stack address was randomized.
In other words, the stack gets less than RLIMIT_STACK space.

Ensure that the gap between the stack and mmap_base always takes stack
randomization and the stack guard gap into account.

>From Daniel Micay's linux-hardened tree.

Reported-by: Florian Weimer 
Signed-off-by: Daniel Micay 
Signed-off-by: Rik van Riel 
---
 arch/x86/mm/mmap.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c
index 19ad095b41df..7c35dd73dbd4 100644
--- a/arch/x86/mm/mmap.c
+++ b/arch/x86/mm/mmap.c
@@ -95,13 +95,18 @@ unsigned long arch_mmap_rnd(void)
 static unsigned long mmap_base(unsigned long rnd, unsigned long task_size)
 {
unsigned long gap = rlimit(RLIMIT_STACK);
+   unsigned long pad = stack_maxrandom_size(task_size) + stack_guard_gap;
unsigned long gap_min, gap_max;
 
+   /* Values close to RLIM_INFINITY can overflow. */
+   if (gap + pad > gap)
+   gap += pad;
+
/*
 * Top of mmap area (just below the process stack).
 * Leave an at least ~128 MB hole with possible stack randomization.
 */
-   gap_min = SIZE_128M + stack_maxrandom_size(task_size);
+   gap_min = SIZE_128M;
gap_max = (task_size / 6) * 5;
 
if (gap < gap_min)
-- 
2.9.4

[PATCH 1/3] x86/mmap: properly account for stack randomization in mmap_base

[riel]

From: Rik van Riel 

When RLIMIT_STACK is, for example, 256MB, the current code results in
a gap between the top of the task and mmap_base of 256MB, failing to
take into account the amount by which the stack address was randomized.
In other words, the stack gets less than RLIMIT_STACK space.

Ensure that the gap between the stack and mmap_base always takes stack
randomization and the stack guard gap into account.

>From Daniel Micay's linux-hardened tree.

Reported-by: Florian Weimer 
Signed-off-by: Daniel Micay 
Signed-off-by: Rik van Riel 
---
 arch/x86/mm/mmap.c | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c
index 19ad095b41df..7c35dd73dbd4 100644
--- a/arch/x86/mm/mmap.c
+++ b/arch/x86/mm/mmap.c
@@ -95,13 +95,18 @@ unsigned long arch_mmap_rnd(void)
 static unsigned long mmap_base(unsigned long rnd, unsigned long task_size)
 {
unsigned long gap = rlimit(RLIMIT_STACK);
+   unsigned long pad = stack_maxrandom_size(task_size) + stack_guard_gap;
unsigned long gap_min, gap_max;
 
+   /* Values close to RLIM_INFINITY can overflow. */
+   if (gap + pad > gap)
+   gap += pad;
+
/*
 * Top of mmap area (just below the process stack).
 * Leave an at least ~128 MB hole with possible stack randomization.
 */
-   gap_min = SIZE_128M + stack_maxrandom_size(task_size);
+   gap_min = SIZE_128M;
gap_max = (task_size / 6) * 5;
 
if (gap < gap_min)
-- 
2.9.4