tree 173980b12459aa55586d3f4f56630abf5af1ffcc
parent b68e9f857271189bd7a59b74c99890de9195b0e1
author Simon Derr <[EMAIL PROTECTED]> Fri, 05 Aug 2005 09:52:03 -0700
committer Linus Torvalds <[EMAIL PROTECTED]> Fri, 05 Aug 2005 11:43:14 -0700

[PATCH] __vm_enough_memory() signedness fix

We have found what seems to be a small bug in __vm_enough_memory() when
sysctl_overcommit_memory is set to OVERCOMMIT_NEVER.

When this bug occurs the systems fails to boot, with /sbin/init whining
about fork() returning ENOMEM.

We hunted down the problem to this:

The deferred update mecanism used in vm_acct_memory(), on a SMP system,
allows the vm_committed_space counter to have a negative value.

This should not be a problem since this counter is known to be inaccurate.

But in __vm_enough_memory() this counter is compared to the `allowed'
variable, which is an unsigned long.  This comparison is broken since it
will consider the negative values of vm_committed_space to be huge positive
values, resulting in a memory allocation failure.

Signed-off-by: <[EMAIL PROTECTED]>
Signed-off-by: <[EMAIL PROTECTED]>
Signed-off-by: Andrew Morton <[EMAIL PROTECTED]>
Signed-off-by: Linus Torvalds <[EMAIL PROTECTED]>

 mm/mmap.c  |    6 +++++-
 mm/nommu.c |    6 +++++-
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/mm/mmap.c b/mm/mmap.c
--- a/mm/mmap.c
+++ b/mm/mmap.c
@@ -143,7 +143,11 @@ int __vm_enough_memory(long pages, int c
           leave 3% of the size of this process for other processes */
        allowed -= current->mm->total_vm / 32;
 
-       if (atomic_read(&vm_committed_space) < allowed)
+       /*
+        * cast `allowed' as a signed long because vm_committed_space
+        * sometimes has a negative value
+        */
+       if (atomic_read(&vm_committed_space) < (long)allowed)
                return 0;
 
        vm_unacct_memory(pages);
diff --git a/mm/nommu.c b/mm/nommu.c
--- a/mm/nommu.c
+++ b/mm/nommu.c
@@ -1167,7 +1167,11 @@ int __vm_enough_memory(long pages, int c
           leave 3% of the size of this process for other processes */
        allowed -= current->mm->total_vm / 32;
 
-       if (atomic_read(&vm_committed_space) < allowed)
+       /*
+        * cast `allowed' as a signed long because vm_committed_space
+        * sometimes has a negative value
+        */
+       if (atomic_read(&vm_committed_space) < (long)allowed)
                return 0;
 
        vm_unacct_memory(pages);
-
To unsubscribe from this list: send the line "unsubscribe git-commits-head" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Reply via email to