Re: [PATCH 2.6] fix mprotect() with len=(size_t)(-1) to return -ENOMEM
Hi Arjan, You wrote: > shouldn't we just fix the alignment code instead that the overflow case > doesn't align to 0??? > that sounds really odd. How? You have to align and you are out of bits for representing the next number. What is the next number you can round to? "null" right! Just remember that integer math with limited bits is always ring math ;-) I love to abuse this for buffers and save an if. Regards Ingo Oeser - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 2.6] fix mprotect() with len=(size_t)(-1) to return -ENOMEM
On Mon, 2005-03-14 at 17:55 +0800, Gordon Jin wrote: > This patch fixes a corner case in sys_mprotect(): > > Case: len is so large that will overflow to 0 after page alignment. shouldn't we just fix the alignment code instead that the overflow case doesn't align to 0??? that sounds really odd. - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 2.6] fix mprotect() with len=(size_t)(-1) to return -ENOMEM
This patch fixes a corner case in sys_mprotect(): Case: len is so large that will overflow to 0 after page alignment. E.g. len=(size_t)(-1), i.e. 0xff...ff. Expected result: POSIX spec says it should return -ENOMEM. Current result: len is aligned to 0, then treated the same as len=0 and return success. --- linux-2.6.11.3/mm/mprotect.c.orig 2005-03-14 13:40:28.0 -0800 +++ linux-2.6.11.3/mm/mprotect.c2005-03-14 13:42:41.0 -0800 @@ -232,14 +232,14 @@ sys_mprotect(unsigned long start, size_t if (start & ~PAGE_MASK) return -EINVAL; + if (!len) + return 0; len = PAGE_ALIGN(len); end = start + len; - if (end < start) + if (end <= start) return -ENOMEM; if (prot & ~(PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM)) return -EINVAL; - if (end == start) - return 0; /* * Does the application expect PROT_READ to imply PROT_EXEC: */ - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
[PATCH 2.6] fix mprotect() with len=(size_t)(-1) to return -ENOMEM
This patch fixes a corner case in sys_mprotect(): Case: len is so large that will overflow to 0 after page alignment. E.g. len=(size_t)(-1), i.e. 0xff...ff. Expected result: POSIX spec says it should return -ENOMEM. Current result: len is aligned to 0, then treated the same as len=0 and return success. --- linux-2.6.11.3/mm/mprotect.c.orig 2005-03-14 13:40:28.0 -0800 +++ linux-2.6.11.3/mm/mprotect.c2005-03-14 13:42:41.0 -0800 @@ -232,14 +232,14 @@ sys_mprotect(unsigned long start, size_t if (start ~PAGE_MASK) return -EINVAL; + if (!len) + return 0; len = PAGE_ALIGN(len); end = start + len; - if (end start) + if (end = start) return -ENOMEM; if (prot ~(PROT_READ | PROT_WRITE | PROT_EXEC | PROT_SEM)) return -EINVAL; - if (end == start) - return 0; /* * Does the application expect PROT_READ to imply PROT_EXEC: */ - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 2.6] fix mprotect() with len=(size_t)(-1) to return -ENOMEM
On Mon, 2005-03-14 at 17:55 +0800, Gordon Jin wrote: This patch fixes a corner case in sys_mprotect(): Case: len is so large that will overflow to 0 after page alignment. shouldn't we just fix the alignment code instead that the overflow case doesn't align to 0??? that sounds really odd. - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: [PATCH 2.6] fix mprotect() with len=(size_t)(-1) to return -ENOMEM
Hi Arjan, You wrote: shouldn't we just fix the alignment code instead that the overflow case doesn't align to 0??? that sounds really odd. How? You have to align and you are out of bits for representing the next number. What is the next number you can round to? null right! Just remember that integer math with limited bits is always ring math ;-) I love to abuse this for buffers and save an if. Regards Ingo Oeser - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
