Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Hugh Dickins
On Fri, 26 Oct 2007, Jan Engelhardt wrote: > On Oct 26 2007 13:30, Hugh Dickins wrote: > >> @@ -251,7 +251,8 @@ asmlinkage unsigned long sys_brk(unsigned long brk) > >> * not page aligned -Ram Gupta > >> */ > >>rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur; > >> - if (rlim <

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Hugh Dickins
On Fri, 26 Oct 2007, Jiri Kosina wrote: > On Fri, 26 Oct 2007, Hugh Dickins wrote: > > > > - if (rlim < RLIM_INFINITY && brk - mm->start_data > rlim) > > > + if (rlim < RLIM_INFINITY && brk - mm->start_data - > > > + (mm->start_brk - mm->end_data) > rlim) > > I find the order in

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Jan Engelhardt
On Oct 26 2007 13:30, Hugh Dickins wrote: >> @@ -251,7 +251,8 @@ asmlinkage unsigned long sys_brk(unsigned long brk) >> * not page aligned -Ram Gupta >> */ >> rlim = current->signal->rlim[RLIMIT_DATA].rlim_cur; >> -if (rlim < RLIM_INFINITY && brk - mm->start_data > rlim) >> +

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Jiri Kosina
On Fri, 26 Oct 2007, Hugh Dickins wrote: > > - if (rlim < RLIM_INFINITY && brk - mm->start_data > rlim) > > + if (rlim < RLIM_INFINITY && brk - mm->start_data - > > + (mm->start_brk - mm->end_data) > rlim) > I find the order in that test mysterious. I agree that the order

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Hugh Dickins
On Fri, 26 Oct 2007, Jiri Kosina wrote: > On Thu, 25 Oct 2007, Sami Farin wrote: > > > > > Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) > > > > > worked in a predictable way: > > > > this isn't a valid case afaics; even on "traditional x86" (before we > > > > changed the

[PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Jiri Kosina
On Thu, 25 Oct 2007, Sami Farin wrote: > > > > Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) > > > > worked in a predictable way: > > > this isn't a valid case afaics; even on "traditional x86" (before we > > > changed the address space layout, or even today if you have an > >

[PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Jiri Kosina
On Thu, 25 Oct 2007, Sami Farin wrote: Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) worked in a predictable way: this isn't a valid case afaics; even on traditional x86 (before we changed the address space layout, or even today if you have an unlimited stack

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Hugh Dickins
On Fri, 26 Oct 2007, Jiri Kosina wrote: On Thu, 25 Oct 2007, Sami Farin wrote: Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) worked in a predictable way: this isn't a valid case afaics; even on traditional x86 (before we changed the address space layout, or

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Jiri Kosina
On Fri, 26 Oct 2007, Hugh Dickins wrote: - if (rlim RLIM_INFINITY brk - mm-start_data rlim) + if (rlim RLIM_INFINITY brk - mm-start_data - + (mm-start_brk - mm-end_data) rlim) I find the order in that test mysterious. I agree that the order you proposed is

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Hugh Dickins
On Fri, 26 Oct 2007, Jiri Kosina wrote: On Fri, 26 Oct 2007, Hugh Dickins wrote: - if (rlim RLIM_INFINITY brk - mm-start_data rlim) + if (rlim RLIM_INFINITY brk - mm-start_data - + (mm-start_brk - mm-end_data) rlim) I find the order in that test mysterious. I

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Jan Engelhardt
On Oct 26 2007 13:30, Hugh Dickins wrote: @@ -251,7 +251,8 @@ asmlinkage unsigned long sys_brk(unsigned long brk) * not page aligned -Ram Gupta */ rlim = current-signal-rlim[RLIMIT_DATA].rlim_cur; -if (rlim RLIM_INFINITY brk - mm-start_data rlim) +if (rlim

Re: [PATCH] [RFC] brk randomization: compute RLIMIT_DATA properly (was Re: x86: randomize brk() and RLIMIT_DATA)

2007-10-26 Thread Hugh Dickins
On Fri, 26 Oct 2007, Jan Engelhardt wrote: On Oct 26 2007 13:30, Hugh Dickins wrote: @@ -251,7 +251,8 @@ asmlinkage unsigned long sys_brk(unsigned long brk) * not page aligned -Ram Gupta */ rlim = current-signal-rlim[RLIMIT_DATA].rlim_cur; - if (rlim RLIM_INFINITY brk -

Re: x86: randomize brk() and RLIMIT_DATA

2007-10-25 Thread Sami Farin
On Thu, Oct 25, 2007 at 16:46:26 +0200, Jiri Kosina wrote: > On Thu, 25 Oct 2007, Arjan van de Ven wrote: > > > > Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) > > > worked in a predictable way: > > this isn't a valid case afaics; even on "traditional x86" (before we > >

Re: x86: randomize brk() and RLIMIT_DATA

2007-10-25 Thread Jiri Kosina
On Thu, 25 Oct 2007, Arjan van de Ven wrote: > > Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) > > worked in a predictable way: > this isn't a valid case afaics; even on "traditional x86" (before we > changed the address space layout, or even today if you have an unlimited >

Re: x86: randomize brk() and RLIMIT_DATA

2007-10-25 Thread Arjan van de Ven
On Thu, 25 Oct 2007 16:41:24 +0300 Sami Farin <[EMAIL PROTECTED]> wrote: > Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) > worked in a predictable way: this isn't a valid case afaics; even on "traditional x86" (before we changed the address space layout, or even today if you

x86: randomize brk() and RLIMIT_DATA

2007-10-25 Thread Sami Farin
Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) worked in a predictable way: $ gcc brk.c -fPIC -pie -m64;./a.out;./a.out;./a.out sbrk=0x7f721b815000 main=0x7f721af04860 sbrk succeeded (brk=0x7f721b909240) sbrk=0x7fc3d77e2000 main=0x7fc3d66fa860 sbrk failed: Cannot allocate memory

x86: randomize brk() and RLIMIT_DATA

2007-10-25 Thread Sami Farin
Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) worked in a predictable way: $ gcc brk.c -fPIC -pie -m64;./a.out;./a.out;./a.out sbrk=0x7f721b815000 main=0x7f721af04860 sbrk succeeded (brk=0x7f721b909240) sbrk=0x7fc3d77e2000 main=0x7fc3d66fa860 sbrk failed: Cannot allocate memory

Re: x86: randomize brk() and RLIMIT_DATA

2007-10-25 Thread Arjan van de Ven
On Thu, 25 Oct 2007 16:41:24 +0300 Sami Farin [EMAIL PROTECTED] wrote: Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) worked in a predictable way: this isn't a valid case afaics; even on traditional x86 (before we changed the address space layout, or even today if you have an

Re: x86: randomize brk() and RLIMIT_DATA

2007-10-25 Thread Jiri Kosina
On Thu, 25 Oct 2007, Arjan van de Ven wrote: Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) worked in a predictable way: this isn't a valid case afaics; even on traditional x86 (before we changed the address space layout, or even today if you have an unlimited stack

Re: x86: randomize brk() and RLIMIT_DATA

2007-10-25 Thread Sami Farin
On Thu, Oct 25, 2007 at 16:46:26 +0200, Jiri Kosina wrote: On Thu, 25 Oct 2007, Arjan van de Ven wrote: Would be neat if randomized brk and setrlimit(RLIMIT_DATA, ...) worked in a predictable way: this isn't a valid case afaics; even on traditional x86 (before we changed the address