Hello Theo,
Apparently the new malloc(3) implementation doesn't stop me from writing past
the end of buffer as long as I am inside the last page.
(Please forgive me beforehand if I am missing something too obvious)
consider the following program:
----
// We just want to see how far after end of allocated buffer we can go.
// Allocate a buffer, then try to read past it, see how far we can go before
// System notices.
// myhandler should tell us about this. If you don't trust it just disable and
// examine the core file created.
//
#include <stdio.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <signal.h>
#define PAGE_SIZE 4096
char *s;
int i;
int size;
void myhandler(int sig)
{
printf("Caught Signal %d\n", sig);
printf("s=0x%x, size=%d, i=%d\n", s, size, i);
perror("last err condition");
exit(0);
}
main(int ac, char *av[])
{
if (ac != 2) {
printf("%s <size>\n", av[0]);
exit(1);
}
size = atoi(av[1]);
if (size < 4096) {
printf ("size should be larger than 4K.\n");
exit(1);
}
signal(SIGSEGV, myhandler);
s = (char *)malloc(size);
for (i = 0; i < 2 * size - 1; i++)
s[i] = (char )i;
free(s);
}
----
and here is the execution (a very recent install as you can see):
----
#dmesg | head -6
OpenBSD 3.8-beta (GENERIC.MP) #277: Mon Aug 22 23:04:26 MDT 2005
[EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/GENERIC.MP
cpu0: Intel Pentium III ("GenuineIntel" 686-class) 869 MHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,ME
real mem = 804823040 (785960K)
avail mem = 727044096 (710004K)
# gcc malloc-test.c
# ./a.out 4096
Caught Signal 11
s=0x8a19d000, size=4096, i=4096
last err condition: Undefined error: 0
# ./a.out 4097
Caught Signal 11
s=0x7f0eb000, size=4097, i=8192
last err condition: Undefined error: 0
#
Is this the way it is supposed to be?
cheers,
Masoud Sharbiani
On Mon, Aug 22, 2005 at 05:33:40PM -0600, Theo de Raadt wrote:
> We are heading towards making the real 3.8 release soonish. I would
> like to ask the community to do lots of testing over the next week if
> they can.
>
> This release will bring a lot of new ideas from us. One of them in
> particular is somewhat risky. I think it is time to talk about that
> one, and let people know what is ahead on our road.
>
> Traditionally, Unix malloc(3) has always just "extended the brk",
> which means extending the traditional Unix process data segment to
> allocate more memory. malloc(3) would simply extend the data segment,
> and then calve off little pieces to requesting callers as needed. It
> also remembered which pieces were which, so that free(3) could do it's
> job.
>
> The way this was always done in Unix has had a number of consequences,
> some of which we wanted to get rid of. In particular, malloc & free
> have not been able to provide strong protection against overflows or
> other corruption.
>
> Our malloc implementation is a lot more resistant (than Linux) to
> "heap overflows in the malloc arena", but we wanted to improve things
> even more.
>
> Starting a few months ago, the following changes were made:
>
> - We made the mmap(2) system call return random memory addresses. As well
> the kernel ensures that two objects are not mapped next to each other;
> in effect, this creates unallocated memory which we call a "guard page".
>
> - We have changed malloc(3) to use mmap(2) instead of extending the data
> segment via brk()
>
> - We also changed free(3) to return memory to the kernel, un-allocating
> them out of the process.
>
> - As before, objects smaller than a page are allocated within shared
> pages that malloc(3) maintains. But their allocation is now somewhat
> randomized as well.
>
> - A number of other similar changes which are too dangerous for normal
> software or cause too much of a slowdown are available as malloc options
> as described in the manual page. These are very powerful for debugging
> buggy applications.
>
> Other results:
>
> - When you free an object that is >= 1 page in size, it is actually
> returned to the system. Attempting to read or write to it after
> you free is no longer acceptable. That memory is unmapped. You get
> a SIGSEGV.
>
> - For a decade and a bit, we have been fixing software for buffer overflows.
> Now we are finding a lot of software that reads before the start of the
> buffer, or reads too far off the end of the buffer. You get a SIGSEGV.
>
> To some of you, this will sound like what the Electric Fence toolkit
> used to be for. But these features are enabled by default. Electric
> Fence was also very slow. It took nearly 3 years to write these
> OpenBSD changes since performance was a serious consideration. (Early
> versions caused a nearly 50% slowdown).
>
> Our changes have tremendous benefits, but until some bugs in external
> packages are found and fixed, there are some risks as well. Some
> software making incorrect assumptions will be running into these new
> security technologies.
>
> I discussed this in talks I have given before: I said that we were
> afraid to go ahead with guard pages, because a lot of software is just
> written to such low standards. Applications over-read memory all the
> time, go 1 byte too far, read 1 byte too early, access memory after free,
> etc etc etc.
>
> Oh well -- we've decided that we will try to ship with this protection
> mechanism in any case, and try to solve the problems as we run into
> them.
>
> Two examples:
>
> Over the last two months, some OpenBSD users noticed that the X server
> was crashing occasionally. Two bugs have been diagnosed and fixed by
> us. One was a use-after-free bug in the X shared library linker. The
> other was a buffer-over-read bug deep down in the very lowest level
> fb* pixmap compositing routines. The latter bug in particular was
> very difficult to diagnose and fix, and is about 10 years old. We
> have found other bugs like this in other external software, and even a
> few in the base OpenBSD tree (though those were found a while back,
> even as we started experimenting with the new malloc code).
>
> I would bet money that the X fb* bug has crashed Linux (and other) X
> servers before. It is just that it was very rare, and noone ever
> chased it. The new malloc we have just makes code get lucky less
> often, which lets us get to the source of a bug easier. As a
> programmer, I appreciate anything which makes bugs easier to
> reproduce.
>
> We expect that our malloc will find more bugs in software, and this
> might hurt our user community in the short term. We know that what
> this new malloc is doing is perfectly legal, but that realistically
> some open source software is of such low quality that it is just not
> ready for these things to happen.
>
> We ask our users to help us uncover and fix more of these bugs in
> applications. Some will even be exploitable. Instead of saying that
> OpenBSD is busted in this regard, please realize that the software
> which is crashing is showing how shoddily it was written. Then help
> us fix it. For everyone.. not just OpenBSD users.
