Re: Why process memory starts so high up in virtual space with FreeBSD malloc?
On Mon, 01 Dec 2008 14:57:23 -0800, Yuri [EMAIL PROTECTED] wrote: Giorgos Keramidas wrote: The FreeBSD malloc(3) implementation can use either mmap() or sbrk() to obtain memory from the system. It does not 'waste a high percentage of memory' but it simply maps only high addresses (with an unmapped 'hole' in lower addresses). But the hole it leaves with MALLOC_OPTIONS='dM' is way larger than the one left by 'Dm' option. Usually malloc will keep allocating addresses higher than this initial value and will never come back and fill some parts of this gap. Therefore wasting this space. The 'D' and 'M' options set what malloc() will _prefer_, they do not force malloc() to use _only_ the particular type of memory space. As Dan explained in another post, both memory types will be used if there is need for more address space. ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Why process memory starts so high up in virtual space with FreeBSD malloc?
I am compiling the following program:
#include stdlib.h
main() { printf(0x%x\n, malloc(1)); }
in 32-bit 7.1-PRERELEASE and get 0x28201100 which is ~673MB of 4GB
address space or 16%.
When I run the same program with the google malloc (from
devel/google-perftools)
I get much lower value 0x80aa0e8 which is ~135MB of 4GB address space or
~3%.
Why FreeBSD memory allocator wastes such a high percentage of the memory
space?
Yuri
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Why process memory starts so high up in virtual space with FreeBSD malloc?
On Mon, 01 Dec 2008 13:28:48 -0800, Yuri [EMAIL PROTECTED] wrote:
I am compiling the following program:
#include stdlib.h
main() { printf(0x%x\n, malloc(1)); }
You should probably use printf(%p, ptr) to print pointers :)
in 32-bit 7.1-PRERELEASE and get 0x28201100 which is ~673MB of 4GB
address space or 16%.
When I run the same program with the google malloc (from
devel/google-perftools)
I get much lower value 0x80aa0e8 which is ~135MB of 4GB address space or
~3%.
Why FreeBSD memory allocator wastes such a high percentage of the
memory space?
The FreeBSD malloc(3) implementation can use either mmap() or sbrk() to
obtain memory from the system. It does not 'waste a high percentage of
memory' but it simply maps only high addresses (with an unmapped 'hole'
in lower addresses).
You can tune FreeBSD malloc(3) to prefer either sbrk() or mmap() by
setting the 'D' and 'M' malloc options. The following program prints
different initial allocation results when these two malloc() options are
configured differently:
% cat -n foo.c
1 #include stdio.h
2 #include stdlib.h
3
4 int
5 main(void)
6 {
7 char *p;
8
9 p = malloc(1);
10 if (p == NULL)
11 return EXIT_FAILURE;
12
13 printf(%p\n, malloc(1));
14 free(p);
15 return EXIT_SUCCESS;
16 }
%
For example:
% env MALLOC_OPTIONS='Dm' ./foo
0x8101102
% env MALLOC_OPTIONS='dM' ./foo
0x28201102
%
More details about the 'D', 'M' and other malloc() options should be
available in the manpage for your release.
- Giorgos
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Why process memory starts so high up in virtual space with FreeBSD malloc?
Giorgos Keramidas wrote: The FreeBSD malloc(3) implementation can use either mmap() or sbrk() to obtain memory from the system. It does not 'waste a high percentage of memory' but it simply maps only high addresses (with an unmapped 'hole' in lower addresses). But the hole it leaves with MALLOC_OPTIONS='dM' is way larger than the one left by 'Dm' option. Usually malloc will keep allocating addresses higher than this initial value and will never come back and fill some parts of this gap. Therefore wasting this space. Yuri ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Why process memory starts so high up in virtual space with FreeBSD malloc?
In the last episode (Dec 01), Yuri said: Giorgos Keramidas wrote: The FreeBSD malloc(3) implementation can use either mmap() or sbrk() to obtain memory from the system. It does not 'waste a high percentage of memory' but it simply maps only high addresses (with an unmapped 'hole' in lower addresses). But the hole it leaves with MALLOC_OPTIONS='dM' is way larger than the one left by 'Dm' option. Usually malloc will keep allocating addresses higher than this initial value and will never come back and fill some parts of this gap. Therefore wasting this space. Have you actually verified that space is wasted? Note that the default for malloc is DM, which lets malloc use both break- and mmap-based memory. Depending on the libc version, one or the other will be preferred, but both will be used if necessary. Also, unless you have tuned your kernel, you may only be able to allocate 512MB of memory via break() (the kern.maxdsize tunable lets you adjust this). Here's what I get with a simple test program on a month-old 7.1-PRE (before the preference got switched to mmap): $ MALLOC_OPTIONS=Dm ./malloctest Malloced 535822336 bytes. First: 0x808, Last: 0x27e8 $ MALLOC_OPTIONS=dM ./malloctest Malloced 2542796800 bytes. First: 0x2820, Last: 0xbfa0 $ MALLOC_OPTIONS=DM ./malloctest Malloced 3078619136 bytes. First: 0x808, Last: 0xbfa0 $ ./malloctest Malloced 3078619136 bytes. First: 0x808, Last: 0xbfa0 So using only break(), I can allocate 511 MB. Using only mmap(), I can allocate 2.36 GB. Using both (the default) I can allocate 2.86 GB. -- Dan Nelson [EMAIL PROTECTED] ___ freebsd-questions@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-questions To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Why process memory starts so high up in virtual space with FreeBSD malloc?
In the last episode (Dec 01), Dan Nelson said:
Here's what I get with a simple test program on a month-old 7.1-PRE
Gah. silly mailing-list attachment stripper.
#include stdlib.h
#include stdio.h
int main(void)
{
size_t malloced = 0;
size_t chunksize = 1024*1024;
void *first = NULL;
void *last = NULL;
void *ptr;
while ((ptr = malloc(chunksize)) != NULL) {
if (first == NULL)
first = ptr;
last = ptr;
malloced += chunksize;
}
printf(Malloced %zu bytes. First: %p, Last: %p\n, malloced, first, last);
exit(0);
}
--
Dan Nelson
[EMAIL PROTECTED]
___
freebsd-questions@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-questions
To unsubscribe, send any mail to [EMAIL PROTECTED]
