So I attempted to reproduce my issue with a debian 2.6.26 kernel, and
was unable to. The debian kernel always put mmap areas either at the
top of the address space or in the middle, depending on whether the
stack size ulimit is finite or not.
After further investigation, it seems that the behavior of smushing
all the shared-library mappings down at the bottom, conflicting with
the heap, is not in the upstream kernel, but is due to the execshield
patch that Fedora adds. Looking at the patch here:
http://cvs.fedora.redhat.com/viewvc/devel/kernel/linux-2.6-execshield.patch?revision=1.90&view=markup
I see it makes mmap use "arch_get_unmapped_exec_area" (added in the
patch) to get a free memory area if you call mmap with PROT_EXEC and
(via arch_pick_mmap_layout) "if (!(current->personality &
READ_IMPLIES_EXEC))".
And, arch_get_unmapped_exec_area starts filling from the bottom up.
Also, despite it having a comment saying "Must not let a PROT_EXEC
mapping get into the brk area", it appears that does sometimes happen,
as well. And when it happens, it causes the heap to have no room to
grow, thus breaking bash. Argh!
Although I'm not using the debian kernel or the debian execshield
patch, I suspect this behavior is also present in the Debian
execshield kernel patch but haven't verified.
This behavior was changed in Fedora CVS revision 1.103, with comment
"Fix from H.J. Lu, we were doing 32-bit randomization on 64-bit
vaddr". So the condition under which it will call
arch_pick_mmap_layout now includes "mmap_is_ia32()". Thus it appears
that on x86_64, the normal top-down stack layout will now be used for
PROT_EXEC mappings as well, therefore leaving the heap plenty of room.
Given that I found that the behavior was a kernel bug not expected
kernel behavior, that it wasn't ever in a "normal" kernel, and that
it's apparently been fixed, I wouldn't be too upset to see this bug
report result in no change to bash.
But I still think that the brain-damaged malloc included with bash
would probably be better off disabled. Every other app can use mmap to
get memory when sbrk runs out; bash was the only thing which sometimes
failed.
James
--
To UNSUBSCRIBE, email to [email protected]
with a subject of "unsubscribe". Trouble? Contact [email protected]
