On Fri, Oct 14, 2005 at 10:24:42AM -0700, Jeff Hobbs wrote: > The setting of default (not fixed) is the basis of many > threading systems, pthreads included. Pthreads allows you to > adjust this with pthread_attr_setstacksize, if that's available > in your pthreads implementation (for Linux, that's mostly yes). > Note that this defines a *minimum* stack size.
"Minimum" in what sense? In practice it seems to also be a maximum, due to address space limitations, e.g.: http://groups.google.com/group/linux.kernel/tree/browse_frm/thread/a815d1e89633ab17/3211d06e15f5d3cd In AOLserver, in NsCreateThread() the stacksize config parameter is ultimately passed to pthread_attr_setstacksize() when each thread is created. So the question really becomes, just what the heck does pthread_attr_setstacksize() do on Linux and/or other platforms? Ok, I wasn't able to find a single definitive explanation, but from googling I THINK I mostly understand this now. I believe it is true that: Whatever stack size you use in pthread_attr_setstacksize() becomes the amount (in practice the exact amount, both minimum and maximum) of address space (virtual memory) allocated for that thread's stack. On Linux and presumably all other modern systems as well, the physical memory is allocated lazily only when actually needed, but the address space is allocated eagerly up front. Thus, if your actual stack usage EXCEEDS the stack size you set, boom, your process segfaults or other Bad Things happen. On a 64 bit system, with their very large address space, you could just set large stacks and be fine. But on 32 bits, setting large stacks quickly limits how many threads you can have, which is a SYSTEM WIDE (not per process) limitation. Thus people try to get away with smaller stacks, which then break horribly when their application code blows through the stack size. Presumably, there is no way for for the OS to dynamically assign additional address space when it's needed. (I don't know why exactly, but that sounds right from how this stuff actually works underneath.) Thus, if you have code blowing your stack, your options are only: 1. Set a bigger stack size, if necessary switching to x86-64 to get more address space. 2. Tweak your code to reduce your stack usage. There are somewhat crude profiling tools to help with this. 3. In theory, more radical changes: Change your programming language to not use the C stack at all. I believe that's what e.g. Stackless Python does - it's using the heap rather than the C stack. (Most functional languges already work that way.) It seems that even for high-overhead interpreted languages like Python or Tcl, people prefer not to go stackless that because doing so makes calling back and forth from Not-C to C and vice versa more complicated. Even C could avoid using the stack by instead allocating function activation frames on the heap, but I guess no one wants to do that because it would make function calls slower. Maybe it would also break binary compatibility between code compiled with other compilers as well, I'm not sure. -- Andrew Piskorski <[EMAIL PROTECTED]> http://www.piskorski.com/ -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to <[EMAIL PROTECTED]> with the body of "SIGNOFF AOLSERVER" in the email message. You can leave the Subject: field of your email blank.
