On 09/20/2010 05:09 PM, Kees Cook wrote: > On Mon, Sep 20, 2010 at 03:21:29AM -0700, Scott Ritchie wrote: >> Would there be any harm in raising this? >> >> I ask because I've seen a real world application hit this limit. The >> application in question is multithreaded and opens separate threads and >> files to work on for each core; on a 4 core machine it stays under the >> limit, while on an 8 core machine it hits it and runs into strange errors. >> >> I feel that, as we get more and more cores on machines applications like >> this are going to increasingly be a problem with a ulimit of only 1024. > > There are some unexpected things that can happen when open fd limit > (RLIMIT_NOFILE) is raised above 1024, and the main one is that the libc > select() helper macros don't expect the vectors to ever be more than > 1024[1] bits long. To quote "man select": > > An fd_set is a fixed size buffer. Executing FD_CLR() or FD_SET() with > a value of fd that is negative or is equal to or larger than FD_SETSIZE > will result in undefined behavior. > > "Undefined behavior" here mean, "will write all over your stack/heap, > etc". While there are defensive workarounds (see Apache, SSH, etc), > some applications using select can be made to crash if the system > RLIMIT_NOFILE is larger than 1024. > > So, while it can certainly make sense to raise it for individual processes > that have been validated to do the right thing with the select macros, > I would strongly recommend against raising it system-wide until glibc > fixes this bug[2] and everything that uses the macros has been recompiled > (or changed to not use select() any more). > > -Kees > > [1] FD_SETSIZE on Linux is 1024: > /usr/include/sys/select.h:#define FD_SETSIZE __FD_SETSIZE > /usr/include/bits/typesizes.h:#define __FD_SETSIZE 1024 > > [2] http://sourceware.org/bugzilla/show_bug.cgi?id=10352 > But since this is "won't fix", I guess I really just mean "never". >
Upon investigation, it seems that there is support for a "soft" (default) and a "hard" (apps-can-increase-themselves) limit. Would it be ok if we raised the hard limit? -Scott -- ubuntu-devel mailing list ubuntu-devel@lists.ubuntu.com Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel
