On Wed, Oct 10, 2012 at 2:53 AM, Mowry, Robert <[email protected]> wrote: > I can understand the motivation behind wanting this to work, but when I > read the POSIX spec for threaded programs that call fork(), it seems > pretty clear that malloc() and free() etc are not defined to be safe to > call in the child process:
Hi Robert, you are absolutely right, and I agree that avoiding to rely on this kind of behaviour is safe for most applications. However Redis is system software, and for it to work properly and at the best of what current hardware can do, we actually rely on many things that are common in modern unix-like operating systems and are not specified by the POSIX standard (including the behaviour of the virtual memory that must support copy on write). So to write more complex and convoluted code to escape the malloc+threads+fork problem is pointless in the case of Redis since there are anyway bigger problems to make it a truly portable software. As long as we are sure that the systems we support more closely (Linux, Darwin, and *BSD basically) provide a version of malloc() that is safe in this context, we try to be pragmatic and go forward with it. That said Redis is *almost* single threaded, but we use a few threads to call system calls such as close(2) that may block our main thread resulting into latency spikes. At the same time we need to fork() as fork is used by Redis to dump the DB file on disk in the case of the RDB persistence, or to rewrite the append only file (that is a different persistence method). So we have a big advantage in this setup that can not be easily simulated by other means... as long as it works in a reliable way, we need to say at the limits (and over) of POSIX... However we ship Redis with a copy of jemalloc inside our source tree that is used when building on Linux. As long as jemalloc is safe, we are safe under Linux that is where 99.9% of deployments happen. So at least we are not subject to random glibc changes and behaviours that are outside of our control. Cheers, Salvatore -- Salvatore 'antirez' Sanfilippo open source developer - VMware http://invece.org Beauty is more important in computing than anywhere else in technology because software is so complicated. Beauty is the ultimate defence against complexity. — David Gelernter _______________________________________________ jemalloc-discuss mailing list [email protected] http://www.canonware.com/mailman/listinfo/jemalloc-discuss
