http://issues.apache.org/bugzilla/show_bug.cgi?id=34537
Thanks! Tom
Jeff Trawick wrote:
Apache keeps the read handle to the pipe open in the child process. Thus, if the piped logger goes away and a new piped logger doesn't
inherit the same handles, a child process can hang writing to the
logger since the child process itself is keeping the read side of the
pipe open.
The scenario where this leads to a hang is during graceful restart. The child process is busy writing to the logging pipe (enough to cause
them to block due to kernel pipe flow control). Then the parent
process whacks the piped logger and closes its handles to that pipe as
part of restart processing. The child process is stuck forever trying
to write to the pipe.
Comments/concerns? Is there a cleaner way to accomplish this?
In the discussion of PR 26467 there is a reference to an older dev@ thread, including this text:
Joe Orton says, "To fix this properly, I suppose piped loggers should not get SIGTERMedduring a graceful restart, they should read till EOF then
exit(0): then
when the last child attached to the piped logger for a particular generation quits, the pipe is closed and the piped logger terminates gracefully too, without losing log messages."
In that scenario, the patch I've attached here would not normally be needed, but would be a backup if that doesn't work. It is not clear to me that we can change that aspect of piped loggers anyway, since it can break third-party code.
A couple of tangents:
a) Wouldn't it be nice if there was a way to indicate (when registering an other-child) that the other-child process has to be maintained until all of the current generation of children goes away...
Thus, mod_cgid requests wouldn't fail during graceful restart, and this piped logging hang would potentially be avoided as well. We'd still hit the logger with SIGTERM, but only after the logger is no longer needed.
i. I still like changing the server to avoid the piped logging hang regardless of what happens elsewhere. ii. mod_cgid would need some tweaks to use a new Unix socket for new generations, so that any last requests issued by the old generation would be serviced by the old generation cgid daemon but requests issued by the new generation would be serviced by the new generation cgid daemon
b) wouldn't it be nice to have a hook similar to child-init which would be run in non-MPM child processes (e.g., mod_cgid's child or mod_fastcgi's child or children created in various other non-Apache-distributed modules)
The module would need to add the ap_run_non-mpm-child-init() call, but then core would be able to close the listeners in the child and close the piped logger read handles in the child, and potentially other unsanitary conditions could be fixed. For now we have to assume that such non-mpm children really do exit at the right time.
It is disturbing to see child processes with file descriptors they don't need, whether or not I have a use-case for when it breaks something.
