The kind of errors I mentioned ("permission denied" errors that
seem to occur without an obvious reason) have cost me at least two weeks of debugging the hard way (with ProcessExplorer etc) and caused my manager to loose his trust in Python at all... I think it is well worth the effort to keep this trouble away from the Python programmers if possible.And throughout the standard library modules, "open" is used, causing these problems as soon as sub-processes come into play. Apart from shutil.copyfile, other examples of using open that can cause trouble are in socket.py (tell me any good reason why socket handles should be inherited to child processes) and even in logging.py. For example, I used RotatingFileHandler for logging my daemon program activity. Sometimes, the logging itself caused errors, when a still-running child process had inherited the log file handle and log rotation occured.
I just wanted to express to the group at large that these experiences aren't just Henning's; we spent a *tremendous* amount of time and effort debugging serious problems that arose from file handles getting shared to subprocesses where it wasn't really expected. Specifically, the RotatingFileHandler example above. It blatantly just breaks when subprocesses are used and its an extremely obtuse process to discover why. It was very costly to the company because it came up at a bad time and was *so* obtuse of an error. At first it looked like some sort of thread-safety problem, so a lot of prying went into that before we got stumped... after all, we *knew* no other process touched that file, and the logging module (and RotatingFileHandler) claimed and looked thread-safe, so.. how could it be having a Permission Denied error when it very clearly is closing the file before rotating it? Eventually the culprit was found, but it was very painful. A couple similar issues have arisen since, and they're only slightly easier to debug once you are expecting it. But the fact that the simple and obvious features provided in the stdlib break as a result of you launching a subprocess at some point sorta sucks :) So, yeah. Anything even remotely or vaguely approaching Henning's patch would be really, really appreciated. --SH
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
