Richard Oudkerk added the comment: > Richard, apart from performance, what's the advantage of this approach over > the > fork+exec version?
It is really just performance. For context running the unittests in a 1 cpu linux VM gives me fork: real 0m53.868s user 0m1.496s sys 0m9.757s fork+exec: real 1m30.951s user 0m24.598s sys 0m25.614s forkserver: real 0m54.087s user 0m1.572s # excludes descendant processes sys 0m2.336s # excludes descendant processes So running the unit tests using fork+exec takes about 4 times as much cpu time. Starting then immediately joining a trivial process in a loop gives fork: 0.025 seconds/process fork+exec: 0.245 seconds/process forkserver: 0.016 seconds/process So latency is about 10 times higher with fork+exec. > Because it seems more complicated, and although I didn't have a look a this > last > patch, I guess that most of the fork+exec version could be factorized with the > Windows version, no? The different fork methods are now implemented in separate files. The line counts are 117 popen_spawn_win32.py 80 popen_fork.py 184 popen_spawn_posix.py 191 popen_forkserver.py I don't think any more sharing between the win32 and posix cases is possible. (Note that popen_spawn_posix.py implements a cleanup helper process which is also used by the "forkserver" method.) > Since it's only intented to be used as a "debugging"/special-purpose > replacement - it > would probably be better if it could be made as simple as possible. Actually, avoiding the whole fork+threads mess is a big motivation. multiprocessing uses threads in a few places (like implementing Queue), and tries to do so as safely as possible. But unless you turn off garbage collection you cannot really control what code might be running in a background thread when the main thread forks. > Also, as you've noted, FD passing isn't supported by all Unices out there > (and we've had some reliability issues on OS-X, too). OSX does not seem to allow passing multiple ancilliary messages at once -- but you can send multiple fds in a single ancilliary message. Also, when you send fds on OSX you have to wait for a response from the other end before doing anything else. Not doing that was the cause of the previous fd passing failures in test_multiprocessing. ---------- _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue8713> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
