Charles-François Natali added the comment:
Richard, your patch is indeed better, and fixes the problem.
With a test it should be ready to go in.
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17555
Charles-François Natali added the comment:
That's due to a leak in forker-registered handlers:
The _afterfork_registry is never cleared, so spawning processes recursively
keeps feeling it, which ends up consuming a huge amount of memory and slowing
process creation greatly.
Could you try the
Changes by Charles-François Natali cf.nat...@gmail.com:
--
nosy: +sbt
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17555
___
___
Python-bugs-list
Richard Oudkerk added the comment:
It seems to be a problem with ForkAwareThreadLock. Could you try the attached
patch?
--
Added file: http://bugs.python.org/file29593/forkawarethreadlock.patch
___
Python tracker rep...@bugs.python.org
Richard Oudkerk added the comment:
_afterfork_registry is not supposed to be cleared. But the problem with
ForkAwareThreadLocal meant that the size of the registry at generation n is
2**n!
--
___
Python tracker rep...@bugs.python.org
New submission from Marc Brünink:
Importing multiprocessing.managers and creating new processes recursively is
not a good idea. You will run out of memory very fast. On my
machine with 8GB it only takes about 20 forks until I start swapping.
--
components: Library (Lib)
files:
