I am writing some code that was spawning a lot of processes with spawnProcess,
and it was going a bit slower than I expected; it was taking about 51 seconds
on my M1 Max mac. So I hopped over to my local copy of Twisted and applied
this patch:
1 file changed, 9 insertions(+)
src/twisted/internet/process.py | 9 +++++++++
modified src/twisted/internet/process.py
@@ -367,6 +367,15 @@ class _BaseProcess(BaseProcess):
@type environment: L{dict}.
@param kwargs: keyword arguments to L{_setupChild} method.
"""
+ from os import posix_spawn, POSIX_SPAWN_DUP2, POSIX_SPAWN_CLOSE
+ fdmap = kwargs.get("fdmap", {})
+ actions = [
+ (POSIX_SPAWN_DUP2, childFD, parentFD)
+ for (parentFD, childFD) in fdmap.items()
+ ]
+ self.pid = posix_spawn(executable, args, environment,
file_actions=actions)
+ return
+
collectorEnabled = gc.isenabled()
gc.disable()
try:
This is obviously not a complete reimplementation and might have some slight
performance advantage due to its incompleteness, but the task in question then
took 16 seconds; i.e. a 319% speedup.
Anyone want to do this implementation for real? Or want to commit to a review,
if I did it?
-g
_______________________________________________
Twisted mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/twisted.python.org/
Message archived at
https://mail.python.org/archives/list/[email protected]/message/QXMKPK2KGDP52UHCH33I3BTFIBSFZW5H/
Code of Conduct: https://twisted.org/conduct
