On Sat, Jun 13, 2020 at 3:50 AM Edwin Zimmerman <ed...@211mainstreet.net> wrote:
>
> My previous timings were slightly inaccurate, as they compared spawning
> processes on Windows to forking on Linux. Also, I changed my timing code to
> run all process synchronously, to avoid hitting resource limits.
>
> Updated Windows (Windows 7 this time, on a four core processor):
>
> >>> timeit.timeit('x=multiprocessing.Process(target=exit);x.start();x.join()',
> >>> number=1000,globals = globals())
> 84.7111053659259
>
Thanks, I was actually going to ask about joining the processes, since
you don't really get a good indication of timings from asynchronous
operations like that. Another interesting data point is that starting
and joining in batches makes a fairly huge difference to performance,
at least on my Linux system. Starting with your example and rescaling
the number by ten to compensate for performance differences:
>>> timeit.timeit('x=multiprocessing.Process(target=exit);x.start();x.join()',
>>> number=10000,globals = globals())
14.261007152497768
Just for completeness and consistency, confirmed that adding a list
comp around it doesn't change the timings:
>>> timeit.timeit('xx=[multiprocessing.Process(target=exit) for _ in
>>> range(1)];[x.start() for x in xx];[x.join() for x in xx]',
>>> number=10000,globals = globals())
14.030426062643528
But doing a hundred at a time and then joining them all cuts the time in half!
>>> timeit.timeit('xx=[multiprocessing.Process(target=exit) for _ in
>>> range(100)];[x.start() for x in xx];[x.join() for x in xx]',
>>> number=100,globals = globals())
5.470761131495237
The difference is even more drastic with spawn, although since it's
slower, I also lowered the number of iterations.
>>> ctx = multiprocessing.get_context('spawn')
>>> timeit.timeit('x=ctx.Process(target=exit);x.start();x.join()',
>>> number=1000,globals = globals())
40.82687543518841
>>> timeit.timeit('xx=[ctx.Process(target=exit) for _ in range(100)];[x.start()
>>> for x in xx];[x.join() for x in xx]', number=10,globals =
>>> globals())8.566341979429126
8.566341979429126
Would be curious to know if that's the same on Windows.
ChrisA
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/YFRM3LNO37B5JXNYPO2T7CAVYQRGAYES/
Code of Conduct: http://python.org/psf/codeofconduct/
