Am 18.04.2016 um 21:33 schrieb Imran Geriskovan:
On 4/18/16, Gustavo Carneiro <gjcarne...@gmail.com> wrote:
I don't think you need the threads.
1. If your tasks are I/O bound, coroutines are a safer way to do things,
and probably even have better performance;
Thread vs Coroutine context switching is an interesting topic.
Do you have any data for comparison?
My 2cts:
OS native (= non-green) threads are an OS scheduler driven, preemptive
multitasking approach, necessarily with context switching overhead that
is higher than a cooperative multitasking approach like asyncio event loop.
Eg the context switching with threads involves saving and restoring the
whole CPU core register set. OS native threads also involves bounding
back and forth between kernel- and userspace.
Practical evidence: name one high performance network server that is
using threads (and only threads), and not some event loop thing;)
You want N threads/processes where N is related to number of cores
and/or effective IO concurrency _and_ each thread/process run an event
loop thing. And because of the GIL, you want processes, not threads on
(C)Python.
The effective IO concurrency depends on the number of IO queues your
hardware supports (the NICs or the storage devices). The IO queues
should have affinity to the (nearest) CPU core on an SMP system also.
For network, I once did some experiments of how far Python can go. Here
is Python (PyPy) doing 630k HTTP requests/sec (12.6 GB/sec) using 40 cores:
https://github.com/crossbario/crossbarexamples/tree/master/benchmark/web
Note: that is Twisted, not asyncio, but the latter should behave the
same qualitatively.
Cheers,
/Tobias
Regards,
Imran
