Hi folks,
Video of the presentation I gave last weekend at PyData NYC
regarding PyParallel just went live: https://vimeo.com/79539317
Slides are here:
https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores-1
The work was driven by the async I/O discussions around this time
last year on python-ideas. That resulted in me sending this:
http://markmail.org/thread/kh3qgjbydvxt3exw#query:+page:1+mid:arua62vllzugjy2v+state:results
....where I attempted to argue that there was a better way of
doing async I/O on Windows than the status quo of single-threaded,
non-blocking I/O with an event multiplex syscall.
I wasn't successful in convincing anyone at the time; I had no code
to back it up and I didn't articulate my plans for GIL removal at
the time either (figuring the initial suggestion would be met with
enough scepticism as is).
So, in the video above, I spend a lot of time detailing how IOCP
works on Windows, how it presents us with a better environment than
UNIX for doing asynchronous I/O, and how it paired nicely with the
other work I did on coming up with a way for multiple threads to
execute simultaneously across all cores without introducing any
speed penalties.
I'm particularly interested to hear if the video/slides helped
UNIX-centric people gain a better understanding of how Windows does
IOCP and why it would be preferable when doing async I/O.
The reverse is also true: if you still think single-threaded, non-
blocking synchronous I/O via kqueue/epoll is better than the
approach afforded by IOCP, I'm interested in hearing why.
As crazy as it sounds, my long term goal would be to try and
influence Linux and BSD kernels to implement thread-agnostic I/O
support such that an IOCP-like mechanism could be exposed; Solaris
and AIX already do this via event ports and AIX's verbatim copy of
Windows' IOCP API.
(There is some promising work already being done on Linux; see
recent MegaPipe paper for an example.)
Regards,
Trent.
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
