On 2/11/07, "Martin v. Löwis" <[EMAIL PROTECTED]> wrote: > Richard Tew schrieb: > > If these generator coroutine microthreads went ahead and part > > of it was improving the support for asynchronous calls in the > > runtime and standard library, this would also be something > > which also benefited Stackless Python. Even if they didn't go > > ahead I would be interested in hearing about whether these > > sort of changes would be of interest or not for whatever reason. > > For me to respond to such a question, I first need to understand > what changes precisely you propose. "wrapping calls out of the Python > runtime to things like network and file IO, so that microthreads > can naturally take advantage of them." is a bit vague: how precisely > would you wrap them?
I was meaning wrapping on the Stackless side of things and it was not meant as a reference to how better support for asynchronous calls might be added. You can see how wrapping is done with Stackless channels in the examples I linked to at the end of this mail on the off chance you were curious about that instead and I am happy to give any further detail needed. > I assume you would like to see operating system mechanism be used > that currently aren't used (similar to the way asynchore uses > select/poll). Yes. > If so, what mechanism would you like to use, and on what systems? It depends what you mean by mechanism. I can't go into low level details because I do not have the necessary experience or knowledge to know which approach would be best. But I do have some general thoughts and would be willing to do any work involved, if there was a feasible solution and the changes required were thought worthwhile. Let me describe the situation I currently face in order to give context, please excuse the Windows specifics and the lack of consideration for other platforms. I currently use Windows. By using asyncore and monkeypatching in a replacement socket module based on it [1] I can give users of Stackless socket operations which invisibly block at the microthread level allowing the scheduler to continue to run. However there is no support for asynchronous file IO. Now I can monkeypatch in a replacement file object which uses IO completion ports [2], but at that stage I need to poll two different resources for events. In order to avoid this it makes sense to stop using asyncore for sockets and to write a new replacement socket object based on IO completion ports. Then there are other calls which come into the mix. Like in the subprocess module. As I understand it I would need to monkeypatch the Popen.wait, and os.waitpid calls so that they were polled internally to the monkeypatching and Stackless could handle the wake up of the caller there as well. And there might be other blocking calls in the runtime which it would be worth handling also, which might not have methods any way of operating asynchronously (like sockets, files and subprocesses). The ideal mechanism at the high level would be expanding asyncore into a "one-stop shop". Where all these things can be passed into it and it can do the work to notify of events on the objects in a standard way. However even if this were judged feasible, one immediate hurdle is that files passed to it would need to have been opened in overlapped mode for IO completion ports could be used with it. I presume generally opening all files in overlapped mode on Windows would address this thought. And I guess all the other "stuff" which does not relate to IO completion ports like subprocesses would be bundled into a WaitForMultipleObject call. Perhaps there is a better way. And I of course have no concept of how this might be done on other platforms. Thanks, Richard. [1] http://svn.python.org/view/stackless/sandbox/examples/stacklesssocket.py [2] http://svn.python.org/view/stackless/sandbox/libraries/slpmonkeypatch/resources/iocp.py _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
