Re: [Python-Dev] Generally boared by installation (Re: Setting project home path the best way)
On Wed, Nov 28, 2012 at 2:46 PM, Eric Snow ericsnowcurren...@gmail.comwrote: I knew there was one more: http://bugs.python.org/issue16499 (CLI option for isolated mode). Along with another PYIOENCODING related one that the Blender folks reported (Christian Heimes pointed it out to me earlier today). Anyway, I created a page on the wiki for the data gathering process: http://wiki.python.org/moin/CPythonInterpreterInitialization It's now a matter of going through and sorting out: 1. What gets set during startup? 2. Where does it get set (or modified)? 3. How is that configured? Once we have a good view of that, *then* we can start looking for ways to simplify the code, make the whole system more embedding friendly (i.e. by giving the embedding app total control via a simple and clean API) and still support the proposals for improvements. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Socket timeout and completion based sockets
I'm sorry, I thought it was something that people did more often, to create different implementations of of the socket api, for which cPython provided a mere reference implementation. I know of at least three different alternative implementations, so I thought that the question were clear enough: Is the timeout mechanism supposed to be re-startable for an api that aims to conform to the socket module, or is that a mere coincidence falling out from the select/bsd based reference implementation in cPython? The docs don't say either way. (For c-level timeout mechanisms implemented for various c implementations of the bsd socket api, it is not uncommon to see it stated that after a socket operation times out, the socket is in an undefined state and should be discarded, e.g. here: http://msdn.microsoft.com/en-us/library/windows/desktop/ms740476(v=vs.85).aspx If a send or receive operation times out on a socket, the socket state is indeterminate, and should not be used; TCP sockets in this state have a potential for data loss, since the operation could be canceled at the same moment the operation was to be completed.) Anyway, as for concrete requirements: The issue I have always seen with various asynchronous libraries is their lack of composability. Everyone writes their own application loop and event queue. Merely having a standard spec and reference implementation of an application main loop object, and main event queue object, in the spirit of WSGI, would possibly remedy this. You could then hopefully assemble various different libraries in the same application, including greenlet(*) based ones. (*) Greenlets or stackless can be just another way of hiding asynchronous operations from the programmer. My favourite one, in fact. The main trick here is unwinding and replaying of calling contexts, the specific implementation by stack-slicking is mere technical detail, since it can be achieved in other ways (see soft-switching in stackless python) Cheers, K -Original Message- From: gvanros...@gmail.com [mailto:gvanros...@gmail.com] On Behalf Of Guido van Rossum Sent: 27. nóvember 2012 15:54 with stackless python. It would have been nice if you had given more context and stated your objective upfront instead of asking what appeared to be an obscure question about a technical detail Finally, I am not at all interested in greenlets ... very much unspecified at this point. NOW WOULD BE A GOOD TIME TO WRITE DOWN YOUR REQUIREMENTS. (*) Greenlets are a fine mechanism for some application areas, but ultimately not fit for the standard library, and they have some significant downsides. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Socket timeout and completion based sockets
Le Wed, 28 Nov 2012 12:13:15 +, Kristján Valur Jónsson krist...@ccpgames.com a écrit : I'm sorry, I thought it was something that people did more often, to create different implementations of of the socket api, for which cPython provided a mere reference implementation. I know of at least three different alternative implementations, so I thought that the question were clear enough: Is the timeout mechanism supposed to be re-startable for an api that aims to conform to the socket module, or is that a mere coincidence falling out from the select/bsd based reference implementation in cPython? I think recv() and send() (and other simple ops) should certainly be restartable. sendall() is another matter, I think it should be considered a best effort thing. Anyway, as for concrete requirements: The issue I have always seen with various asynchronous libraries is their lack of composability. Note: if you are using an asynchronous library, you probably shouldn't be using any form of socket timeout. Instead, you should be using timer callbacks as provided by the asynchronous library. (and the sockets themselves, of course, should be put in non-blocking mode) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Socket timeout and completion based sockets
On Wed, Nov 28, 2012 at 4:13 AM, Kristján Valur Jónsson krist...@ccpgames.com wrote: I'm sorry, I thought it was something that people did more often, to create different implementations of of the socket api, for which cPython provided a mere reference implementation. I know of at least three different alternative implementations, so I thought that the question were clear enough: Is the timeout mechanism supposed to be re-startable for an api that aims to conform to the socket module, or is that a mere coincidence falling out from the select/bsd based reference implementation in cPython? The docs don't say either way. We're going to have to decide here, since nobody has thought about this enough apparently. I see two possible answers: we can make it implementation-dependent, or we can require conforming implementations to implement properly restartable semantics (if they support timeouts at all). A third option would be to require these semantics *if the timeout option is supported* but leave it up to the implementation to support it at all (ditto for nonblocking, i.e. timeout=0). (For c-level timeout mechanisms implemented for various c implementations of the bsd socket api, it is not uncommon to see it stated that after a socket operation times out, the socket is in an undefined state and should be discarded, e.g. here: http://msdn.microsoft.com/en-us/library/windows/desktop/ms740476(v=vs.85).aspx If a send or receive operation times out on a socket, the socket state is indeterminate, and should not be used; TCP sockets in this state have a potential for data loss, since the operation could be canceled at the same moment the operation was to be completed.) Is this relevant to CPython though? Its socket implementation uses select() to implement timeouts, even on Windows, AFAIK. Anyway, as for concrete requirements: The issue I have always seen with various asynchronous libraries is their lack of composability. Everyone writes their own application loop and event queue. Merely having a standard spec and reference implementation of an application main loop object, and main event queue object, in the spirit of WSGI, would possibly remedy this. You could then hopefully assemble various different libraries in the same application, including greenlet(*) based ones. Hm. I agree with the first part of this -- and indeed I am planning to make it so that tulip's event loop can easily be replaced by another one. I'm less sure about the yield-from-based scheduler, that's the kind of thing for which it doesn't really make sense to have multiple implementations. If greenlets can work with the standard event loop interface, good for them. (Either by providing a conforming implementation that also supports greenlets, or by just using the standard implementation.) (*) Greenlets or stackless can be just another way of hiding asynchronous operations from the programmer. My favourite one, in fact. The main trick here is unwinding and replaying of calling contexts, the specific implementation by stack-slicking is mere technical detail, since it can be achieved in other ways (see soft-switching in stackless python) Yes. While none of these belong in the stdlib, I certainly want to support their viability as a 3rd party alternative to yield-from. Note however that even Christian Tismer has expressed doubts about hiding async blocks completely -- for the same reasons I'm not keen on them myself: when it's not obvious whether a particular call can cause a context switch (e.g. due to something it uses indirectly doing some kind of I/O that requires a context switch to avoid blocking), you're back in the world of threads and explicit locks and all the nightmares that come with that. Using yield or yield-from to mark switch points means you will always be aware of the possibility that a call switches (unfortunately there are other costs). --Guido Cheers, K -Original Message- From: gvanros...@gmail.com [mailto:gvanros...@gmail.com] On Behalf Of Guido van Rossum Sent: 27. nóvember 2012 15:54 with stackless python. It would have been nice if you had given more context and stated your objective upfront instead of asking what appeared to be an obscure question about a technical detail Finally, I am not at all interested in greenlets ... very much unspecified at this point. NOW WOULD BE A GOOD TIME TO WRITE DOWN YOUR REQUIREMENTS. (*) Greenlets are a fine mechanism for some application areas, but ultimately not fit for the standard library, and they have some significant downsides. -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Socket timeout and completion based sockets
On Nov 28, 2012, at 12:04 PM, Guido van Rossum gu...@python.org wrote: Anyway, as for concrete requirements: The issue I have always seen with various asynchronous libraries is their lack of composability. Everyone writes their own application loop and event queue. Merely having a standard spec and reference implementation of an application main loop object, and main event queue object, in the spirit of WSGI, would possibly remedy this. You could then hopefully assemble various different libraries in the same application, including greenlet(*) based ones. Hm. I agree with the first part of this -- and indeed I am planning to make it so that tulip's event loop can easily be replaced by another one. I'm less sure about the yield-from-based scheduler, that's the kind of thing for which it doesn't really make sense to have multiple implementations. If greenlets can work with the standard event loop interface, good for them. (Either by providing a conforming implementation that also supports greenlets, or by just using the standard implementation.) I'm really happy that you are building this in as a core feature of Tulip. It's really important. Very early on, Twisted attempted to avoid this lack of composability by explicitly delegating to other application loops; it's one of my favorite features of Twisted. Granted, no two loops we have attempted to use have themselves been composable, but there's not much we can do about that :). Still, code written on top of Twisted can always be plugged in to any other loop by simply using the appropriate reactor. (There's also a plug-in interface for the reactor and a plug-in discovery mechanism so that third parties can easily provide their own reactors if they have an unusual main loop that isn't supported by twisted itself.) I would also like to bring up https://github.com/lvh/async-pep again. If anyone really wants to dig in and enumerate the use-cases for the _lower-level_ event delivery portions of Tulip, something that would be compatible with Twisted and Tornado and so on, that PEP already has a good skeleton and could use some pull requests. -glyph___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
