Re: [Python-Dev] provisional status for asyncio
On 08/28/2015 08:44 AM, Brett Cannon wrote: On Fri, 28 Aug 2015 at 08:35 Yury Selivanov yselivanov...@gmail.com mailto:yselivanov...@gmail.com wrote: Unfortunately, separating it from the standard library is something that I don't think we can do so late in the 3.5 release candidates process. Ultimately it's Larry's call, but I don't see why we couldn't. If we were talking about something as low-level as the urllib package then I would agree, but beyond its own tests is there anything in the stdlib that depends on asyncio? I'm flexible here. My concern is shipping high-quality software. Removing an entire package outright, even at such a late date, is pretty low-risk. But before I'd allow it, you'd have to get a BDFL pronouncement (or BDFL-delegate pronouncement). //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
Larry, what will the release cycle for 3.5.x look like? Can we do bugfix releases every 3 or 4 months? Yury On 2015-08-29 1:36 PM, Larry Hastings wrote: On 08/28/2015 08:44 AM, Brett Cannon wrote: On Fri, 28 Aug 2015 at 08:35 Yury Selivanov yselivanov...@gmail.com wrote: Unfortunately, separating it from the standard library is something that I don't think we can do so late in the 3.5 release candidates process. Ultimately it's Larry's call, but I don't see why we couldn't. If we were talking about something as low-level as the urllib package then I would agree, but beyond its own tests is there anything in the stdlib that depends on asyncio? I'm flexible here. My concern is shipping high-quality software. Removing an entire package outright, even at such a late date, is pretty low-risk. But before I'd allow it, you'd have to get a BDFL pronouncement (or BDFL-delegate pronouncement). //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 08/29/2015 12:18 PM, Yury Selivanov wrote: Larry, what will the release cycle for 3.5.x look like? Can we do bugfix releases every 3 or 4 months? It's usually more like every six. I've proposed doing them a little more frequently and gotten push back; a new Python release causes a bunch of work for a lot of people. You could just hope for another Heartbleed bug, //arry/ ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 30 August 2015 at 05:16, Yury Selivanov yselivanov...@gmail.com wrote: Since there is at least some possibility that we might have another discussion about asyncio removal from the stdlib in 3.6, should I just reuse the warning we had in 3.4 for asyncio: Note: The asyncio package has been included in the standard library on a provisional basis. Backwards incompatible changes (up to and including removal of the module) may occur if deemed necessary by the core developers. ? I don't see much chance of us actually removing it - the most I see us doing is potentially shifting to a bundled 3rd party library update model if there's still a high churn rate in the API after another release cycle. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 28 August 2015 at 00:25, Nick Coghlan ncogh...@gmail.com wrote: I'd suggest including a clearer motivation there: Note: The asyncio package has been included in the standard library on a provisional basis, and thus may gain new APIs and capabilities in maintenance releases as it matures. Backwards incompatible changes may occur if deemed absolutely necessary by the core developers. I'm happy with a statement like this offering additional guidance, but I think that formally we should stick with the current provisional-or-not situation (with asyncio remaining provisional for another release, if the asyncio devs feel that's needed). Ultimately, end users only really have two choices - use a library or not - so adding extra levels of provisionalness actually complicates their choice rather than simplifying it. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
I think this is a packaging problem. In an ideal world, Python would ship some version of asyncio, but you would also be able to pip install a newer version into your virtual environment, and it would override the version in stdlib. As it stands now, however, if you pip install another version of asyncio, the version in stdlib still takes precedence. What I end up doing in my (non open source) projects is to include a copy of asyncio and manually modify sys.path to point to it. Can we fix pip/virtualenv instead? On 28 Aug 2015 10:02 am, Victor Stinner victor.stin...@gmail.com wrote: Hi, 2015-08-27 23:15 GMT+02:00 Yury Selivanov yselivanov...@gmail.com: Recently, in an asyncio related issue [1], Guido said that new features for asyncio have to wait till 3.6, since asyncio is no longer a provisional package. (...) For example, there is an issue [2] to add starttls support to asyncio. (...) Aside from adding new APIs, we also have to improve debugging capabilities. (...) I would propose something more radical: remove asyncio from the stdlib. PEP 411: While it is considered an unlikely outcome, such packages *may even be removed* from the standard library without a deprecation period if the concerns regarding their API or maintenance prove well-founded. As an asyncio developer, I'm not sure that asyncio fits well into the Python stdlib. The release cycle of feature release is long (18 months? or more?), the release cycle for bugfix release is sometimes also too long (1 month at least). It's also frustrating to have subtle API differences between Python 3.3, 3.4 and 3.5. For example, asyncio.JoinableQueue was removed in Python 3.5, asyncio.Queue can *now* be used instead, but asyncio.JoinableQueue should be used on older Python version... It means that you have to write different code depending on your Python version to support all Python versions. I can give much more examples of missing asyncio features. Example: Windows proactor event loop doesn't support signals (CTRL+C) nor UDP. asyncio is moving so fast, that changes are not documented at all in Misc/NEWS or Doc/whatsnews/x.y.rst. I tried to document changes in my fork Trollius. See its changelog to have an idea how fast asyncio is still changing: http://trollius.readthedocs.org/changelog.html I don't know how users will react to the removal of asyncio from the stdlib (asyncio is not trusted/supported by Python?). The idea is to install it using pip: pip install asyncio. The major difference is that pip install -U asyncio allows to retrieve the latest asyncio version, independently of your Python version. Hum, I don't know if it works with Python 3.4 (which asyncio module is used in this case?). Developing asyncio only on Github would avoid the risk of having patches temporary only in Github or only in CPython. It avoids the need to synchronize the Git (Github) and Mercurial (CPython) repositories. Compare Python release dates with Twisted, Tornado and eventlet release dates. Twisted releases: * 2015-01-30: 15.0.0 * 2015-04-13: 15.1.0 * 2015-05-19: 15.2.0 * 2015-08-04: 15.3.0 Tornado releases: * 2014-07-15: 4.0 * 2015-02-07: 4.1 * 2015-05-27: 4.2 * 2015-07-17: 4.2.1 eventlet releases: * 2015-02-25: 0.17.1 * 2015-04-03: 0.17.2 * 2015-04-09: 0.17.3 * 2015-05-08: 0.17.4 Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/gjcarneiro%40gmail.com ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
Hi, 2015-08-27 23:15 GMT+02:00 Yury Selivanov yselivanov...@gmail.com: Recently, in an asyncio related issue [1], Guido said that new features for asyncio have to wait till 3.6, since asyncio is no longer a provisional package. (...) For example, there is an issue [2] to add starttls support to asyncio. (...) Aside from adding new APIs, we also have to improve debugging capabilities. (...) I would propose something more radical: remove asyncio from the stdlib. PEP 411: While it is considered an unlikely outcome, such packages *may even be removed* from the standard library without a deprecation period if the concerns regarding their API or maintenance prove well-founded. As an asyncio developer, I'm not sure that asyncio fits well into the Python stdlib. The release cycle of feature release is long (18 months? or more?), the release cycle for bugfix release is sometimes also too long (1 month at least). It's also frustrating to have subtle API differences between Python 3.3, 3.4 and 3.5. For example, asyncio.JoinableQueue was removed in Python 3.5, asyncio.Queue can *now* be used instead, but asyncio.JoinableQueue should be used on older Python version... It means that you have to write different code depending on your Python version to support all Python versions. I can give much more examples of missing asyncio features. Example: Windows proactor event loop doesn't support signals (CTRL+C) nor UDP. asyncio is moving so fast, that changes are not documented at all in Misc/NEWS or Doc/whatsnews/x.y.rst. I tried to document changes in my fork Trollius. See its changelog to have an idea how fast asyncio is still changing: http://trollius.readthedocs.org/changelog.html I don't know how users will react to the removal of asyncio from the stdlib (asyncio is not trusted/supported by Python?). The idea is to install it using pip: pip install asyncio. The major difference is that pip install -U asyncio allows to retrieve the latest asyncio version, independently of your Python version. Hum, I don't know if it works with Python 3.4 (which asyncio module is used in this case?). Developing asyncio only on Github would avoid the risk of having patches temporary only in Github or only in CPython. It avoids the need to synchronize the Git (Github) and Mercurial (CPython) repositories. Compare Python release dates with Twisted, Tornado and eventlet release dates. Twisted releases: * 2015-01-30: 15.0.0 * 2015-04-13: 15.1.0 * 2015-05-19: 15.2.0 * 2015-08-04: 15.3.0 Tornado releases: * 2014-07-15: 4.0 * 2015-02-07: 4.1 * 2015-05-27: 4.2 * 2015-07-17: 4.2.1 eventlet releases: * 2015-02-25: 0.17.1 * 2015-04-03: 0.17.2 * 2015-04-09: 0.17.3 * 2015-05-08: 0.17.4 Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 28 August 2015 at 10:46, Gustavo Carneiro gjcarne...@gmail.com wrote: I think this is a packaging problem. In an ideal world, Python would ship some version of asyncio, but you would also be able to pip install a newer version into your virtual environment, and it would override the version in stdlib. As it stands now, however, if you pip install another version of asyncio, the version in stdlib still takes precedence. What I end up doing in my (non open source) projects is to include a copy of asyncio and manually modify sys.path to point to it. Can we fix pip/virtualenv instead? It's not a pip/virtualenv issue - it's a result of the way sys.path is set (in site.py IIRC). And it was a deliberate decision to put the stdlib before user installed packages, precisely so that random PyPI packages can't override the standard behaviour. (Setuptools included some hacks to circumvent this, but they are pretty nasty, and TBH not widely liked...) Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
Victor, On 2015-08-28 5:01 AM, Victor Stinner wrote: Hi, 2015-08-27 23:15 GMT+02:00 Yury Selivanov yselivanov...@gmail.com: Recently, in an asyncio related issue [1], Guido said that new features for asyncio have to wait till 3.6, since asyncio is no longer a provisional package. (...) For example, there is an issue [2] to add starttls support to asyncio. (...) Aside from adding new APIs, we also have to improve debugging capabilities. (...) I would propose something more radical: remove asyncio from the stdlib. I too would enjoy more frequent release schedule of asyncio. Unfortunately, separating it from the standard library is something that I don't think we can do so late in the 3.5 release candidates process. PEP 411: While it is considered an unlikely outcome, such packages *may even be removed* from the standard library without a deprecation period if the concerns regarding their API or maintenance prove well-founded. As an asyncio developer, I'm not sure that asyncio fits well into the Python stdlib. The release cycle of feature release is long (18 months? or more?), the release cycle for bugfix release is sometimes also too long (1 month at least). It's also frustrating to have subtle API differences between Python 3.3, 3.4 and 3.5. For example, asyncio.JoinableQueue was removed in Python 3.5, asyncio.Queue can *now* be used instead, but asyncio.JoinableQueue should be used on older Python version... It means that you have to write different code depending on your Python version to support all Python versions. I can give much more examples of missing asyncio features. Example: Windows proactor event loop doesn't support signals (CTRL+C) nor UDP. asyncio is moving so fast, that changes are not documented at all in Misc/NEWS or Doc/whatsnews/x.y.rst. I tried to document changes in my fork Trollius. See its changelog to have an idea how fast asyncio is still changing: http://trollius.readthedocs.org/changelog.html I don't know how users will react to the removal of asyncio from the stdlib (asyncio is not trusted/supported by Python?). That's another concern. The idea is to install it using pip: pip install asyncio. The major difference is that pip install -U asyncio allows to retrieve the latest asyncio version, independently of your Python version. Hum, I don't know if it works with Python 3.4 (which asyncio module is used in this case?). The one from the stdlib. Developing asyncio only on Github would avoid the risk of having patches temporary only in Github or only in CPython. It avoids the need to synchronize the Git (Github) and Mercurial (CPython) repositories. Compare Python release dates with Twisted, Tornado and eventlet release dates. Twisted releases: * 2015-01-30: 15.0.0 * 2015-04-13: 15.1.0 * 2015-05-19: 15.2.0 * 2015-08-04: 15.3.0 Tornado releases: * 2014-07-15: 4.0 * 2015-02-07: 4.1 * 2015-05-27: 4.2 * 2015-07-17: 4.2.1 eventlet releases: * 2015-02-25: 0.17.1 * 2015-04-03: 0.17.2 * 2015-04-09: 0.17.3 * 2015-05-08: 0.17.4 These are very good stats. It shows that even mature async libraries require frequent release cycles. Anyways, I vote for at least keeping asyncio provisional in 3.5.x. I'd be glad if we can consider to have more bugfix releases of 3.5 (and 3.6), say every 3 months. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On Fri, 28 Aug 2015 at 08:35 Yury Selivanov yselivanov...@gmail.com wrote: Victor, On 2015-08-28 5:01 AM, Victor Stinner wrote: Hi, 2015-08-27 23:15 GMT+02:00 Yury Selivanov yselivanov...@gmail.com: Recently, in an asyncio related issue [1], Guido said that new features for asyncio have to wait till 3.6, since asyncio is no longer a provisional package. (...) For example, there is an issue [2] to add starttls support to asyncio. (...) Aside from adding new APIs, we also have to improve debugging capabilities. (...) I would propose something more radical: remove asyncio from the stdlib. I too would enjoy more frequent release schedule of asyncio. Unfortunately, separating it from the standard library is something that I don't think we can do so late in the 3.5 release candidates process. Ultimately it's Larry's call, but I don't see why we couldn't. If we were talking about something as low-level as the urllib package then I would agree, but beyond its own tests is there anything in the stdlib that depends on asyncio? -Brett PEP 411: While it is considered an unlikely outcome, such packages *may even be removed* from the standard library without a deprecation period if the concerns regarding their API or maintenance prove well-founded. As an asyncio developer, I'm not sure that asyncio fits well into the Python stdlib. The release cycle of feature release is long (18 months? or more?), the release cycle for bugfix release is sometimes also too long (1 month at least). It's also frustrating to have subtle API differences between Python 3.3, 3.4 and 3.5. For example, asyncio.JoinableQueue was removed in Python 3.5, asyncio.Queue can *now* be used instead, but asyncio.JoinableQueue should be used on older Python version... It means that you have to write different code depending on your Python version to support all Python versions. I can give much more examples of missing asyncio features. Example: Windows proactor event loop doesn't support signals (CTRL+C) nor UDP. asyncio is moving so fast, that changes are not documented at all in Misc/NEWS or Doc/whatsnews/x.y.rst. I tried to document changes in my fork Trollius. See its changelog to have an idea how fast asyncio is still changing: http://trollius.readthedocs.org/changelog.html I don't know how users will react to the removal of asyncio from the stdlib (asyncio is not trusted/supported by Python?). That's another concern. The idea is to install it using pip: pip install asyncio. The major difference is that pip install -U asyncio allows to retrieve the latest asyncio version, independently of your Python version. Hum, I don't know if it works with Python 3.4 (which asyncio module is used in this case?). The one from the stdlib. Developing asyncio only on Github would avoid the risk of having patches temporary only in Github or only in CPython. It avoids the need to synchronize the Git (Github) and Mercurial (CPython) repositories. Compare Python release dates with Twisted, Tornado and eventlet release dates. Twisted releases: * 2015-01-30: 15.0.0 * 2015-04-13: 15.1.0 * 2015-05-19: 15.2.0 * 2015-08-04: 15.3.0 Tornado releases: * 2014-07-15: 4.0 * 2015-02-07: 4.1 * 2015-05-27: 4.2 * 2015-07-17: 4.2.1 eventlet releases: * 2015-02-25: 0.17.1 * 2015-04-03: 0.17.2 * 2015-04-09: 0.17.3 * 2015-05-08: 0.17.4 These are very good stats. It shows that even mature async libraries require frequent release cycles. Anyways, I vote for at least keeping asyncio provisional in 3.5.x. I'd be glad if we can consider to have more bugfix releases of 3.5 (and 3.6), say every 3 months. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
2015-08-28 17:44 GMT+02:00 Brett Cannon br...@python.org: Ultimately it's Larry's call, but I don't see why we couldn't. If we were talking about something as low-level as the urllib package then I would agree, but beyond its own tests is there anything in the stdlib that depends on asyncio? At the beginning, asyncio was more fat: it contained a HTTP client and server. The HTTP code was moved out of asyncio (into the aiohttp project, which is great and popular by the way) to keep asyncio small and simple. There is no plan to put libraries based on asyncio into the stdlib. And I don't think that existing modules should start to support asyncio, it's more the opposite :-) For example, urllib.request, http.client and http.server are replaced by aiohttp. I don't think that it would make sense to add new asynchronous methods to these modules. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
2015-08-28 17:34 GMT+02:00 Yury Selivanov yselivanov...@gmail.com: I too would enjoy more frequent release schedule of asyncio. The problem is also to allow users to upgrade easily asyncio to retrieve new features, or simply latest bug fixes. As explained in other emails, if asyncio is part of the stlidb, it's *not* possible to upgrade it using pip install -U asyncio, except if you hack sys.path. Usually, Python 3 comes with the system, for an user, it's hard to upgrade it. For example, Ubuntu Trusty still provides Python 3.4.0. asyncio doesn't have the loop.create_task() method for example in this release. asyncio got many bugfixes and some new features between 3.4.0 and 3.4.3. It's much easier to upgrade a third-party library than upgrading the system python. Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On August 28, 2015 at 6:36:21 AM, Paul Moore (p.f.mo...@gmail.com) wrote: On 28 August 2015 at 10:46, Gustavo Carneiro wrote: I think this is a packaging problem. In an ideal world, Python would ship some version of asyncio, but you would also be able to pip install a newer version into your virtual environment, and it would override the version in stdlib. As it stands now, however, if you pip install another version of asyncio, the version in stdlib still takes precedence. What I end up doing in my (non open source) projects is to include a copy of asyncio and manually modify sys.path to point to it. Can we fix pip/virtualenv instead? It's not a pip/virtualenv issue - it's a result of the way sys.path is set (in site.py IIRC). And it was a deliberate decision to put the stdlib before user installed packages, precisely so that random PyPI packages can't override the standard behaviour. (Setuptools included some hacks to circumvent this, but they are pretty nasty, and TBH not widely liked...) Right, the default sys.path is stdlib user-packages site-packages. I would personally prefer it if looked more like user-packages site-packages stdlib but I’m not sure how popular that opinion is :) - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On August 28, 2015 at 12:01:25 PM, Victor Stinner (victor.stin...@gmail.com) wrote: 2015-08-28 17:34 GMT+02:00 Yury Selivanov : I too would enjoy more frequent release schedule of asyncio. The problem is also to allow users to upgrade easily asyncio to retrieve new features, or simply latest bug fixes. As explained in other emails, if asyncio is part of the stlidb, it's *not* possible to upgrade it using pip install -U asyncio, except if you hack sys.path. Usually, Python 3 comes with the system, for an user, it's hard to upgrade it. For example, Ubuntu Trusty still provides Python 3.4.0. asyncio doesn't have the loop.create_task() method for example in this release. asyncio got many bugfixes and some new features between 3.4.0 and 3.4.3. It's much easier to upgrade a third-party library than upgrading the system python. Unless we fix the sys.path ordering to make it possible. - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 29 August 2015 at 04:44, Gustavo Carneiro gjcarne...@gmail.com wrote: On 28 August 2015 at 17:11, Yury Selivanov yselivanov...@gmail.com wrote: On 2015-08-28 11:44 AM, Brett Cannon wrote: Unfortunately, separating it from the standard library is something that I don't think we can do so late in the 3.5 release candidates process. Ultimately it's Larry's call, but I don't see why we couldn't. If we were talking about something as low-level as the urllib package then I would agree, but beyond its own tests is there anything in the stdlib that depends on asyncio? As Victor already replied, asyncio is pretty self contained, and nothing in the stdlib depends on it. If we really can remove it, let's consider it. -1 for removing it. It is way too late for 3.5 IMHO. You should have proposed it at least 6 months ago. This feels too rushed. +1 for changing sys.path to allow pip-installed asyncio to override the stdlib version (for 3.6). Some developers complain that NodeJS's packaging system is better that Python, maybe this lack of flexibility is one of the reasons... I think ensurepip now offers us a better option for dealing with this kind of situation: create a new category of recommended 3rd party modules, which have their own upgrade cycle and maintenance model, independent of the standard library, but use pip and bundled wheel files to install them by default in the CPython installers. This allows them to be made available by default to new Python users, but also still easily kept in sync across all supported Python versions. We've started inching our way towards this already, with provisional modules, the IDLE enhancement exception for all releases, the network security backports to Python 2.7, and ensurepip itself. This means it isn't just asyncio that could benefit from such an enhancement to our modularisation capabilities - we could also apply it to tkinter, idle and distutils (all of which are often unbundled by downstream redistributors anyway), and potentially even to particularly security sensitive components like the ssl module. From a Linux downstream perspective, APT has long supported a distinction between hard requirements, recommended additions, and suggested ones in the Debian world, and RPM recently gained that capability as well, so such a change would actually make our lives easier, since bundled as a wheel file in the CPython Windows and Mac OS X installers would map to a Recommends or Suggests downstream, rather than a hard requirement. However, we don't need to rush into this. As long as we keep asyncio provisional for 3.5.0, then we can take our time in considering what we want to do in terms of its ongoing maintenance and release management model, and then apply that new approach to at least asyncio for 3.5.x, and consider modularising the development and release cycles for other components of interest in 3.6+ (and potentially 2.7.x in some cases). Regards, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
2015-08-28 18:02 GMT+02:00 Donald Stufft don...@stufft.io: Unless we fix the sys.path ordering to make it possible. The problem is the deadline: Python 3.5 final is scheduled for the September, 13. We have 2 weeks to decide what to do with asyncio. I don't think that it's a good idea to modify how sys.path is built between an RC version (ex: 3.5rc2) and 3.5.0 final... Victor ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 2015-08-28 11:44 AM, Brett Cannon wrote: Unfortunately, separating it from the standard library is something that I don't think we can do so late in the 3.5 release candidates process. Ultimately it's Larry's call, but I don't see why we couldn't. If we were talking about something as low-level as the urllib package then I would agree, but beyond its own tests is there anything in the stdlib that depends on asyncio? As Victor already replied, asyncio is pretty self contained, and nothing in the stdlib depends on it. If we really can remove it, let's consider it. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On August 28, 2015 at 12:07:48 PM, Victor Stinner (victor.stin...@gmail.com) wrote: 2015-08-28 18:02 GMT+02:00 Donald Stufft : Unless we fix the sys.path ordering to make it possible. The problem is the deadline: Python 3.5 final is scheduled for the September, 13. We have 2 weeks to decide what to do with asyncio. I don't think that it's a good idea to modify how sys.path is built between an RC version (ex: 3.5rc2) and 3.5.0 final... Yea, there’s always 3.6 though! - Donald Stufft PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 28 August 2015 at 17:11, Yury Selivanov yselivanov...@gmail.com wrote: On 2015-08-28 11:44 AM, Brett Cannon wrote: Unfortunately, separating it from the standard library is something that I don't think we can do so late in the 3.5 release candidates process. Ultimately it's Larry's call, but I don't see why we couldn't. If we were talking about something as low-level as the urllib package then I would agree, but beyond its own tests is there anything in the stdlib that depends on asyncio? As Victor already replied, asyncio is pretty self contained, and nothing in the stdlib depends on it. If we really can remove it, let's consider it. -1 for removing it. It is way too late for 3.5 IMHO. You should have proposed it at least 6 months ago. This feels too rushed. +1 for changing sys.path to allow pip-installed asyncio to override the stdlib version (for 3.6). Some developers complain that NodeJS's packaging system is better that Python, maybe this lack of flexibility is one of the reasons... -- Gustavo J. A. M. Carneiro Gambit Research The universe is always one step beyond logic. -- Frank Herbert ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 2015-08-27 5:31 PM, Yury Selivanov wrote: On 2015-08-27 5:24 PM, Brett Cannon wrote: My proposal is to amend PEP 411 with two levels of provisional packages: Level 1: Backwards incompatible changes might be introduced in point releases. Level 2: Only backwards compatible changes can be introduced in new point releases. How is this any different from the normal compatibility promise we have for any non-provisional code in the stdlib? And by point release I assume you mean a new minor release, e.g. 3.5 - 3.6. Right, my mistake, I indeed meant minor releases. The difference is that right now we don't introduce new features (regardless of backwards compatibility promises) for any non-provisional code in minor releases, we can only do bug fixes. My proposal is to enable asyncio receiving new strictly backwards compatible APIs/features (and bug fixes too, of course) in minor releases (3.5.x). Turns out I was lost in terminology :) Considering that Python versioning is defined as major.minor.micro, I'll rephrase the proposal: Level 1: Backwards incompatible changes might be introduced in new Python releases (including micro releases) Level 2: Only backwards compatible changes (new APIs including) can be introduced in micro releases. Sorry for the confusion. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
Please use feature release (e.g. 3.5 - 3.6) and bugfix release (e.g. 3.5.0 - 3.5.1). The major/minor terminology is confusing, since something like 2 - 3 isn't just major, it is earthshattering. :-) On Thu, Aug 27, 2015 at 2:39 PM, Yury Selivanov yselivanov...@gmail.com wrote: On 2015-08-27 5:31 PM, Yury Selivanov wrote: On 2015-08-27 5:24 PM, Brett Cannon wrote: My proposal is to amend PEP 411 with two levels of provisional packages: Level 1: Backwards incompatible changes might be introduced in point releases. Level 2: Only backwards compatible changes can be introduced in new point releases. How is this any different from the normal compatibility promise we have for any non-provisional code in the stdlib? And by point release I assume you mean a new minor release, e.g. 3.5 - 3.6. Right, my mistake, I indeed meant minor releases. The difference is that right now we don't introduce new features (regardless of backwards compatibility promises) for any non-provisional code in minor releases, we can only do bug fixes. My proposal is to enable asyncio receiving new strictly backwards compatible APIs/features (and bug fixes too, of course) in minor releases (3.5.x). Turns out I was lost in terminology :) Considering that Python versioning is defined as major.minor.micro, I'll rephrase the proposal: Level 1: Backwards incompatible changes might be introduced in new Python releases (including micro releases) Level 2: Only backwards compatible changes (new APIs including) can be introduced in micro releases. Sorry for the confusion. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 2015-08-27 5:24 PM, Brett Cannon wrote: My proposal is to amend PEP 411 with two levels of provisional packages: Level 1: Backwards incompatible changes might be introduced in point releases. Level 2: Only backwards compatible changes can be introduced in new point releases. How is this any different from the normal compatibility promise we have for any non-provisional code in the stdlib? And by point release I assume you mean a new minor release, e.g. 3.5 - 3.6. Right, my mistake, I indeed meant minor releases. The difference is that right now we don't introduce new features (regardless of backwards compatibility promises) for any non-provisional code in minor releases, we can only do bug fixes. My proposal is to enable asyncio receiving new strictly backwards compatible APIs/features (and bug fixes too, of course) in minor releases (3.5.x). Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] provisional status for asyncio
Recently, in an asyncio related issue [1], Guido said that new features for asyncio have to wait till 3.6, since asyncio is no longer a provisional package. Later, in an off-list conversation, he suggested that this topic should be discussed on python-dev, and that it might indeed make sense to either write a new PEP for cases like this or to augment PEP 411. My opinion on this topic is that we must maintain full backwards compatibility for asyncio from 3.5.0, as it is now widely used, and there is quite a big ecosystem around it. However, asyncio is simply not mature enough to be completely feature frozen for about 2 years. For example, there is an issue [2] to add starttls support to asyncio. It is an essential feature, because some protocols start as clear text and upgrade to TLS later, for example PostgreSQL PQ3 protocol. It's very hard to implement starttls on top of asyncio, lots of code will have to be duplicated -- it's a feature that has to implemented in the asyncio core. Aside from adding new APIs, we also have to improve debugging capabilities. One example is using os.fork() from within a running event loop -- it must be avoided by all means. There are safe ways to fork in asyncio applications (and I plan to document them soon), but asyncio should raise an exception in debug mode if this happens (see issue [3]). These are just two immediate issues that I have in mind. In reality, asyncio is quite young compared to frameworks like Twisted, which had years to mature, and accumulate essential features. My proposal is to amend PEP 411 with two levels of provisional packages: Level 1: Backwards incompatible changes might be introduced in point releases. Level 2: Only backwards compatible changes can be introduced in new point releases. With the above amendments, asyncio status should be restated as a level-2 provisional package. I'm CC-ing authors of PEP 411 Nick and Eli. Thank you, Yury [1] http://bugs.python.org/issue23630 [2] http://bugs.python.org/issue23749 [3] http://bugs.python.org/issue21998 ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On Thu, 27 Aug 2015 at 14:39 Yury Selivanov yselivanov...@gmail.com wrote: On 2015-08-27 5:31 PM, Yury Selivanov wrote: On 2015-08-27 5:24 PM, Brett Cannon wrote: My proposal is to amend PEP 411 with two levels of provisional packages: Level 1: Backwards incompatible changes might be introduced in point releases. Level 2: Only backwards compatible changes can be introduced in new point releases. How is this any different from the normal compatibility promise we have for any non-provisional code in the stdlib? And by point release I assume you mean a new minor release, e.g. 3.5 - 3.6. Right, my mistake, I indeed meant minor releases. The difference is that right now we don't introduce new features (regardless of backwards compatibility promises) for any non-provisional code in minor releases, we can only do bug fixes. My proposal is to enable asyncio receiving new strictly backwards compatible APIs/features (and bug fixes too, of course) in minor releases (3.5.x). Turns out I was lost in terminology :) Considering that Python versioning is defined as major.minor.micro, I'll rephrase the proposal: Level 1: Backwards incompatible changes might be introduced in new Python releases (including micro releases) Level 2: Only backwards compatible changes (new APIs including) can be introduced in micro releases. In that case I don't think it's a good idea for something that has widespread use to get new APIs in a micro release; I lived the 2.2.1/boolean event and I don't want to go through that again. If a module is used enough to warrant not breaking backwards-compatibility then it warrants not being provisional and being like any other module. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 2015-08-27 5:53 PM, Brett Cannon wrote: Considering that Python versioning is defined as major.minor.micro, I'll rephrase the proposal: Level 1: Backwards incompatible changes might be introduced in new Python releases (including micro releases) Level 2: Only backwards compatible changes (new APIs including) can be introduced in micro releases. In that case I don't think it's a good idea for something that has widespread use to get new APIs in a micro release; I lived the 2.2.1/boolean event and I don't want to go through that again. If a module is used enough to warrant not breaking backwards-compatibility then it warrants not being provisional and being like any other module. I wasn't using Python 2.2/2.3, but from what I could google the 2.2.1/boolean event you mention was introducing True/False/bool built-ins. This sounds like a language-level change, as opposed to new API in a stdlib module, which is a different scale. My understanding about adding new features in bugfix releases (3.5.x) is that you might end up in a situation where your 3.5 code developed on 3.5.x suddenly stops working on 3.5.y. Yes, you have to be careful about how you deploy and test your code when using a provisional package. But the thing about asyncio is that it *is* still provisional in 3.4. During 3.4 release cycle we introduced many new features to asyncio, and to be honest, I haven't heard anybody complaining. I believe that main motivation for making asyncio non-provisional was to guarantee that we won't introduce backwards-incompatible changes to it. Given the fact that asyncio sees some adoption, I support that from now on we will guarantee that backwards compatibility is preserved. But withholding new useful (and sometimes essential) features till 3.6.0 is out (March 2017?) sounds wrong to me. I should also mention that asyncio is different from other packages in the stdlib: 1. It's new, it virtually didn't exist before 3.4.0. 2. It's not a module, it's a framework. If it lacks a core feature (like starttls) that is hard to implement as an add-on, you're basically forced either copy/paste a lot of code or to fork asyncio. And if you fork it, how will your dependencies can be upgraded to use that fork? I want to continue (as we did in 3.4.x releases) evolving asyncio on a faster scale than CPython currently evolves, *and* to guarantee that we won't break existing code. That's why I propose to tweak our definition of provisional packages. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 2015-08-27 6:44 PM, Guido van Rossum wrote: Maybe asyncio should just be kept provisional during 3.5, with a separate promise to remain backward compatible? I'm +1. I'm also certain that by 3.6.0 we will stabilize asyncio to the point we can freeze it like any other stdlib module. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On 28 August 2015 at 09:00, Yury Selivanov yselivanov...@gmail.com wrote: Brett, On 2015-08-27 6:46 PM, Brett Cannon wrote: [...] I say it's either fully provisional or it's not. I'm fine with extending its provisional status another feature release as long as it clearly states that in What's New for 3.5, but I don't think this granularity guarantee of not breaking APIs while adding new features is worth it. What if you want to add a new feature that is really hard to do right without breaking compatibility? We all know how trying that is. If you truly want to keep an accelerated development cycle, then short of releasing new stdlib versions every 6 months separate from the language then I say keep it provisional for 3.5. I'm fine with keeping it provisional in 3.5 (and Guido suggests this idea too in this thread). A lot of companies (including big ones) are using asyncio already, despite the fact that it's provisional in 3.4. I seriously doubt that keeping it provisional in 3.5 will do any harm. asyncio documentation in 3.4.x has the following notes section: Note: The asyncio package has been included in the standard library on a provisional basis. Backwards incompatible changes (up to and including removal of the module) may occur if deemed necessary by the core developers. I suggest to add a slightly less strong-worded note to 3.5 documentation: Note: The asyncio package has been included in the standard library on a provisional basis. Backwards incompatible changes may occur if deemed absolutely necessary by the core developers. I'd suggest including a clearer motivation there: Note: The asyncio package has been included in the standard library on a provisional basis, and thus may gain new APIs and capabilities in maintenance releases as it matures. Backwards incompatible changes may occur if deemed absolutely necessary by the core developers. It's a gentler phrasing that still serves to warn away folks that are particularly change averse, while also assuring folks on faster iteration cycles that the update cadence is still 6 months rather than 18-24 months. New standard library APIs could thus evolve through not stable yet - mostly stable - stable. This is essentially Yury's original two levels of provisional idea - if a package survives an entire release cycle, it's pretty clear we're not going to be taking it out, but it's also the case that a comparatively broad API like asyncio may take a couple of feature release cycles to settle down to a point where we can declare it sufficiently complete that we're only going to add new interfaces in future feature releases. asyncio's just the first addition we've made under the PEP 411 guidelines that's big enough to benefit from the extra release cycle of stabilisation. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On Thu, 27 Aug 2015 at 14:16 Yury Selivanov yselivanov...@gmail.com wrote: Recently, in an asyncio related issue [1], Guido said that new features for asyncio have to wait till 3.6, since asyncio is no longer a provisional package. Later, in an off-list conversation, he suggested that this topic should be discussed on python-dev, and that it might indeed make sense to either write a new PEP for cases like this or to augment PEP 411. My opinion on this topic is that we must maintain full backwards compatibility for asyncio from 3.5.0, as it is now widely used, and there is quite a big ecosystem around it. However, asyncio is simply not mature enough to be completely feature frozen for about 2 years. For example, there is an issue [2] to add starttls support to asyncio. It is an essential feature, because some protocols start as clear text and upgrade to TLS later, for example PostgreSQL PQ3 protocol. It's very hard to implement starttls on top of asyncio, lots of code will have to be duplicated -- it's a feature that has to implemented in the asyncio core. Aside from adding new APIs, we also have to improve debugging capabilities. One example is using os.fork() from within a running event loop -- it must be avoided by all means. There are safe ways to fork in asyncio applications (and I plan to document them soon), but asyncio should raise an exception in debug mode if this happens (see issue [3]). These are just two immediate issues that I have in mind. In reality, asyncio is quite young compared to frameworks like Twisted, which had years to mature, and accumulate essential features. My proposal is to amend PEP 411 with two levels of provisional packages: Level 1: Backwards incompatible changes might be introduced in point releases. Level 2: Only backwards compatible changes can be introduced in new point releases. How is this any different from the normal compatibility promise we have for any non-provisional code in the stdlib? And by point release I assume you mean a new minor release, e.g. 3.5 - 3.6. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
Maybe asyncio should just be kept provisional during 3.5, with a separate promise to remain backward compatible? On Thu, Aug 27, 2015 at 3:24 PM, Yury Selivanov yselivanov...@gmail.com wrote: On 2015-08-27 5:53 PM, Brett Cannon wrote: Considering that Python versioning is defined as major.minor.micro, I'll rephrase the proposal: Level 1: Backwards incompatible changes might be introduced in new Python releases (including micro releases) Level 2: Only backwards compatible changes (new APIs including) can be introduced in micro releases. In that case I don't think it's a good idea for something that has widespread use to get new APIs in a micro release; I lived the 2.2.1/boolean event and I don't want to go through that again. If a module is used enough to warrant not breaking backwards-compatibility then it warrants not being provisional and being like any other module. I wasn't using Python 2.2/2.3, but from what I could google the 2.2.1/boolean event you mention was introducing True/False/bool built-ins. This sounds like a language-level change, as opposed to new API in a stdlib module, which is a different scale. My understanding about adding new features in bugfix releases (3.5.x) is that you might end up in a situation where your 3.5 code developed on 3.5.x suddenly stops working on 3.5.y. Yes, you have to be careful about how you deploy and test your code when using a provisional package. But the thing about asyncio is that it *is* still provisional in 3.4. During 3.4 release cycle we introduced many new features to asyncio, and to be honest, I haven't heard anybody complaining. I believe that main motivation for making asyncio non-provisional was to guarantee that we won't introduce backwards-incompatible changes to it. Given the fact that asyncio sees some adoption, I support that from now on we will guarantee that backwards compatibility is preserved. But withholding new useful (and sometimes essential) features till 3.6.0 is out (March 2017?) sounds wrong to me. I should also mention that asyncio is different from other packages in the stdlib: 1. It's new, it virtually didn't exist before 3.4.0. 2. It's not a module, it's a framework. If it lacks a core feature (like starttls) that is hard to implement as an add-on, you're basically forced either copy/paste a lot of code or to fork asyncio. And if you fork it, how will your dependencies can be upgraded to use that fork? I want to continue (as we did in 3.4.x releases) evolving asyncio on a faster scale than CPython currently evolves, *and* to guarantee that we won't break existing code. That's why I propose to tweak our definition of provisional packages. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On Thu, 27 Aug 2015 at 15:24 Yury Selivanov yselivanov...@gmail.com wrote: On 2015-08-27 5:53 PM, Brett Cannon wrote: Considering that Python versioning is defined as major.minor.micro, I'll rephrase the proposal: Level 1: Backwards incompatible changes might be introduced in new Python releases (including micro releases) Level 2: Only backwards compatible changes (new APIs including) can be introduced in micro releases. In that case I don't think it's a good idea for something that has widespread use to get new APIs in a micro release; I lived the 2.2.1/boolean event and I don't want to go through that again. If a module is used enough to warrant not breaking backwards-compatibility then it warrants not being provisional and being like any other module. I wasn't using Python 2.2/2.3, but from what I could google the 2.2.1/boolean event you mention was introducing True/False/bool built-ins. This sounds like a language-level change, as opposed to new API in a stdlib module, which is a different scale. True, but that doesn't mean that 3.5.1 will be able to run code that 3.5.0 has no chance of running because you introduced a new feature in asyncio. My understanding about adding new features in bugfix releases (3.5.x) is that you might end up in a situation where your 3.5 code developed on 3.5.x suddenly stops working on 3.5.y. Yes, you have to be careful about how you deploy and test your code when using a provisional package. But the thing about asyncio is that it *is* still provisional in 3.4. Right, but we're talking about 3.5 here, right? During 3.4 release cycle we introduced many new features to asyncio, and to be honest, I haven't heard anybody complaining. I believe that main motivation for making asyncio non-provisional was to guarantee that we won't introduce backwards-incompatible changes to it. Given the fact that asyncio sees some adoption, I support that from now on we will guarantee that backwards compatibility is preserved. But withholding new useful (and sometimes essential) features till 3.6.0 is out (March 2017?) sounds wrong to me. I should also mention that asyncio is different from other packages in the stdlib: 1. It's new, it virtually didn't exist before 3.4.0. Sure, but now it does. You said yourself that asyncio is seeing adoption, so exists now for at least some people. 2. It's not a module, it's a framework. But it's still in the stdlib and has already gone through one release as provisional. If it lacks a core feature (like starttls) that is hard to implement as an add-on, you're basically forced either copy/paste a lot of code or to fork asyncio. And if you fork it, how will your dependencies can be upgraded to use that fork? I'm not forking so that's not my problem. =) I want to continue (as we did in 3.4.x releases) evolving asyncio on a faster scale than CPython currently evolves, *and* to guarantee that we won't break existing code. That's why I propose to tweak our definition of provisional packages. I still don't like it. I say it's either fully provisional or it's not. I'm fine with extending its provisional status another feature release as long as it clearly states that in What's New for 3.5, but I don't think this granularity guarantee of not breaking APIs while adding new features is worth it. What if you want to add a new feature that is really hard to do right without breaking compatibility? We all know how trying that is. If you truly want to keep an accelerated development cycle, then short of releasing new stdlib versions every 6 months separate from the language then I say keep it provisional for 3.5. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
On Thu, 27 Aug 2015 at 15:44 Guido van Rossum gu...@python.org wrote: Maybe asyncio should just be kept provisional during 3.5, with a separate promise to remain backward compatible? My worry is that promising backwards-compatibility while still trying to change things is going to lead to needlessly hampering the evolution of the framework. If you want to say you will try hard to not break code and stay provisional I'm all for that (although I assume we already do that anyway), but making our standard backwards-compatibility promise on top of new features just seems too messy to me without knowing where the new features will take you. Past that it seems like asyncio is cheating on our bugfix release compatibility promises because it was lucky enough to be added as provisional. I realize asyncio went in like it did partially to motivate `yield from` and now async/await, and that's fine. But I say go all-in and just stay provisional another release and then lock it down in 3.6. -Brett On Thu, Aug 27, 2015 at 3:24 PM, Yury Selivanov yselivanov...@gmail.com wrote: On 2015-08-27 5:53 PM, Brett Cannon wrote: Considering that Python versioning is defined as major.minor.micro, I'll rephrase the proposal: Level 1: Backwards incompatible changes might be introduced in new Python releases (including micro releases) Level 2: Only backwards compatible changes (new APIs including) can be introduced in micro releases. In that case I don't think it's a good idea for something that has widespread use to get new APIs in a micro release; I lived the 2.2.1/boolean event and I don't want to go through that again. If a module is used enough to warrant not breaking backwards-compatibility then it warrants not being provisional and being like any other module. I wasn't using Python 2.2/2.3, but from what I could google the 2.2.1/boolean event you mention was introducing True/False/bool built-ins. This sounds like a language-level change, as opposed to new API in a stdlib module, which is a different scale. My understanding about adding new features in bugfix releases (3.5.x) is that you might end up in a situation where your 3.5 code developed on 3.5.x suddenly stops working on 3.5.y. Yes, you have to be careful about how you deploy and test your code when using a provisional package. But the thing about asyncio is that it *is* still provisional in 3.4. During 3.4 release cycle we introduced many new features to asyncio, and to be honest, I haven't heard anybody complaining. I believe that main motivation for making asyncio non-provisional was to guarantee that we won't introduce backwards-incompatible changes to it. Given the fact that asyncio sees some adoption, I support that from now on we will guarantee that backwards compatibility is preserved. But withholding new useful (and sometimes essential) features till 3.6.0 is out (March 2017?) sounds wrong to me. I should also mention that asyncio is different from other packages in the stdlib: 1. It's new, it virtually didn't exist before 3.4.0. 2. It's not a module, it's a framework. If it lacks a core feature (like starttls) that is hard to implement as an add-on, you're basically forced either copy/paste a lot of code or to fork asyncio. And if you fork it, how will your dependencies can be upgraded to use that fork? I want to continue (as we did in 3.4.x releases) evolving asyncio on a faster scale than CPython currently evolves, *and* to guarantee that we won't break existing code. That's why I propose to tweak our definition of provisional packages. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] provisional status for asyncio
Brett, On 2015-08-27 6:46 PM, Brett Cannon wrote: [...] I say it's either fully provisional or it's not. I'm fine with extending its provisional status another feature release as long as it clearly states that in What's New for 3.5, but I don't think this granularity guarantee of not breaking APIs while adding new features is worth it. What if you want to add a new feature that is really hard to do right without breaking compatibility? We all know how trying that is. If you truly want to keep an accelerated development cycle, then short of releasing new stdlib versions every 6 months separate from the language then I say keep it provisional for 3.5. I'm fine with keeping it provisional in 3.5 (and Guido suggests this idea too in this thread). A lot of companies (including big ones) are using asyncio already, despite the fact that it's provisional in 3.4. I seriously doubt that keeping it provisional in 3.5 will do any harm. asyncio documentation in 3.4.x has the following notes section: Note: The asyncio package has been included in the standard library on a provisional basis. Backwards incompatible changes (up to and including removal of the module) may occur if deemed necessary by the core developers. I suggest to add a slightly less strong-worded note to 3.5 documentation: Note: The asyncio package has been included in the standard library on a provisional basis. Backwards incompatible changes may occur if deemed absolutely necessary by the core developers. Yury ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
