OK, go ahead and write the PEP! I'm actually happy with the responses you gave, so your last email will make a good start for some of the contents of the PEP.
On Tue, Oct 31, 2017 at 9:19 AM, Nick Coghlan <ncogh...@gmail.com> wrote: > On 1 November 2017 at 00:53, Guido van Rossum <gu...@python.org> wrote: > >> On Tue, Oct 31, 2017 at 4:42 AM, Nick Coghlan <ncogh...@gmail.com> wrote: >> >>> On 31 October 2017 at 02:29, Guido van Rossum <gu...@python.org> wrote: >>> >>>> What's your proposed process to arrive at the list of recommended >>>> packages? >>>> >>> >>> I'm thinking it makes the most sense to treat inclusion in the >>> recommended packages list as a possible outcome of proposals for standard >>> library inclusion, rather than being something we'd provide a way to >>> propose specifically. >>> >> >> I don't think that gets you off the hook for a process proposal. We need >> some criteria to explain why a module should be on the recommended list -- >> not just a ruling as to why it shouldn't be in the stdlib. >> > > The developer guide already has couple of sections on this aspect: > > * https://devguide.python.org/stdlibchanges/#acceptable-types-of-modules > * https://devguide.python.org/stdlibchanges/#requirements > > I don't think either of those sections is actually quite right (since > we've approved new modules that wouldn't meet them), but they're not > terrible as a starting point in general, and they're accurate for the > recommended packages use case. > > >> We'd only use it in cases where a proposal would otherwise meet the >>> criteria for stdlib inclusion, but the logistics of actually doing so don't >>> work for some reason. >>> >> >> But that would exclude most of the modules you mention below, since one >> of the criteria is that their development speed be matched with Python's >> release cycle. I think there must be some form of "popularity" combined >> with "best of breed". In particular I'd like to have a rule that explains >> why flask and Django would never make the list. (I don't know what that >> rule is, or I would tell you -- my gut tells me it's something to do with >> having their own community *and* competing for the same spot.) >> > > The developer guide words this as "The module needs to be considered > best-of-breed.". In some categories (like WSGI frameworks), there are > inherent trade-offs that mean there will *never* be a single best-of-breed > solution, since projects like Django, Flask, and Pyramid occupy > deliberately different points in the space of available design decisions. > > Running the initial 5 proposals through that filter: >>> >>> * six: a cross-version compatibility layer clearly needs to be outside >>> the standard library >>> >> >> Hm... Does six still change regularly? If not I think it *would* be a >> candidate for actual stdlib inclusion. Just like we added u"..." literals >> to Python 3.4. >> > > It still changes as folks porting new projects discover additional > discrepancies between the 2.x and 3.x standard library layouts and > behaviour (e.g. we found recently that the 3.x subprocess module's > emulation of the old commands module APIs actually bit shifts the status > codes relative to the 2.7 versions). The rate of change has definitely > slowed down a lot since the early days, but it isn't zero. > > In addition, the only folks that need it are those that already care about > older versions of Python - if you can start with whatever the latest > version of Python is, and don't have any reason to support users still > running older version, you can happily pretend those older versions don't > exist, and hence don't need a compatibility library like six. As a separate > library, it can just gracefully fade away as folks stop depending on it as > they drop Python 2.7 support. By contrast, if we were to bring it into the > 3.x standard library, then we'd eventually have to figure out when we could > deprecate and remove it again. > > >> * setuptools: we want to update this in line with the PyPA interop specs, >>> not the Python language version >>> >> >> But does that exclude stdlib inclusion? Why would those specs change, and >> why couldn't they wait for a new Python release? >> > > The specs mainly change when we want to offer publishers new capabilities > while still maintaining compatibility with older installation clients (and > vice-versa: we want folks still running Python 2.7 to be able to publish > wheel files and use recently added metadata fields like > Description-Content-Type). > > The reason we can't wait for new Python releases is because when we add > such things, we need them to work on *all* supported Python releases > (including 2.7 and security-release-only 3.x versions). > > There are also other drivers for setuptools updates, which include: > > - operating system build toolchain changes (e.g. finding new versions of > Visual Studio or XCode) > - changes to PyPI's operations (e.g. the legacy upload API getting turned > off due to persistent service stability problems, switching to HTTPS only > access) > > With setuptools as a separate project, a whole lot of package publication > problems can be solved via "pip install --upgrade setuptools wheel" in a > virtual environment, which is a luxury we don't have with plain distutils. > > >> * cffi: updates may be needed for PyPA interop specs, Python >> implementation updates or C language definition updates >> >> Hm, again, I don't recall that this was debated -- I think it's a failure >> that it's not in the stdlib. >> > > A couple of years ago I would have agreed with you, but I've spent enough > time on packaging problems now to realise that cffi actually qualifies as a > build tool due to the way it generates extension module wrappers when used > in "out-of-line" mode. > > Being outside the standard library means that cffi still has significant > flexibility to evolve how it separates its buildtime functionality from its > runtime functionality, and may eventually be adjusted so that only > "_cffi_backend" needs to be installed at runtime for the out-of-line > compilation mode, without the full C header parsing and inline extension > module compilation capabilities of CFFI itself (see > https://cffi.readthedocs.io/en/latest/cdef.html#preparing- > and-distributing-modules for the details). Being separate also means that > cffi can be updated to generate more efficient code even for existing > Python versions. > > >> * requests: updates are more likely to be driven by changes in network >>> protocols and client platform APIs than Python language changes >>> >> >> Here I agree. There's no alternative (except aiohttp, but that's >> asyncio-based) and it can't be in the stdlib because it's actively being >> developed. >> >> >>> * regex: we don't want two regex engines in the stdlib, transparently >>> replacing _sre would be difficult, and _sre is still good enough for most >>> purposes >>> >> >> I think this needn't be recommended at all. For 99.9% of regular >> expression uses, re is just fine. Let's just work on a strategy for >> introducing regex into the stdlib. >> > > Given your informational PEP suggestion below, I'd probably still include > it, but in a separate section from the others (e.g. the others might be > listed as "Misaligned Feature Release Cycles", which is an inherent > logistical problem that no amount of coding can fix, while regex would > instead be categorised as "Technical Challenges"). > > >> Of the 5, I'd suggest that regex is the only one that could potentially >>> still make its way into the standard library some day - it would just >>> require someone with both the time and inclination to create a CPython >>> variant that used _regex instead of _sre as the default regex engine, and >>> then gathered evidence to show that it was "compatible enough" with _sre to >>> serve as the default engine for CPython. >>> >>> For the first four, there are compelling arguments that their drivers >>> for new feature additions are such that their release cycles shouldn't ever >>> be tied to the rate at which we update the Python language definition. >>> >> >> As you can tell from my arguing, the reasons need to be written up in >> more detail. >> >> >>> And is it really just going to be a list of names, or is there going to >>>> be some documentation (about the vetting, not about the contents of the >>>> packages) for each name? >>>> >>> >>> I'm thinking a new subsection in https://docs.python.org/devgui >>> de/stdlibchanges.html for "Recommended Third Party Packages" would make >>> sense, covering what I wrote above. >>> >> >> That's too well hidden for my taste. >> >> >>> It also occurred to me that since the recommendations are independent of >>> the Python version, they don't really belong in the version specific >>> documentation. >>> >> >> But that doesn't mean they can't (also) be listed there. (And each >> probably has its version dependencies.) >> >> >>> While the Developer's Guide isn't really the right place for the list >>> either (except as an easier way to answer "Why isn't <X> in the standard >>> library?" questions), it could be a good interim option until I get around >>> to actually writing a first draft of https://github.com/python/redi >>> stributor-guide/ (which I was talking to Barry about at the dev sprint, >>> but didn't end up actually creating any content for since I went down a >>> signal handling rabbit hole instead). >>> >> >> Hm, let's not put more arbitrary check boxes in the way of progress. >> Maybe it can be an informational PEP that's occasionally updated? >> > > If I'm correctly reading that as "Could the list of Recommended Third > Party Packages be an informational PEP?", then I agree that's probably a > good way to tackle it, since it will cover both the developer-centric "Why > isn't this in the standard library yet?" aspect *and* the "Redistributors > should probably provide this" aspect. > > Cheers, > Nick. > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > -- --Guido van Rossum (python.org/~guido)
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
