2017-11-12 3:40 GMT-08:00 Ethan Smith <et...@ethanhs.me>: > Hello, > > I re-wrote my PEP to have typing opt-in be per-package rather than > per-distribution. This greatly simplifies things, and thanks to the > feedback and suggestions of Nick Coghlan, it is entirely compatible with > older packaging tooling. > > The main idea is there are two types of packages: > - types are packaged with runtime code (inline or stubs in the same > package) > - types are in a separate package (a third party or maintainer wants to > ship type information, but not with runtime code). > > The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/ > > And as always, duplicated below. > > Cheers, > > Ethan Smith > > --------------------------------------------------- > > PEP: 561 > Title: Distributing and Packaging Type Information > Author: Ethan Smith <et...@ethanhs.me> > Status: Draft > Type: Standards Track > Content-Type: text/x-rst > Created: 09-Sep-2017 > Python-Version: 3.7 > Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017 > > > Abstract > ======== > > PEP 484 introduced type hinting to Python, with goals of making typing > gradual and easy to adopt. Currently, typing information must be distributed > manually. This PEP provides a standardized means to leverage existing tooling > to package and distribute type information with minimal work and an ordering > for type checkers to resolve modules and collect this information for type > checking. > > > Rationale > ========= > > Currently, package authors wish to distribute code that has inline type > information. Additionally, maintainers would like to distribute stub files > to keep Python 2 compatibility while using newer annotation syntax. However, > there is no standard method to distribute packages with type information. > Also, if one wished to ship stub files privately the only method available > would be via setting ``MYPYPATH`` or the equivalent to manually point to > stubs. If the package can be released publicly, it can be added to > typeshed [1]_. However, this does not scale and becomes a burden on the > maintainers of typeshed. In addition, it ties bug fixes in stubs to releases > of the tool using typeshed. > > PEP 484 has a brief section on distributing typing information. In this > section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for > shipping stub files. However, manually adding a path to stub files for each > third party library does not scale. The simplest approach people have taken > is to add ``site-packages`` to their ``MYPYPATH``, but this causes type > checkers to fail on packages that are highly dynamic (e.g. sqlalchemy > and Django). > > > Definition of Terms > =================== > > The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are > to be interpreted as described in RFC 2119. > > "inline" - the types are part of the runtime code using PEP 526 and 3107 > syntax. > > "stubs" - files containing only type information, empty of runtime code. > > "Distributions" are the packaged files which are used to publish and > distribute > a release. [3]_ > > "Module" a file containing Python runtime code or stubbed type information. > > "Package" a directory or directories that namespace Python modules. > > > Specification > ============= > > There are several motivations and methods of supporting typing in a package. > This PEP recognizes three (3) types of packages that users of typing wish to > create: > > 1. The package maintainer would like to add type information inline. > > 2. The package maintainer would like to add type information via stubs. > > 3. A third party or package maintainer would like to share stub files for > a package, but the maintainer does not want to include them in the source > of the package. > > This PEP aims to support these scenarios and make them simple to add to > packaging and deployment. > > The two major parts of this specification are the packaging specifications > and the resolution order for resolving module type information. The type > checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec > of PEP 484 [2]_. > > New third party stub libraries SHOULD distribute stubs via the third party > packaging methods proposed in this PEP in place of being added to typeshed. > Typeshed will remain in use, but if maintainers are found, third party stubs > in typeshed MAY be split into their own package. > > > Packaging Type Information > -------------------------- > > In order to make packaging and distributing type information as simple and > easy as possible, packaging and distribution is done through existing > frameworks. > > Package maintainers who wish to support type checking of their code MUST add > a ``py.typed`` file to their package supporting typing. This marker is > recursive, if a top-level package includes it, all sub-packages MUST support > type checking as well. To have this file installed with the package, > maintainers can use existing packaging options such as ``package_data`` in > distutils, shown below. > > Distutils option example:: > > ... > package_data = { > 'pkg': ['py.typed'], > }, > ... > > For namespace packages, the ``py.typed`` file should be in the submodules of > the namespace, to avoid conflicts and for clarity. > > Stub Only Packages > '''''''''''''''''' > > For package maintainers wishing to ship stub files containing all of their > type information, it is preferred that the ``*.pyi`` stubs are alongside the > corresponding ``*.py`` files. However, the stubs can also be put in a separate > package and distributed separately. Third parties can also find this method > useful if they wish to distribute stub files. The name of the stub package > MUST follow the scheme ``pkg_stubs`` for type stubs for the package named > ``pkg``. The normal resolution order of checking ``*.pyi`` before ``*.py`` > will be maintained. > > This is very minor, but what do you think of using "pkg-stubs" instead of "pkg_stubs" (using a hyphen rather than an underscore)? This would make the name illegal to import as a normal Python package, which makes it clear that it's not a normal package. Also, there could be real packages named "_stubs".
> > Third parties seeking to distribute stub files are encouraged to contact the > maintainer of the package about distribution alongside the package. If the > maintainer does not wish to maintain or package stub files or type information > inline, then a third party stub only package can be created. > > In addition, stub only distributions SHOULD indicate which version(s) > of the runtime package are supported by indicating the runtime distribution's > version(s) through normal dependency data. For example, if there was a > stub package ``flyingcircus_stubs``, it can indicate the versions of the > runtime ``Flyingcircus`` distribution supported through ``install_requires`` > in distutils based tools, or the equivalent in other packaging tools. > > > Type Checker Module Resolution Order > ------------------------------------ > > The following is the order that type checkers supporting this PEP SHOULD > resolve modules containing type information: > > 1. User code - the files the type checker is running on. > > 2. Stubs or Python source manually put in the beginning of the path. Type > checkers SHOULD provide this to allow the user complete control of which > stubs to use, and patch broken stubs/inline types from packages. > > 3. Stub packages - these packages can supersede the installed packages. > They can be found at ``pkg_stubs`` for package ``pkg``. > > 4. Inline packages - if there is nothing overriding the installed > package, and it opts into type checking, inline types SHOULD be used. > > 5. Typeshed (if used) - Provides the stdlib types and several third party > libraries. > > Type checkers that check a different Python version than the version they run > on MUST find the type information in the ``site-packages``/``dist-packages`` > of that Python version. This can be queried e.g. > ``pythonX.Y -c 'import site; print(site.getsitepackages())'``. It is also > recommended > that the type checker allow for the user to point to a particular Python > binary, in case it is not in the path. > > > Implementation > ============== > > The proposed scheme of indicating support for typing is completely backwards > compatible, and requires no modification to tooling. A sample package with > inline types is available [typed_pkg]_, as well as a sample package checker > [pkg_checker]_ which reads the metadata of installed packages and reports on > their status as either not typed, inline typed, or a stub package. > > > Acknowledgements > ================ > > This PEP would not have been possible without the ideas, feedback, and support > of Ivan Levkivskyi, Jelle Zijlstra, Nick Coghlan, Daniel F Moisset, Nathaniel > Smith, and Guido van Rossum. > > > Version History > =============== > > * 2017-11-12 > > * Rewritten to use existing tooling only > * No need to indicate kind of type information in metadata > * Name of marker file changed from ``.typeinfo`` to ``py.typed`` > > * 2017-11-10 > > * Specification re-written to use package metadata instead of distribution > metadata. > * Removed stub only packages and merged into third party packages spec. > * Removed suggestion for typecheckers to consider checking runtime > versions > * Implementations updated to reflect PEP changes. > > * 2017-10-26 > > * Added implementation references. > * Added acknowledgements and version history. > > * 2017-10-06 > > * Rewritten to use .distinfo/METADATA over a distutils specific command. > * Clarify versioning of third party stub packages. > > * 2017-09-11 > > * Added information about current solutions and typeshed. > * Clarify rationale. > > > References > ========== > .. [1] Typeshed (https://github.com/python/typeshed) > > .. [2] PEP 484, Storing and Distributing Stub Files > > (https://www.python.org/dev/peps/pep-0484/#storing-and-distributing-stub-files) > > .. [3] PEP 426 definitions > (https://www.python.org/dev/peps/pep-0426/) > > .. [typed_pkg] Sample typed package > (https://github.com/ethanhs/sample-typed-package) > > .. [pkg_checker] Sample package checker > (https://github.com/ethanhs/check_typedpkg) > > Copyright > ========= > > This document has been placed in the public domain. > > > > .. > Local Variables: > mode: indented-text > indent-tabs-mode: nil > sentence-end-double-space: t > fill-column: 70 > coding: utf-8 > End: > > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > jelle.zijlstra%40gmail.com > >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
