Re: [Python-Dev] PEP 561 rework

2017-11-14 Thread Ethan Smith 
A note was added [1] about the solution for module only distributions and
is live on Python.org.

[1] https://github.com/python/peps/pull/468

Ethan Smith

On Tue, Nov 14, 2017 at 1:02 AM, Sebastian Rittau 
wrote:

> Am 14.11.2017 um 02:38 schrieb Guido van Rossum:
>
> On Mon, Nov 13, 2017 at 3:50 PM, Sebastian Rittau 
> wrote:
>
>>
>
>> I am really looking forward to the implementation of this PEP and I am
>> glad that it is close to acceptance. One thing that is not really clear to
>> me is how module-only packages are handled. Say I have a package "foo" that
>> installs the file "foo.py" to site-packages, where would I install
>> "foo.pyi" and py.typed to? Or is this case not supported and I have to
>> convert the foo module into a package containing just __init__.py?
>>
>
> Good call. I think that conversion to a package is indeed the best
> approach -- it doesn't seem worth it to add more special-casing for this
> scenario.
>
> Ethan, if you agree, you should just add a sentence about this to the PEP.
>
> This seems like the best solution, especially since setuptools does not
> really support installing package data for pure modules.
>
>  - Sebastian
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> ethan%40ethanhs.me
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-14 Thread Sebastian Rittau 

Am 14.11.2017 um 02:38 schrieb Guido van Rossum:
On Mon, Nov 13, 2017 at 3:50 PM, Sebastian Rittau > wrote:


I am really looking forward to the implementation of this PEP and
I am glad that it is close to acceptance. One thing that is not
really clear to me is how module-only packages are handled. Say I
have a package "foo" that installs the file "foo.py" to
site-packages, where would I install "foo.pyi" and py.typed to? Or
is this case not supported and I have to convert the foo module
into a package containing just __init__.py?


Good call. I think that conversion to a package is indeed the best 
approach -- it doesn't seem worth it to add more special-casing for 
this scenario.


Ethan, if you agree, you should just add a sentence about this to the PEP.

This seems like the best solution, especially since setuptools does not 
really support installing package data for pure modules.


 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-13 Thread Ethan Smith 
On Mon, Nov 13, 2017 at 3:50 PM, Sebastian Rittau 
wrote:

> Hello everyone,
>
>
> Am 14.11.2017 um 00:29 schrieb Guido van Rossum:
>
>> This is a nice piece of work. I expect to accept it pretty much verbatim
>> (with some small edits, see https://github.com/python/peps/pull/467). I
>> agree with Nick that we don't have to do anything specifically about
>> control of foo_stubs packages -- nor do I think we need to worry about
>> foo_stubs vs. foo-stubs.
>>
>> Everyone else: if you think this should not go through, now's the time to
>> reply-all here!
>>
> I am really looking forward to the implementation of this PEP and I am
> glad that it is close to acceptance. One thing that is not really clear to
> me is how module-only packages are handled. Say I have a package "foo" that
> installs the file "foo.py" to site-packages, where would I install
> "foo.pyi" and py.typed to? Or is this case not supported and I have to
> convert the foo module into a package containing just __init__.py?
>
> The PEP as of right now provides no support for module only distributions.
I don't think it would be possible to support inline typing in module only
distributions, as a random mymod.py in site/dist-packages is hard to trust.
Refactoring into a package is probably the best option. Alternatively, I
believe a mymod_stubs with just an __init__.pyi might work as a work around
as well.

Ethan


>  - Sebastian
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ethan%
> 40ethanhs.me
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-13 Thread Guido van Rossum 
On Mon, Nov 13, 2017 at 3:50 PM, Sebastian Rittau 
wrote:

> Am 14.11.2017 um 00:29 schrieb Guido van Rossum:
>
>> This is a nice piece of work. I expect to accept it pretty much verbatim
>> (with some small edits, see https://github.com/python/peps/pull/467). I
>> agree with Nick that we don't have to do anything specifically about
>> control of foo_stubs packages -- nor do I think we need to worry about
>> foo_stubs vs. foo-stubs.
>>
>> Everyone else: if you think this should not go through, now's the time to
>> reply-all here!
>>
>

> I am really looking forward to the implementation of this PEP and I am
> glad that it is close to acceptance. One thing that is not really clear to
> me is how module-only packages are handled. Say I have a package "foo" that
> installs the file "foo.py" to site-packages, where would I install
> "foo.pyi" and py.typed to? Or is this case not supported and I have to
> convert the foo module into a package containing just __init__.py?
>

Good call. I think that conversion to a package is indeed the best approach
-- it doesn't seem worth it to add more special-casing for this scenario.

Ethan, if you agree, you should just add a sentence about this to the PEP.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-13 Thread Sebastian Rittau 

Hello everyone,


Am 14.11.2017 um 00:29 schrieb Guido van Rossum:
This is a nice piece of work. I expect to accept it pretty much 
verbatim (with some small edits, see 
https://github.com/python/peps/pull/467). I agree with Nick that we 
don't have to do anything specifically about control of foo_stubs 
packages -- nor do I think we need to worry about foo_stubs vs. foo-stubs.


Everyone else: if you think this should not go through, now's the time 
to reply-all here!
I am really looking forward to the implementation of this PEP and I am 
glad that it is close to acceptance. One thing that is not really clear 
to me is how module-only packages are handled. Say I have a package 
"foo" that installs the file "foo.py" to site-packages, where would I 
install "foo.pyi" and py.typed to? Or is this case not supported and I 
have to convert the foo module into a package containing just __init__.py?


 - Sebastian
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-13 Thread Guido van Rossum 
Hi Ethan!

This is a nice piece of work. I expect to accept it pretty much verbatim
(with some small edits, see https://github.com/python/peps/pull/467). I
agree with Nick that we don't have to do anything specifically about
control of foo_stubs packages -- nor do I think we need to worry about
foo_stubs vs. foo-stubs.

Everyone else: if you think this should not go through, now's the time to
reply-all here!

--Guido

On Mon, Nov 13, 2017 at 7:58 AM, Ivan Levkivskyi 
wrote:

> Thanks Ethan for all the work!
>
> I will be glad to see this accepted and implemented in mypy.
>
> --
> Ivan
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-13 Thread Ivan Levkivskyi 
Thanks Ethan for all the work!

I will be glad to see this accepted and implemented in mypy.

--
Ivan
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-13 Thread Nick Coghlan 
On 13 November 2017 at 17:33, Ethan Smith  wrote:
> On Sun, Nov 12, 2017 at 8:07 PM, Nathaniel Smith  wrote:
>> PyPI doesn't distinguish between the names 'foo-stubs' and 'foo_stubs'
>> -- they get normalized together. So even if you use 'foo-stubs' as the
>> directory name on sys.path to avoid collisions at import time, it
>> still won't allow someone to distribute a separate 'foo_stubs' package
>> on PyPI.
>>
>> If you do go with a fixed naming convention like this, the PEP should
>> probably also instruct the PyPI maintainers that whoever owns 'foo'
>> automatically has the right to control the name 'foo-stubs' as well.
>> Or maybe some tweak to PEP 541 is needed.
>
> As I understand it however, the distribution name need not map to to the
> package name in any way. So regardless of if foo-stubs is seen as foo_stubs,
> I could name the distribution Albatross if I wished, and install the
> foo-stubs package into site/dist-packages, and it would work. Also I'm not
> sure if the PyPI change would require an edict from a PEP, but if so, I
> wouldn't be opposed to the idea, I think it would be nice to default the
> stub packages to the owners of the normal packages (people should, to my
> understanding, be able to make alternate distributions without hassle).

Questions like the following aren't new ones:

- If I am responsible for the name 'foo' on PyPI, how much influence,
if any, should I have over the use of 'foo' as a prefix in other
distribution package names?
- If I ship a distribution package through PyPI containing an import
package named 'bar', how much influence, if any, should I have over
the use of 'bar' as an import package or module name in other
distribution packages?

I expect that the PSF will need to address them directly some day, but
I don't think PEP 561 itself needs to address them (and the first
version of PEP 541 probably won't either).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-12 Thread Ethan Smith 
On Sun, Nov 12, 2017 at 8:07 PM, Nathaniel Smith  wrote:

> On Sun, Nov 12, 2017 at 11:21 AM, Ethan Smith  wrote:
> >
> >
> > On Sun, Nov 12, 2017 at 9:53 AM, Jelle Zijlstra <
> jelle.zijls...@gmail.com>
> > wrote:
> >>
> >> 2017-11-12 3:40 GMT-08:00 Ethan Smith :
> >>> The name of the stub
> >>> package
> >>> MUST follow the scheme ``pkg_stubs`` for type stubs for the package
> named
> >>> ``pkg``. The normal resolution order of checking ``*.pyi`` before
> >>> ``*.py``
> >>> will be maintained.
> >>
> >> This is very minor, but what do you think of using "pkg-stubs" instead
> of
> >> "pkg_stubs" (using a hyphen rather than an underscore)? This would make
> the
> >> name illegal to import as a normal Python package, which makes it clear
> that
> >> it's not a normal package. Also, there could be real packages named
> >> "_stubs".
> >
> > I suppose this makes sense. I checked PyPI and as of a few weeks ago
> there
> > were no packages with the name pattern, but I like the idea of making it
> > explicitly non-runtime importable. I cannot think of any reason not to do
> > it, and the avoidance of confusion about the package being importable is
> a
> > benefit. I will make the change with my next round of edits.
>
> PyPI doesn't distinguish between the names 'foo-stubs' and 'foo_stubs'
> -- they get normalized together. So even if you use 'foo-stubs' as the
> directory name on sys.path to avoid collisions at import time, it
> still won't allow someone to distribute a separate 'foo_stubs' package
> on PyPI.
>
> If you do go with a fixed naming convention like this, the PEP should
> probably also instruct the PyPI maintainers that whoever owns 'foo'
> automatically has the right to control the name 'foo-stubs' as well.
> Or maybe some tweak to PEP 541 is needed.
>

As I understand it however, the distribution name need not map to to the
package name in any way. So regardless of if foo-stubs is seen as
foo_stubs, I could name the distribution Albatross if I wished, and install
the foo-stubs package into site/dist-packages, and it would work. Also I'm
not sure if the PyPI change would require an edict from a PEP, but if so, I
wouldn't be opposed to the idea, I think it would be nice to default the
stub packages to the owners of the normal packages (people should, to my
understanding, be able to make alternate distributions without hassle).

Ethan

>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-12 Thread Nathaniel Smith 
On Sun, Nov 12, 2017 at 11:21 AM, Ethan Smith  wrote:
>
>
> On Sun, Nov 12, 2017 at 9:53 AM, Jelle Zijlstra 
> wrote:
>>
>> 2017-11-12 3:40 GMT-08:00 Ethan Smith :
>>> The name of the stub
>>> package
>>> MUST follow the scheme ``pkg_stubs`` for type stubs for the package named
>>> ``pkg``. The normal resolution order of checking ``*.pyi`` before
>>> ``*.py``
>>> will be maintained.
>>
>> This is very minor, but what do you think of using "pkg-stubs" instead of
>> "pkg_stubs" (using a hyphen rather than an underscore)? This would make the
>> name illegal to import as a normal Python package, which makes it clear that
>> it's not a normal package. Also, there could be real packages named
>> "_stubs".
>
> I suppose this makes sense. I checked PyPI and as of a few weeks ago there
> were no packages with the name pattern, but I like the idea of making it
> explicitly non-runtime importable. I cannot think of any reason not to do
> it, and the avoidance of confusion about the package being importable is a
> benefit. I will make the change with my next round of edits.

PyPI doesn't distinguish between the names 'foo-stubs' and 'foo_stubs'
-- they get normalized together. So even if you use 'foo-stubs' as the
directory name on sys.path to avoid collisions at import time, it
still won't allow someone to distribute a separate 'foo_stubs' package
on PyPI.

If you do go with a fixed naming convention like this, the PEP should
probably also instruct the PyPI maintainers that whoever owns 'foo'
automatically has the right to control the name 'foo-stubs' as well.
Or maybe some tweak to PEP 541 is needed.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

2017-11-12 Thread Ethan Smith 
On Sun, Nov 12, 2017 at 9:53 AM, Jelle Zijlstra 
wrote:

>
>
> 2017-11-12 3:40 GMT-08:00 Ethan Smith :
>
>> Hello,
>>
>> I re-wrote my PEP to have typing opt-in be per-package rather than
>> per-distribution. This greatly simplifies things, and thanks to the
>> feedback and suggestions of Nick Coghlan, it is entirely compatible with
>> older packaging tooling.
>>
>> The main idea is there are two types of packages:
>>  - types are packaged with runtime code (inline or stubs in the same
>> package)
>>  - types are in a separate package (a third party or maintainer wants to
>> ship type information, but not with runtime code).
>>
>> The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/
>>
>> And as always, duplicated below.
>>
>> Cheers,
>>
>> Ethan Smith
>>
>> ---
>>
>> PEP: 561
>> Title: Distributing and Packaging Type Information
>> Author: Ethan Smith 
>> Status: Draft
>> Type: Standards Track
>> Content-Type: text/x-rst
>> Created: 09-Sep-2017
>> Python-Version: 3.7
>> Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017
>>
>>
>> Abstract
>> 
>>
>> PEP 484 introduced type hinting to Python, with goals of making typing
>> gradual and easy to adopt. Currently, typing information must be distributed
>> manually. This PEP provides a standardized means to leverage existing tooling
>> to package and distribute type information with minimal work and an ordering
>> for type checkers to resolve modules and collect this information for type
>> checking.
>>
>>
>> Rationale
>> =
>>
>> Currently, package authors wish to distribute code that has inline type
>> information. Additionally, maintainers would like to distribute stub files
>> to keep Python 2 compatibility while using newer annotation syntax. However,
>> there is no standard method to distribute packages with type information.
>> Also, if one wished to ship stub files privately the only method available
>> would be via setting ``MYPYPATH`` or the equivalent to manually point to
>> stubs. If the package can be released publicly, it can be added to
>> typeshed [1]_. However, this does not scale and becomes a burden on the
>> maintainers of typeshed. In addition, it ties bug fixes in stubs to releases
>> of the tool using typeshed.
>>
>> PEP 484 has a brief section on distributing typing information. In this
>> section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
>> shipping stub files. However, manually adding a path to stub files for each
>> third party library does not scale. The simplest approach people have taken
>> is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
>> checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
>> and Django).
>>
>>
>> Definition of Terms
>> ===
>>
>> The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are
>> to be interpreted as described in RFC 2119.
>>
>> "inline" - the types are part of the runtime code using PEP 526 and 3107
>> syntax.
>>
>> "stubs" - files containing only type information, empty of runtime code.
>>
>> "Distributions" are the packaged files which are used to publish and 
>> distribute
>> a release. [3]_
>>
>> "Module" a file containing Python runtime code or stubbed type information.
>>
>> "Package" a directory or directories that namespace Python modules.
>>
>>
>> Specification
>> =
>>
>> There are several motivations and methods of supporting typing in a package.
>> This PEP recognizes three (3) types of packages that users of typing wish to
>> create:
>>
>> 1. The package maintainer would like to add type information inline.
>>
>> 2. The package maintainer would like to add type information via stubs.
>>
>> 3. A third party or package maintainer would like to share stub files for
>>a package, but the maintainer does not want to include them in the source
>>of the package.
>>
>> This PEP aims to support these scenarios and make them simple to add to
>> packaging and deployment.
>>
>> The two major parts of this specification are the packaging specifications
>> and the resolution order for resolving module type information. The type
>> checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec
>> of PEP 484 [2]_.
>>
>> New third party stub libraries SHOULD distribute stubs via the third party
>> packaging methods proposed in this PEP in place of being added to typeshed.
>> Typeshed will remain in use, but if maintainers are found, third party stubs
>> in typeshed MAY be split into their own package.
>>
>>
>> Packaging Type Information
>> --
>>
>> In order to make packaging and distributing type information as simple and
>> easy as possible, packaging and distribution is done through existing
>> frameworks.
>>
>> Package maintainers who wish to support type checking of their code MUST add
>> a ``py.typed``

Re: [Python-Dev] PEP 561 rework

2017-11-12 Thread Jelle Zijlstra 
2017-11-12 3:40 GMT-08:00 Ethan Smith :

> Hello,
>
> I re-wrote my PEP to have typing opt-in be per-package rather than
> per-distribution. This greatly simplifies things, and thanks to the
> feedback and suggestions of Nick Coghlan, it is entirely compatible with
> older packaging tooling.
>
> The main idea is there are two types of packages:
>  - types are packaged with runtime code (inline or stubs in the same
> package)
>  - types are in a separate package (a third party or maintainer wants to
> ship type information, but not with runtime code).
>
> The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/
>
> And as always, duplicated below.
>
> Cheers,
>
> Ethan Smith
>
> ---
>
> PEP: 561
> Title: Distributing and Packaging Type Information
> Author: Ethan Smith 
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 09-Sep-2017
> Python-Version: 3.7
> Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017
>
>
> Abstract
> 
>
> PEP 484 introduced type hinting to Python, with goals of making typing
> gradual and easy to adopt. Currently, typing information must be distributed
> manually. This PEP provides a standardized means to leverage existing tooling
> to package and distribute type information with minimal work and an ordering
> for type checkers to resolve modules and collect this information for type
> checking.
>
>
> Rationale
> =
>
> Currently, package authors wish to distribute code that has inline type
> information. Additionally, maintainers would like to distribute stub files
> to keep Python 2 compatibility while using newer annotation syntax. However,
> there is no standard method to distribute packages with type information.
> Also, if one wished to ship stub files privately the only method available
> would be via setting ``MYPYPATH`` or the equivalent to manually point to
> stubs. If the package can be released publicly, it can be added to
> typeshed [1]_. However, this does not scale and becomes a burden on the
> maintainers of typeshed. In addition, it ties bug fixes in stubs to releases
> of the tool using typeshed.
>
> PEP 484 has a brief section on distributing typing information. In this
> section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
> shipping stub files. However, manually adding a path to stub files for each
> third party library does not scale. The simplest approach people have taken
> is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
> checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
> and Django).
>
>
> Definition of Terms
> ===
>
> The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are
> to be interpreted as described in RFC 2119.
>
> "inline" - the types are part of the runtime code using PEP 526 and 3107
> syntax.
>
> "stubs" - files containing only type information, empty of runtime code.
>
> "Distributions" are the packaged files which are used to publish and 
> distribute
> a release. [3]_
>
> "Module" a file containing Python runtime code or stubbed type information.
>
> "Package" a directory or directories that namespace Python modules.
>
>
> Specification
> =
>
> There are several motivations and methods of supporting typing in a package.
> This PEP recognizes three (3) types of packages that users of typing wish to
> create:
>
> 1. The package maintainer would like to add type information inline.
>
> 2. The package maintainer would like to add type information via stubs.
>
> 3. A third party or package maintainer would like to share stub files for
>a package, but the maintainer does not want to include them in the source
>of the package.
>
> This PEP aims to support these scenarios and make them simple to add to
> packaging and deployment.
>
> The two major parts of this specification are the packaging specifications
> and the resolution order for resolving module type information. The type
> checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec
> of PEP 484 [2]_.
>
> New third party stub libraries SHOULD distribute stubs via the third party
> packaging methods proposed in this PEP in place of being added to typeshed.
> Typeshed will remain in use, but if maintainers are found, third party stubs
> in typeshed MAY be split into their own package.
>
>
> Packaging Type Information
> --
>
> In order to make packaging and distributing type information as simple and
> easy as possible, packaging and distribution is done through existing
> frameworks.
>
> Package maintainers who wish to support type checking of their code MUST add
> a ``py.typed`` file to their package supporting typing. This marker is
> recursive, if a top-level package includes it, all sub-packages MUST support
> type checking as well. To have this file installed with the package,
>

[Python-Dev] PEP 561 rework

2017-11-12 Thread Ethan Smith 
Hello,

I re-wrote my PEP to have typing opt-in be per-package rather than
per-distribution. This greatly simplifies things, and thanks to the
feedback and suggestions of Nick Coghlan, it is entirely compatible with
older packaging tooling.

The main idea is there are two types of packages:
 - types are packaged with runtime code (inline or stubs in the same
package)
 - types are in a separate package (a third party or maintainer wants to
ship type information, but not with runtime code).

The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/

And as always, duplicated below.

Cheers,

Ethan Smith

---

PEP: 561
Title: Distributing and Packaging Type Information
Author: Ethan Smith 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 09-Sep-2017
Python-Version: 3.7
Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017


Abstract


PEP 484 introduced type hinting to Python, with goals of making typing
gradual and easy to adopt. Currently, typing information must be distributed
manually. This PEP provides a standardized means to leverage existing tooling
to package and distribute type information with minimal work and an ordering
for type checkers to resolve modules and collect this information for type
checking.


Rationale
=

Currently, package authors wish to distribute code that has inline type
information. Additionally, maintainers would like to distribute stub files
to keep Python 2 compatibility while using newer annotation syntax. However,
there is no standard method to distribute packages with type information.
Also, if one wished to ship stub files privately the only method available
would be via setting ``MYPYPATH`` or the equivalent to manually point to
stubs. If the package can be released publicly, it can be added to
typeshed [1]_. However, this does not scale and becomes a burden on the
maintainers of typeshed. In addition, it ties bug fixes in stubs to releases
of the tool using typeshed.

PEP 484 has a brief section on distributing typing information. In this
section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
shipping stub files. However, manually adding a path to stub files for each
third party library does not scale. The simplest approach people have taken
is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
and Django).


Definition of Terms
===

The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are
to be interpreted as described in RFC 2119.

"inline" - the types are part of the runtime code using PEP 526 and 3107
syntax.

"stubs" - files containing only type information, empty of runtime code.

"Distributions" are the packaged files which are used to publish and distribute
a release. [3]_

"Module" a file containing Python runtime code or stubbed type information.

"Package" a directory or directories that namespace Python modules.


Specification
=

There are several motivations and methods of supporting typing in a package.
This PEP recognizes three (3) types of packages that users of typing wish to
create:

1. The package maintainer would like to add type information inline.

2. The package maintainer would like to add type information via stubs.

3. A third party or package maintainer would like to share stub files for
   a package, but the maintainer does not want to include them in the source
   of the package.

This PEP aims to support these scenarios and make them simple to add to
packaging and deployment.

The two major parts of this specification are the packaging specifications
and the resolution order for resolving module type information. The type
checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec
of PEP 484 [2]_.

New third party stub libraries SHOULD distribute stubs via the third party
packaging methods proposed in this PEP in place of being added to typeshed.
Typeshed will remain in use, but if maintainers are found, third party stubs
in typeshed MAY be split into their own package.


Packaging Type Information
--

In order to make packaging and distributing type information as simple and
easy as possible, packaging and distribution is done through existing
frameworks.

Package maintainers who wish to support type checking of their code MUST add
a ``py.typed`` file to their package supporting typing. This marker is
recursive, if a top-level package includes it, all sub-packages MUST support
type checking as well. To have this file installed with the package,
maintainers can use existing packaging options such as ``package_data`` in
distutils, shown below.

Distutils option example::

...
package_data = {
'pkg': ['py.typed'],
},
...

For namespace packages, the ``py.typed`` file should be in the submodules of
the