Re: [Python-Dev] PEP 561 rework

[Ethan Smith]

On Sun, Nov 12, 2017 at 8:07 PM, Nathaniel Smith  wrote:

> On Sun, Nov 12, 2017 at 11:21 AM, Ethan Smith  wrote:
> >
> >
> > On Sun, Nov 12, 2017 at 9:53 AM, Jelle Zijlstra <
> jelle.zijls...@gmail.com>
> > wrote:
> >>
> >> 2017-11-12 3:40 GMT-08:00 Ethan Smith :
> >>> The name of the stub
> >>> package
> >>> MUST follow the scheme ``pkg_stubs`` for type stubs for the package
> named
> >>> ``pkg``. The normal resolution order of checking ``*.pyi`` before
> >>> ``*.py``
> >>> will be maintained.
> >>
> >> This is very minor, but what do you think of using "pkg-stubs" instead
> of
> >> "pkg_stubs" (using a hyphen rather than an underscore)? This would make
> the
> >> name illegal to import as a normal Python package, which makes it clear
> that
> >> it's not a normal package. Also, there could be real packages named
> >> "_stubs".
> >
> > I suppose this makes sense. I checked PyPI and as of a few weeks ago
> there
> > were no packages with the name pattern, but I like the idea of making it
> > explicitly non-runtime importable. I cannot think of any reason not to do
> > it, and the avoidance of confusion about the package being importable is
> a
> > benefit. I will make the change with my next round of edits.
>
> PyPI doesn't distinguish between the names 'foo-stubs' and 'foo_stubs'
> -- they get normalized together. So even if you use 'foo-stubs' as the
> directory name on sys.path to avoid collisions at import time, it
> still won't allow someone to distribute a separate 'foo_stubs' package
> on PyPI.
>
> If you do go with a fixed naming convention like this, the PEP should
> probably also instruct the PyPI maintainers that whoever owns 'foo'
> automatically has the right to control the name 'foo-stubs' as well.
> Or maybe some tweak to PEP 541 is needed.
>

As I understand it however, the distribution name need not map to to the
package name in any way. So regardless of if foo-stubs is seen as
foo_stubs, I could name the distribution Albatross if I wished, and install
the foo-stubs package into site/dist-packages, and it would work. Also I'm
not sure if the PyPI change would require an edict from a PEP, but if so, I
wouldn't be opposed to the idea, I think it would be nice to default the
stub packages to the owners of the normal packages (people should, to my
understanding, be able to make alternate distributions without hassle).

Ethan

>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

13.11.17 02:00, Guido van Rossum пише:
It's hard to keep those two in sync, since the actual Grammar file is 
constrained by being strictly LL(1)... Can you get someone else to 
review the implementation?


I haven't change the grammar, just have changed checks in the CST to AST 
transformer. Maybe it would be better to change the grammar, but I'm not 
expert in this. Maybe Benjamin could provide better solution.


There are other related differences between language specification and 
the implementation. The following examples are valid syntax:


@deco(x for x in [1])
def f(): ...

class C(x for x in [1]): ...

The latter always raises a TypeError at runtime ("cannot create 
'generator' instances"), but is compiled successfully.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

[Nathaniel Smith]

On Sun, Nov 12, 2017 at 11:21 AM, Ethan Smith  wrote:
>
>
> On Sun, Nov 12, 2017 at 9:53 AM, Jelle Zijlstra 
> wrote:
>>
>> 2017-11-12 3:40 GMT-08:00 Ethan Smith :
>>> The name of the stub
>>> package
>>> MUST follow the scheme ``pkg_stubs`` for type stubs for the package named
>>> ``pkg``. The normal resolution order of checking ``*.pyi`` before
>>> ``*.py``
>>> will be maintained.
>>
>> This is very minor, but what do you think of using "pkg-stubs" instead of
>> "pkg_stubs" (using a hyphen rather than an underscore)? This would make the
>> name illegal to import as a normal Python package, which makes it clear that
>> it's not a normal package. Also, there could be real packages named
>> "_stubs".
>
> I suppose this makes sense. I checked PyPI and as of a few weeks ago there
> were no packages with the name pattern, but I like the idea of making it
> explicitly non-runtime importable. I cannot think of any reason not to do
> it, and the avoidance of confusion about the package being importable is a
> benefit. I will make the change with my next round of edits.

PyPI doesn't distinguish between the names 'foo-stubs' and 'foo_stubs'
-- they get normalized together. So even if you use 'foo-stubs' as the
directory name on sys.path to avoid collisions at import time, it
still won't allow someone to distribute a separate 'foo_stubs' package
on PyPI.

If you do go with a fixed naming convention like this, the PEP should
probably also instruct the PyPI maintainers that whoever owns 'foo'
automatically has the right to control the name 'foo-stubs' as well.
Or maybe some tweak to PEP 541 is needed.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Nathaniel Smith]

On Sun, Nov 12, 2017 at 1:24 AM, Nick Coghlan  wrote:
> This change will lead to DeprecationWarning being displayed by default for:
>
> * code executed directly at the interactive prompt
> * code executed directly as part of a single-file script

Technically it's orthogonal, but if you're trying to get better
warnings in the REPL, then you might also want to look at:

https://bugs.python.org/issue1539925
https://github.com/ipython/ipython/issues/6611

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Standardise the AST (Re: PEP 563: Postponed Evaluation of Annotations)

[Nick Coghlan]

On 13 November 2017 at 10:26, Guido van Rossum  wrote:
> [Greg]
>> Proponents of Lisp point to the advantages of easily
>> being able to express Lisp programs using Lisp data
>> structures. There would also be benefits in having a
>> standard way to represent Python programs using Python
>> data structures.
>
> But we have to weigh the advantages against other requirements. IIRC Lisp
> had almost no syntax so I presume the mapping to data structures was nearly
> trivial compared to Python.

As far as I recall, the base primitives in Lisp are something like
UnaryOp, BinOp, and an asymmetric head/tail BinOp variant for working
with sequences.

That said, I think it could be genuinely useful to formally define a
"Simple Python expression" concept that:

- allowed symbolic Python expressions (including literals & displays)
- prohibited the use of keywords (avoiding the harder cases like
lambda, yield, await, and comprehensions)
- allowed name references (but didn't resolve them at compile time)
- defined an "escape prefix" to explicitly indicate references to
Python globals & builtins

That specific set of characteristics is drawn from the syntax used for
queries on pandas data frames:
https://pandas.pydata.org/pandas-docs/stable/generated/pandas.DataFrame.query.html

Right now, the data frame API relies on strings for that capability,
and pandas.eval to do the evaluation.

pandas.eval in turn provides two levels of configurability:

* which parser/compiler to use (which affects operand precedence)
* which execution engine to use (numexpr is much faster for SciPy
components than a regular Python eval)

For an integrated-into-Python variant, we presumably wouldn't allow
configurable operand precedence (we'd use the same rules as normal
expressions), but we could still offer a runtime expression type that
was compiled at the same time as everything else, but rather than
accepting parameters like a regular function, instead accepted a
namespace to use when evaluating the expression.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Standardise the AST (Re: PEP 563: Postponed Evaluation of Annotations)

[Guido van Rossum]

On Sun, Nov 12, 2017 at 2:39 PM, Greg Ewing 
wrote:

> Guido van Rossum wrote:
>
>> The PEP answers that clearly (under Implementation):
>>
>>  > If an annotation was already a string, this string is preserved
>>  > verbatim.
>>
>
> This bothers me, because it means the transformation from
> what you write in the source and the object you get at
> run time is not reversible. Something interpreting the
> annotations as Python expressions at run time has no way
> to know whether a Python expression was written in the
> source or a string literal whose contents happen to look
> like an expression.
>

The rule is a form of normalization. It is the best way to ensure that the
following two will mean the same thing:

def foo(a: int): ...
def foo(a: 'int'): ...

To reverse the transformation you have two options: always generate string
literals, or always generate non-string annotations. The resulting code
could look like either of the above two, and the meaning should be
identical. (There are edge cases, e.g.:

def foo(a: 'int)'): ...
def foo(a: '"int"'): ...

but they aren't currently valid as  PEP 484 type annotations and it's no
burden to keep them out of future syntactic extensions of annotations.)


> I still think that the run-time form of a non-evaluated
> annotation should be some form of AST. That's been rejected
> on the grounds that the structure of an AST is considered
> an implementation detail that can change between Python
> versions.
>
> However, that in itself seems like a bad thing to me.
> There *should* be a standard and stable form of AST
> provided that doesn't change unless the syntax of Python
> changes. It doesn't have to be the same as what's used
> internally by the compiler.
>

But Python's syntax changes in nearly every release. And even though we
nearly always take pains not to invalidate source code that used to be
valid, that's not so easy at the AST level, which elides many details (such
as whitespace and parentheses).


> Proponents of Lisp point to the advantages of easily
> being able to express Lisp programs using Lisp data
> structures. There would also be benefits in having a
> standard way to represent Python programs using Python
> data structures.
>

But we have to weigh the advantages against other requirements. IIRC Lisp
had almost no syntax so I presume the mapping to data structures was nearly
trivial compared to Python.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Disallow ambiguous syntax f(x for x in [1],)

[Guido van Rossum]

It's hard to keep those two in sync, since the actual Grammar file is
constrained by being strictly LL(1)... Can you get someone else to review
the implementation?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Nick Coghlan]

On 13 November 2017 at 03:10, Serhiy Storchaka  wrote:
> 12.11.17 11:24, Nick Coghlan пише:
>>
>> The PEP also proposes repurposing the existing FutureWarning category
>> to explicitly mean "backwards compatibility warnings that should be
>> shown to users of Python applications" since:
>>
>> - we don't tend to use FutureWarning for its original nominal purpose
>> (changes that will continue to run but will do something different)
>
> FutureWarning currently is used for its original nominal purpose in the re
> and ElementTree modules.

If the future warnings relate to regex and XML parsing, they'd still
fall under the "for display to users" category, since those modules
can't tell if the input data was application provided or part of an
end user interface like a configuration file.

> It even had been added in 2.7 for behavior that
> already have been changed in Python 3 or will be changed in future versions
> (emitted only with the -3 option).

That's closer to the original purpose, but with them being 2.7 only,
and gated behind the -3 switch, I think we can ignore them when it
comes to defining the expected usage in 3.7+

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

12.11.17 18:57, Guido van Rossum пише:

Sounds good to me.


Thanks! Here is an implementation: https://bugs.python.org/issue32012.

I have found that formally trailing comma after generator expression is 
not allowed by the grammar defined in the language reference:


   call: `primary` "(" [`argument_list` [","] | `comprehension`] ")"

But the actual Grammar file contains different rules.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Standardise the AST (Re: PEP 563: Postponed Evaluation of Annotations)

[Greg Ewing]

Guido van Rossum wrote:

The PEP answers that clearly (under Implementation):

 > If an annotation was already a string, this string is preserved
 > verbatim.


This bothers me, because it means the transformation from
what you write in the source and the object you get at
run time is not reversible. Something interpreting the
annotations as Python expressions at run time has no way
to know whether a Python expression was written in the
source or a string literal whose contents happen to look
like an expression.

I still think that the run-time form of a non-evaluated
annotation should be some form of AST. That's been rejected
on the grounds that the structure of an AST is considered
an implementation detail that can change between Python
versions.

However, that in itself seems like a bad thing to me.
There *should* be a standard and stable form of AST
provided that doesn't change unless the syntax of Python
changes. It doesn't have to be the same as what's used
internally by the compiler.

Proponents of Lisp point to the advantages of easily
being able to express Lisp programs using Lisp data
structures. There would also be benefits in having a
standard way to represent Python programs using Python
data structures.

--
Greg
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Analog of PEP 448 for dicts (unpacking in assignment with dict rhs)

[Ben Usman]

Anyway, considering that this has been discussed a lot in the original post
in 2016, I suggest stopping any further discussions here to avoid littering
dev mailing list. Sorry for starting the thread in the first place and
thank you, Jelle, for pointing me to the original discussion.

On Nov 12, 2017 14:33, "Ben Usman"  wrote:

Sounds like that happens quite often.

Yep, I totally agree with your point, I think I mentioned something like
this in the post as a possible partial solution: a drop-in replacement for
an ugly list compression people seem to be using now to solve the problem.
It's easy to implement, but the adoption by community is questionable. I
mean, if this is a relatively rare use case, but those who need it seem to
have their own one-liners for that already, is there even a need for a
method or function like this in standard library? To unify to improve
readability (single standard "getitems" instead of many different get_n,
gets, get_mutliple)? The only motivation I can think of, and even it is
questionable.

On Nov 12, 2017 05:06, "Nick Coghlan"  wrote:

On 11 November 2017 at 16:22, Jelle Zijlstra 
wrote:
> 2017-11-10 19:53 GMT-08:00 Ben Usman :
>> I was not able to find any PEPs that suggest this (search keywords:
>> "PEP 445 dicts", "dictionary unpacking assignment", checked PEP-0),
>> however, let me know if I am wrong.
>>
> It was discussed at great length on Python-ideas about a year ago. There
is
> a thread called "Unpacking a dict" from May 2016.

I tend to post this every time the topic comes up, but: it's highly
unlikely we'll get syntax for this when we don't even have a builtin
to extract multiple items from a mapping in a single operation.

So if folks would like dict unpacking syntax, then a suitable place to
start would be a proposal for a "getitems"  builtin that allowed
operations like:

b, a  = getitems(d, ("b", "a"))

operator.itemgetter and operator.attrgetter may provide some
inspiration for possible proposals.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Analog of PEP 448 for dicts (unpacking in assignment with dict rhs)

[Ben Usman]

Sounds like that happens quite often.

Yep, I totally agree with your point, I think I mentioned something like
this in the post as a possible partial solution: a drop-in replacement for
an ugly list compression people seem to be using now to solve the problem.
It's easy to implement, but the adoption by community is questionable. I
mean, if this is a relatively rare use case, but those who need it seem to
have their own one-liners for that already, is there even a need for a
method or function like this in standard library? To unify to improve
readability (single standard "getitems" instead of many different get_n,
gets, get_mutliple)? The only motivation I can think of, and even it is
questionable.

On Nov 12, 2017 05:06, "Nick Coghlan"  wrote:

On 11 November 2017 at 16:22, Jelle Zijlstra 
wrote:
> 2017-11-10 19:53 GMT-08:00 Ben Usman :
>> I was not able to find any PEPs that suggest this (search keywords:
>> "PEP 445 dicts", "dictionary unpacking assignment", checked PEP-0),
>> however, let me know if I am wrong.
>>
> It was discussed at great length on Python-ideas about a year ago. There
is
> a thread called "Unpacking a dict" from May 2016.

I tend to post this every time the topic comes up, but: it's highly
unlikely we'll get syntax for this when we don't even have a builtin
to extract multiple items from a mapping in a single operation.

So if folks would like dict unpacking syntax, then a suitable place to
start would be a proposal for a "getitems"  builtin that allowed
operations like:

b, a  = getitems(d, ("b", "a"))

operator.itemgetter and operator.attrgetter may provide some
inspiration for possible proposals.

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

[Ethan Smith]

On Sun, Nov 12, 2017 at 9:53 AM, Jelle Zijlstra 
wrote:

>
>
> 2017-11-12 3:40 GMT-08:00 Ethan Smith :
>
>> Hello,
>>
>> I re-wrote my PEP to have typing opt-in be per-package rather than
>> per-distribution. This greatly simplifies things, and thanks to the
>> feedback and suggestions of Nick Coghlan, it is entirely compatible with
>> older packaging tooling.
>>
>> The main idea is there are two types of packages:
>>  - types are packaged with runtime code (inline or stubs in the same
>> package)
>>  - types are in a separate package (a third party or maintainer wants to
>> ship type information, but not with runtime code).
>>
>> The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/
>>
>> And as always, duplicated below.
>>
>> Cheers,
>>
>> Ethan Smith
>>
>> ---
>>
>> PEP: 561
>> Title: Distributing and Packaging Type Information
>> Author: Ethan Smith 
>> Status: Draft
>> Type: Standards Track
>> Content-Type: text/x-rst
>> Created: 09-Sep-2017
>> Python-Version: 3.7
>> Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017
>>
>>
>> Abstract
>> 
>>
>> PEP 484 introduced type hinting to Python, with goals of making typing
>> gradual and easy to adopt. Currently, typing information must be distributed
>> manually. This PEP provides a standardized means to leverage existing tooling
>> to package and distribute type information with minimal work and an ordering
>> for type checkers to resolve modules and collect this information for type
>> checking.
>>
>>
>> Rationale
>> =
>>
>> Currently, package authors wish to distribute code that has inline type
>> information. Additionally, maintainers would like to distribute stub files
>> to keep Python 2 compatibility while using newer annotation syntax. However,
>> there is no standard method to distribute packages with type information.
>> Also, if one wished to ship stub files privately the only method available
>> would be via setting ``MYPYPATH`` or the equivalent to manually point to
>> stubs. If the package can be released publicly, it can be added to
>> typeshed [1]_. However, this does not scale and becomes a burden on the
>> maintainers of typeshed. In addition, it ties bug fixes in stubs to releases
>> of the tool using typeshed.
>>
>> PEP 484 has a brief section on distributing typing information. In this
>> section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
>> shipping stub files. However, manually adding a path to stub files for each
>> third party library does not scale. The simplest approach people have taken
>> is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
>> checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
>> and Django).
>>
>>
>> Definition of Terms
>> ===
>>
>> The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are
>> to be interpreted as described in RFC 2119.
>>
>> "inline" - the types are part of the runtime code using PEP 526 and 3107
>> syntax.
>>
>> "stubs" - files containing only type information, empty of runtime code.
>>
>> "Distributions" are the packaged files which are used to publish and 
>> distribute
>> a release. [3]_
>>
>> "Module" a file containing Python runtime code or stubbed type information.
>>
>> "Package" a directory or directories that namespace Python modules.
>>
>>
>> Specification
>> =
>>
>> There are several motivations and methods of supporting typing in a package.
>> This PEP recognizes three (3) types of packages that users of typing wish to
>> create:
>>
>> 1. The package maintainer would like to add type information inline.
>>
>> 2. The package maintainer would like to add type information via stubs.
>>
>> 3. A third party or package maintainer would like to share stub files for
>>a package, but the maintainer does not want to include them in the source
>>of the package.
>>
>> This PEP aims to support these scenarios and make them simple to add to
>> packaging and deployment.
>>
>> The two major parts of this specification are the packaging specifications
>> and the resolution order for resolving module type information. The type
>> checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec
>> of PEP 484 [2]_.
>>
>> New third party stub libraries SHOULD distribute stubs via the third party
>> packaging methods proposed in this PEP in place of being added to typeshed.
>> Typeshed will remain in use, but if maintainers are found, third party stubs
>> in typeshed MAY be split into their own package.
>>
>>
>> Packaging Type Information
>> --
>>
>> In order to make packaging and distributing type information as simple and
>> easy as possible, packaging and distribution is done through existing
>> frameworks.
>>
>> Package maintainers who wish to support type checking of their code MUST add
>> a ``py.typed`` 

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Koos Zevenhoven]

On Nov 12, 2017 19:10, "Guido van Rossum"  wrote:

On Sun, Nov 12, 2017 at 4:14 AM, Koos Zevenhoven  wrote:

> So actually my question is: What should happen when the annotation is
> already a string literal?
>

The PEP answers that clearly (under Implementation):

> If an annotation was already a string, this string is preserved
> verbatim.


Oh sorry, I was looking for a spec, so I somehow assumed I can ignore the
gory implementation details just like I routinely ignore things like
headers and footers of emails.

There's two thing I don't understand here:

* What does it mean to preserve the string verbatim? No matter how I read
it, I can't tell if it's with quotes or without.

Maybe I'm missing some context.


-- Koos (mobile)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 561 rework

[Jelle Zijlstra]

2017-11-12 3:40 GMT-08:00 Ethan Smith :

> Hello,
>
> I re-wrote my PEP to have typing opt-in be per-package rather than
> per-distribution. This greatly simplifies things, and thanks to the
> feedback and suggestions of Nick Coghlan, it is entirely compatible with
> older packaging tooling.
>
> The main idea is there are two types of packages:
>  - types are packaged with runtime code (inline or stubs in the same
> package)
>  - types are in a separate package (a third party or maintainer wants to
> ship type information, but not with runtime code).
>
> The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/
>
> And as always, duplicated below.
>
> Cheers,
>
> Ethan Smith
>
> ---
>
> PEP: 561
> Title: Distributing and Packaging Type Information
> Author: Ethan Smith 
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 09-Sep-2017
> Python-Version: 3.7
> Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017
>
>
> Abstract
> 
>
> PEP 484 introduced type hinting to Python, with goals of making typing
> gradual and easy to adopt. Currently, typing information must be distributed
> manually. This PEP provides a standardized means to leverage existing tooling
> to package and distribute type information with minimal work and an ordering
> for type checkers to resolve modules and collect this information for type
> checking.
>
>
> Rationale
> =
>
> Currently, package authors wish to distribute code that has inline type
> information. Additionally, maintainers would like to distribute stub files
> to keep Python 2 compatibility while using newer annotation syntax. However,
> there is no standard method to distribute packages with type information.
> Also, if one wished to ship stub files privately the only method available
> would be via setting ``MYPYPATH`` or the equivalent to manually point to
> stubs. If the package can be released publicly, it can be added to
> typeshed [1]_. However, this does not scale and becomes a burden on the
> maintainers of typeshed. In addition, it ties bug fixes in stubs to releases
> of the tool using typeshed.
>
> PEP 484 has a brief section on distributing typing information. In this
> section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
> shipping stub files. However, manually adding a path to stub files for each
> third party library does not scale. The simplest approach people have taken
> is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
> checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
> and Django).
>
>
> Definition of Terms
> ===
>
> The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are
> to be interpreted as described in RFC 2119.
>
> "inline" - the types are part of the runtime code using PEP 526 and 3107
> syntax.
>
> "stubs" - files containing only type information, empty of runtime code.
>
> "Distributions" are the packaged files which are used to publish and 
> distribute
> a release. [3]_
>
> "Module" a file containing Python runtime code or stubbed type information.
>
> "Package" a directory or directories that namespace Python modules.
>
>
> Specification
> =
>
> There are several motivations and methods of supporting typing in a package.
> This PEP recognizes three (3) types of packages that users of typing wish to
> create:
>
> 1. The package maintainer would like to add type information inline.
>
> 2. The package maintainer would like to add type information via stubs.
>
> 3. A third party or package maintainer would like to share stub files for
>a package, but the maintainer does not want to include them in the source
>of the package.
>
> This PEP aims to support these scenarios and make them simple to add to
> packaging and deployment.
>
> The two major parts of this specification are the packaging specifications
> and the resolution order for resolving module type information. The type
> checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec
> of PEP 484 [2]_.
>
> New third party stub libraries SHOULD distribute stubs via the third party
> packaging methods proposed in this PEP in place of being added to typeshed.
> Typeshed will remain in use, but if maintainers are found, third party stubs
> in typeshed MAY be split into their own package.
>
>
> Packaging Type Information
> --
>
> In order to make packaging and distributing type information as simple and
> easy as possible, packaging and distribution is done through existing
> frameworks.
>
> Package maintainers who wish to support type checking of their code MUST add
> a ``py.typed`` file to their package supporting typing. This marker is
> recursive, if a top-level package includes it, all sub-packages MUST support
> type checking as well. To have this file installed with the package,
> 

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Guido van Rossum]

On Sun, Nov 12, 2017 at 4:14 AM, Koos Zevenhoven  wrote:

> So actually my question is: What should happen when the annotation is
> already a string literal?
>

The PEP answers that clearly (under Implementation):

> If an annotation was already a string, this string is preserved
> verbatim.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Serhiy Storchaka]

12.11.17 11:24, Nick Coghlan пише:

The PEP also proposes repurposing the existing FutureWarning category
to explicitly mean "backwards compatibility warnings that should be
shown to users of Python applications" since:

- we don't tend to use FutureWarning for its original nominal purpose
(changes that will continue to run but will do something different)


FutureWarning currently is used for its original nominal purpose in the 
re and ElementTree modules. It even had been added in 2.7 for behavior 
that already have been changed in Python 3 or will be changed in future 
versions (emitted only with the -3 option).


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Disallow ambiguous syntax f(x for x in [1],)

[Guido van Rossum]

Sounds good to me.

On Sun, Nov 12, 2017 at 7:17 AM, Serhiy Storchaka 
wrote:

> Initially generator expressions always had to be written inside
> parentheses, as documented in PEP 289 [1]. The additional parenthesis could
> be omitted on calls with only one argument, because in this case the
> generator expression already is written inside parentheses. You could write
> just `list(x for x in [1])` instead of `list((x for x in [1]))`. The
> following code was an error:
>
> >>> list(x for x in [1], *[])
>   File "", line 1
> SyntaxError: invalid syntax
> >>> list(x for x in [1],)
>   File "", line 1
> SyntaxError: invalid syntax
>
> You needed to add explicit parenthesis in these cases:
>
> >>> list((x for x in [1]), *[])
> [1]
> >>> list((x for x in [1]),)
> [1]
>
> But in Python 2.5 the following examples were accepted:
>
> >>> list(x for x in [1], *[])
> [1]
> >>> list(x for x in [1], *{})
> [1]
> >>> list(x for x in [1],)
> [1]
>
> However I haven't found anything about this change in the "What's New In
> Python 2.5" document [2].
>
> The former two cases were found to be a mistake and it was fixed in Python
> 3.5.
>
> >>> list(x for x in [1], *[])
>   File "", line 1
> SyntaxError: Generator expression must be parenthesized if not sole
> argument
> >>> list(x for x in [1], *{})
>   File "", line 1
> SyntaxError: Generator expression must be parenthesized if not sole
> argument
>
> But `list(x for x in [1],)` still is accepted. I think it would be better
> it this raises a SyntaxError.
>
> 1. This syntax is ambiguous, because at first look it is not clear whether
> it is equivalent to `list((x for x in [1]),)` or to `list(x for x in
> ([1],))`.
>
> 2. It is bad from the aesthetic point of view, because this is the only
> case when the generator expression has not written inside parentheses. I
> believe that allowing to omit parenthesis in a call with a single generator
> expression argument was caused by aesthetic reasons.
>
> 3. I believe the trailing comma in function call was allowed because this
> simplified adding, removing and commenting out arguments.
>
> func(first_argument,
>  second_argument,
>  #third_argument,
> )
>
> You shouldn't touch other lines by adding or removing a comma when add or
> remove arguments. But this reason is not applicable to the case of `list((x
> for x in [1]),)`, because the generator expression without parenthesis
> should be the only argument. Therefore there is no reasons to allow this
> syntax.
>
> 4. 2to3 didn't supported this syntax for recent times [4]. Finally it was
> changed, but I think that it would be better to disallow this syntax for
> reasons mentioned above.
>
> [1] https://www.python.org/dev/peps/pep-0289/
> [2] https://docs.python.org/2.5/whatsnew/whatsnew25.html
> [3] https://docs.python.org/3.5/whatsnew/3.5.html#changes-in-pyt
> hon-behavior
> [4] https://bugs.python.org/issue27494
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
> 40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Analog of PEP 448 for dicts (unpacking in assignment with dict rhs)

[Mario Corchero]

Do you mean making getitems call itemgetter?

At the moment we can already do with itemgetter:

  from operator import itemgetter
  a,b = itemgetter("a", "b")(d)


> I tend to post this every time the topic comes up, but: it's highly
> unlikely we'll get syntax for this when we don't even have a builtin
> to extract multiple items from a mapping in a single operation.

You mean subitems as attrgetter does? That would be actually quite cool!

  d = dict(a=dict(b=1), b=dict(c=2))
  ab, ac = itemgetter("a.b", "b.c", separator=".")(d)


I've created an issue in case something like that is desired:
https://bugs.python.org/issue32010
No real strong push for it, happy to just close it if it does not get
interest.

That said I am not sure it solves Ben requests as he seamed to be
targetting a way to bind the variable name with the dictionary keys
implicitly.

On 12 November 2017 at 10:06, Nick Coghlan  wrote:

> On 11 November 2017 at 16:22, Jelle Zijlstra 
> wrote:
> > 2017-11-10 19:53 GMT-08:00 Ben Usman :
> >> I was not able to find any PEPs that suggest this (search keywords:
> >> "PEP 445 dicts", "dictionary unpacking assignment", checked PEP-0),
> >> however, let me know if I am wrong.
> >>
> > It was discussed at great length on Python-ideas about a year ago. There
> is
> > a thread called "Unpacking a dict" from May 2016.
>
> I tend to post this every time the topic comes up, but: it's highly
> unlikely we'll get syntax for this when we don't even have a builtin
> to extract multiple items from a mapping in a single operation.
>
> So if folks would like dict unpacking syntax, then a suitable place to
> start would be a proposal for a "getitems"  builtin that allowed
> operations like:
>
> b, a  = getitems(d, ("b", "a"))
>
> operator.itemgetter and operator.attrgetter may provide some
> inspiration for possible proposals.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> mariocj89%40gmail.com
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Antoine Pitrou]

On Sun, 12 Nov 2017 19:24:12 +1000
Nick Coghlan  wrote:
> I've written a short(ish) PEP for the proposal to change the default
> warnings filters to show DeprecationWarning in __main__:
> https://www.python.org/dev/peps/pep-0565/

Thank you for writing this.  This is a nice summary.  You finally
convinced me that it was a slight improvement rather than a pointless
complication. So I'm +0.5 :-)

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Guido van Rossum]

On Sun, Nov 12, 2017 at 1:24 AM, Nick Coghlan  wrote:

> In Python 2.7 and Python 3.2, the default warning filters were updated to
> hide
> DeprecationWarning by default, such that deprecation warnings in
> development
> tools that were themselves written in Python (e.g. linters, static
> analysers,
> test runners, code generators) wouldn't be visible to their users unless
> they
> explicitly opted in to seeing them.
>

Looking at the official What's New entry for the change (
https://docs.python.org/3/whatsnew/2.7.html#changes-to-the-handling-of-deprecation-warnings)
it's not just about development tools. It's about any app (or tool, or
utility, or program) written in Python whose users just treat it as "some
app", not as something that's necessarily part of their Python environment.
While in extreme cases such apps can *bundle* their own Python interpreter
(like Dropbox does), many developers opt to assume or ensure that Python is
avaiable, perhaps via the OS package management system. (Because my day job
is software development I am having a hard time coming up with concrete
examples that aren't development tools, but AFAIK at Dropbox the
*deployment* of e.g. Go binaries is managed through utilities written in
Python. The Go developers couldn't care less about that.)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Analog of PEP 448 for dicts (unpacking in assignment with dict rhs)

[Serhiy Storchaka]

12.11.17 12:06, Nick Coghlan пише:

So if folks would like dict unpacking syntax, then a suitable place to
start would be a proposal for a "getitems"  builtin that allowed
operations like:

 b, a  = getitems(d, ("b", "a"))

operator.itemgetter and operator.attrgetter may provide some
inspiration for possible proposals.


I don't see any relations between this getitems and operator.itemgetter 
or operator.attrgetter. getitems can be implemented as:


(the most obvious way)

def getitems(mapping, keys):
for key in keys:
yield mapping[key]

or

def getitems(mapping, keys):
return map(functools.partial(operator.getitem, mapping), keys)

or (simpler but rough equivalent)

def getitems(mapping, keys):
return map(mapping.__getitem__, keys)

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Disallow ambiguous syntax f(x for x in [1],)

[Serhiy Storchaka]

Initially generator expressions always had to be written inside 
parentheses, as documented in PEP 289 [1]. The additional parenthesis 
could be omitted on calls with only one argument, because in this case 
the generator expression already is written inside parentheses. You 
could write just `list(x for x in [1])` instead of `list((x for x in 
[1]))`. The following code was an error:


>>> list(x for x in [1], *[])
  File "", line 1
SyntaxError: invalid syntax
>>> list(x for x in [1],)
  File "", line 1
SyntaxError: invalid syntax

You needed to add explicit parenthesis in these cases:

>>> list((x for x in [1]), *[])
[1]
>>> list((x for x in [1]),)
[1]

But in Python 2.5 the following examples were accepted:

>>> list(x for x in [1], *[])
[1]
>>> list(x for x in [1], *{})
[1]
>>> list(x for x in [1],)
[1]

However I haven't found anything about this change in the "What's New In 
Python 2.5" document [2].


The former two cases were found to be a mistake and it was fixed in 
Python 3.5.


>>> list(x for x in [1], *[])
  File "", line 1
SyntaxError: Generator expression must be parenthesized if not sole argument
>>> list(x for x in [1], *{})
  File "", line 1
SyntaxError: Generator expression must be parenthesized if not sole argument

But `list(x for x in [1],)` still is accepted. I think it would be 
better it this raises a SyntaxError.


1. This syntax is ambiguous, because at first look it is not clear 
whether it is equivalent to `list((x for x in [1]),)` or to `list(x for 
x in ([1],))`.


2. It is bad from the aesthetic point of view, because this is the only 
case when the generator expression has not written inside parentheses. I 
believe that allowing to omit parenthesis in a call with a single 
generator expression argument was caused by aesthetic reasons.


3. I believe the trailing comma in function call was allowed because 
this simplified adding, removing and commenting out arguments.


func(first_argument,
 second_argument,
 #third_argument,
)

You shouldn't touch other lines by adding or removing a comma when add 
or remove arguments. But this reason is not applicable to the case of 
`list((x for x in [1]),)`, because the generator expression without 
parenthesis should be the only argument. Therefore there is no reasons 
to allow this syntax.


4. 2to3 didn't supported this syntax for recent times [4]. Finally it 
was changed, but I think that it would be better to disallow this syntax 
for reasons mentioned above.


[1] https://www.python.org/dev/peps/pep-0289/
[2] https://docs.python.org/2.5/whatsnew/whatsnew25.html
[3] https://docs.python.org/3.5/whatsnew/3.5.html#changes-in-python-behavior
[4] https://bugs.python.org/issue27494

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] PEP 563: Postponed Evaluation of Annotations

[Koos Zevenhoven]

On Sun, Nov 12, 2017 at 7:07 AM, Guido van Rossum  wrote:

> On Fri, Nov 10, 2017 at 11:02 PM, Nick Coghlan  wrote:
>
>> On 11 November 2017 at 01:48, Guido van Rossum  wrote:
>> > I don't mind the long name. Of all the options so far I really only like
>> > 'string_annotations' so let's go with that.
>>
>> +1 from me.
>>
>
> I'd like to reverse my stance on this. We had `from __future__ import
> division` for many years in Python 2, and nobody argued that it implied
> that Python 2 doesn't have division -- it just meant to import the future
> *version* of division. So I think the original idea, `from __future__
> import annotations` is fine. I don't expect there will be *other* things
> related to annotations that we'll be importing from the future.
>
>
Furthermore, *​nobody* expects the majority of programmers to look at
__annotations__ either. But those who do need to care about the
'implementation detail' of whether it's a string won't be surprised to find
nested strings like "'ForwardReferencedThing'". But one might fear that
those cases get ruthlessly converted into being equivalent to just
"ForwardReferencedThing".

So actually my question is: What should happen when the annotation is
already a string literal?

-- Koos
  ​
-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 561 rework

[Ethan Smith]

Hello,

I re-wrote my PEP to have typing opt-in be per-package rather than
per-distribution. This greatly simplifies things, and thanks to the
feedback and suggestions of Nick Coghlan, it is entirely compatible with
older packaging tooling.

The main idea is there are two types of packages:
 - types are packaged with runtime code (inline or stubs in the same
package)
 - types are in a separate package (a third party or maintainer wants to
ship type information, but not with runtime code).

The PEP is live on python.org: https://www.python.org/dev/peps/pep-0561/

And as always, duplicated below.

Cheers,

Ethan Smith

---

PEP: 561
Title: Distributing and Packaging Type Information
Author: Ethan Smith 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 09-Sep-2017
Python-Version: 3.7
Post-History: 10-Sep-2017, 12-Sep-2017, 06-Oct-2017, 26-Oct-2017


Abstract


PEP 484 introduced type hinting to Python, with goals of making typing
gradual and easy to adopt. Currently, typing information must be distributed
manually. This PEP provides a standardized means to leverage existing tooling
to package and distribute type information with minimal work and an ordering
for type checkers to resolve modules and collect this information for type
checking.


Rationale
=

Currently, package authors wish to distribute code that has inline type
information. Additionally, maintainers would like to distribute stub files
to keep Python 2 compatibility while using newer annotation syntax. However,
there is no standard method to distribute packages with type information.
Also, if one wished to ship stub files privately the only method available
would be via setting ``MYPYPATH`` or the equivalent to manually point to
stubs. If the package can be released publicly, it can be added to
typeshed [1]_. However, this does not scale and becomes a burden on the
maintainers of typeshed. In addition, it ties bug fixes in stubs to releases
of the tool using typeshed.

PEP 484 has a brief section on distributing typing information. In this
section [2]_ the PEP recommends using ``shared/typehints/pythonX.Y/`` for
shipping stub files. However, manually adding a path to stub files for each
third party library does not scale. The simplest approach people have taken
is to add ``site-packages`` to their ``MYPYPATH``, but this causes type
checkers to fail on packages that are highly dynamic (e.g. sqlalchemy
and Django).


Definition of Terms
===

The definition of "MAY", "MUST", and "SHOULD", and "SHOULD NOT" are
to be interpreted as described in RFC 2119.

"inline" - the types are part of the runtime code using PEP 526 and 3107
syntax.

"stubs" - files containing only type information, empty of runtime code.

"Distributions" are the packaged files which are used to publish and distribute
a release. [3]_

"Module" a file containing Python runtime code or stubbed type information.

"Package" a directory or directories that namespace Python modules.


Specification
=

There are several motivations and methods of supporting typing in a package.
This PEP recognizes three (3) types of packages that users of typing wish to
create:

1. The package maintainer would like to add type information inline.

2. The package maintainer would like to add type information via stubs.

3. A third party or package maintainer would like to share stub files for
   a package, but the maintainer does not want to include them in the source
   of the package.

This PEP aims to support these scenarios and make them simple to add to
packaging and deployment.

The two major parts of this specification are the packaging specifications
and the resolution order for resolving module type information. The type
checking spec is meant to replace the ``shared/typehints/pythonX.Y/`` spec
of PEP 484 [2]_.

New third party stub libraries SHOULD distribute stubs via the third party
packaging methods proposed in this PEP in place of being added to typeshed.
Typeshed will remain in use, but if maintainers are found, third party stubs
in typeshed MAY be split into their own package.


Packaging Type Information
--

In order to make packaging and distributing type information as simple and
easy as possible, packaging and distribution is done through existing
frameworks.

Package maintainers who wish to support type checking of their code MUST add
a ``py.typed`` file to their package supporting typing. This marker is
recursive, if a top-level package includes it, all sub-packages MUST support
type checking as well. To have this file installed with the package,
maintainers can use existing packaging options such as ``package_data`` in
distutils, shown below.

Distutils option example::

...
package_data = {
'pkg': ['py.typed'],
},
...

For namespace packages, the ``py.typed`` file should be in the submodules of
the 

Re: [Python-Dev] Analog of PEP 448 for dicts (unpacking in assignment with dict rhs)

[Nick Coghlan]

On 11 November 2017 at 16:22, Jelle Zijlstra  wrote:
> 2017-11-10 19:53 GMT-08:00 Ben Usman :
>> I was not able to find any PEPs that suggest this (search keywords:
>> "PEP 445 dicts", "dictionary unpacking assignment", checked PEP-0),
>> however, let me know if I am wrong.
>>
> It was discussed at great length on Python-ideas about a year ago. There is
> a thread called "Unpacking a dict" from May 2016.

I tend to post this every time the topic comes up, but: it's highly
unlikely we'll get syntax for this when we don't even have a builtin
to extract multiple items from a mapping in a single operation.

So if folks would like dict unpacking syntax, then a suitable place to
start would be a proposal for a "getitems"  builtin that allowed
operations like:

b, a  = getitems(d, ("b", "a"))

operator.itemgetter and operator.attrgetter may provide some
inspiration for possible proposals.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] PEP 565: Show DeprecationWarning in __main__

[Nick Coghlan]

I've written a short(ish) PEP for the proposal to change the default
warnings filters to show DeprecationWarning in __main__:
https://www.python.org/dev/peps/pep-0565/

The core proposal itself is just the idea in
https://bugs.python.org/issue31975 (i.e. adding
"default::DeprecationWarning:__main__" to the default filter set), but
the PEP fills in some details on the motivation for the original
change to the defaults, and why the current proposal is to add a new
filter for __main__, rather than dropping the default
DeprecationWarning filter entirely.

The PEP also proposes repurposing the existing FutureWarning category
to explicitly mean "backwards compatibility warnings that should be
shown to users of Python applications" since:

- we don't tend to use FutureWarning for its original nominal purpose
(changes that will continue to run but will do something different)
- FutureWarning was added in 2.3, so it's available in all still
supported versions of Python, and is shown by default in all of them
- it's at least arguably a less-jargony spelling of
DeprecationWarning, and hence more appropriate for displaying to end
users that may not have encountered the specific notion of "API
deprecation"

Cheers,
Nick.

==
PEP: 565
Title: Show DeprecationWarning in __main__
Author: Nick Coghlan 
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 12-Nov-2017
Python-Version: 3.7
Post-History: 12-Nov-2017


Abstract


In Python 2.7 and Python 3.2, the default warning filters were updated to hide
DeprecationWarning by default, such that deprecation warnings in development
tools that were themselves written in Python (e.g. linters, static analysers,
test runners, code generators) wouldn't be visible to their users unless they
explicitly opted in to seeing them.

However, this change has had the unfortunate side effect of making
DeprecationWarning markedly less effective at its primary intended purpose:
providing advance notice of breaking changes in APIs (whether in CPython, the
standard library, or in third party libraries) to users of those APIs.

To improve this situation, this PEP proposes a single adjustment to the
default warnings filter: displaying deprecation warnings attributed to the main
module by default.

This change will mean that code entered at the interactive prompt and code in
single file scripts will revert to reporting these warnings by default, while
they will continue to be silenced by default for packaged code distributed as
part of an importable module.

The PEP also proposes a number of small adjustments to the reference
interpreter and standard library documentation to help make the warnings
subsystem more approachable for new Python developers.


Specification
=

The current set of default warnings filters consists of::

ignore::DeprecationWarning
ignore::PendingDeprecationWarning
ignore::ImportWarning
ignore::BytesWarning
ignore::ResourceWarning

The default ``unittest`` test runner then uses ``warnings.catch_warnings()``
``warnings.simplefilter('default')`` to override the default filters while
running test cases.

The change proposed in this PEP is to update the default warning filter list
to be::

default::DeprecationWarning:__main__
ignore::DeprecationWarning
ignore::PendingDeprecationWarning
ignore::ImportWarning
ignore::BytesWarning
ignore::ResourceWarning

This means that in cases where the nominal location of the warning (as
determined by the ``stacklevel`` parameter to ``warnings.warn``) is in the
``__main__`` module, the first occurrence of each DeprecationWarning will once
again be reported.

This change will lead to DeprecationWarning being displayed by default for:

* code executed directly at the interactive prompt
* code executed directly as part of a single-file script

While continuing to be hidden by default for:

* code imported from another module in a ``zipapp`` archive's ``__main__.py``
  file
* code imported from another module in an executable package's ``__main__``
  submodule
* code imported from an executable script wrapper generated at installation time
  based on a ``console_scripts`` or ``gui_scripts`` entry point definition

As a result, API deprecation warnings encountered by development tools written
in Python should continue to be hidden by default for users of those tools

While not its originally intended purpose, the standard library documentation
will also be updated to explicitly recommend the use of
``FutureWarning`` (rather
than ``DeprecationWarning``) for backwards compatibility warnings that are
intended to be seen by *users* of an application.

This will give the following three distinct categories of backwards
compatibility warning, with three different intended audiences:

* ``PendingDeprecationWarning``: reported by default only in test runners that
  override the default set of warning filters. The intended audience is Python