[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Inada Naoki]

On Thu, Oct 21, 2021 at 1:08 PM Henry Fredrick Schreiner
 wrote:
>
> > typing features from future Python versions
>
> I second both of these uses, but especially this (which seems to be missing 
> from the original post), it’s been by far the main reason I’ve used this mode 
> and I’ve seen this used, and is the main feature to look forward to when 
> dropping Python 3.7 support. The new features coming to typing make static 
> typing much easier, but no libraries can drop Python 3.7/3.8/3.9 support; but 
> static typing doesn’t need old versions.
>

I agree with this point. We shouldn't emit DeprecationWarning for
`from __future__ import annotations` at least 3 versions (3.11 ~
3.13).

> When talking about speed, one of the important things to consider here is the 
> difference between the two proposals. PEP 649 was about the same as the 
> current performance, but PEP 563 was significantly faster, since it doesn’t 
> instantiate or deal with objects at all, which both the current default and 
> PEP 563 do. You could even protect imports with TYPE_CHECKING with PEP 563, 
> and further reduce the runtime cost of adding types - which could be seen as 
> a reason to avoid adding types. To the best of my knowledge, it hasn’t been a 
> blocker for packages, but something to include.
>
> Also, one of the original points for static typing is that strings can be 
> substituted for objects. “Type” is identical, from a static typing 
> perspective, to Type. You can swap one for the other, and for a Python 3.6+ 
> codebase, using something like “A | B” (with the quotes) is a valid way to 
> have static types in 3.6 that pass MyPy (though are not usable at runtime, 
> obviously, but that’s often not a requirement). NumPy, for example, makes 
> heavy usage of unions and other newer additions in static typing.
>

We may be able to provide tool to rewrite Python sources like 2to3:

* Remove `from __future__ import annotations`
* Stringify annotations, if it is not constants (e.g. `None`, `42`,
`"foo"` are not rewrote).

This tool can ease transition from PEP 563 to 649, and solve
performance issues too.
PEP 649 can have the performance as PEP 563 if all annotations are constants.

-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZFGRO6UGFVOBPTW2EFNLUC7WEYPCRLAQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Henry Fredrick Schreiner]

> typing features from future Python versions

I second both of these uses, but especially this (which seems to be missing 
from the original post), it’s been by far the main reason I’ve used this mode 
and I’ve seen this used, and is the main feature to look forward to when 
dropping Python 3.7 support. The new features coming to typing make static 
typing much easier, but no libraries can drop Python 3.7/3.8/3.9 support; but 
static typing doesn’t need old versions.

When talking about speed, one of the important things to consider here is the 
difference between the two proposals. PEP 649 was about the same as the current 
performance, but PEP 563 was significantly faster, since it doesn’t instantiate 
or deal with objects at all, which both the current default and PEP 563 do. You 
could even protect imports with TYPE_CHECKING with PEP 563, and further reduce 
the runtime cost of adding types - which could be seen as a reason to avoid 
adding types. To the best of my knowledge, it hasn’t been a blocker for 
packages, but something to include.

Also, one of the original points for static typing is that strings can be 
substituted for objects. “Type” is identical, from a static typing perspective, 
to Type. You can swap one for the other, and for a Python 3.6+ codebase, using 
something like “A | B” (with the quotes) is a valid way to have static types in 
3.6 that pass MyPy (though are not usable at runtime, obviously, but that’s 
often not a requirement). NumPy, for example, makes heavy usage of unions and 
other newer additions in static typing.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/L3IKBO5YZNQ2B5Y6VA7KX352VCOCQEBB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Packing a long list of numbers into memory

[Tim Peters]

Sorry for the spam! A bunch of these were backed up in the moderation
queue. I used the UI to set the list to auto-discard future messages
from this address, but then clicked "Accept" in the mistaken sense of
"yes, accept my request to auto-nuke this clown". But it took "Accept"
to mean "sure thing, boss! I'll send this to everyone ASAP!!".
Computer software. Everyone, please, get a useful job instead ;-)

On Wed, Oct 20, 2021 at 10:43 PM  wrote:
>
> that is great to see this post . https://bit.ly/3C551OO
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-dev@python.org/message/5BMKWRYJVHTKLIW4MJM7J7SEYN2OXNJE/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FZDBRF5D7ZBTXRUVJXPSBRFA2RRMI7LE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Packing a long list of numbers into memory

[joeevansjoe6]

that is great to see this post . https://bit.ly/3C551OO
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5BMKWRYJVHTKLIW4MJM7J7SEYN2OXNJE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 505 (None-aware operators) for Python 3.11

[Steven D'Aprano]

On Tue, Oct 19, 2021 at 05:09:42PM -0700, Michael Selik wrote:

> None and its ilk often conflate too many qualities. For example, is it
> missing because it doesn't exist, it never existed, or because we never
> received a value, despite knowing it must exist?

Does it matter if different functions have different semantic 
interpretations for None?


> The languages SAS and R
> support at least 27 varieties of NA, allowing un-tagged, and tagged with
> the letters A-Z to help someone create distinctions between different kinds
> of nothingness. IEEE-754 allows about 16 million possible NaNs, which I
> believe was intended to allow floating point instructions to pass error
> messages along.

Yes, and after something like 30-40 years of IEEE-754 supporting NAN 
payloads, the number of systems that actually use them can probably be 
counted on the fingers of one hand :-(

Ironically, one of those systems is R, which -- so I have been lead to 
believe -- uses distict NANs to represent those 27 tagged NA values.

Back in the 1980s, one of the earliest systems which supported IEEE-754 
maths was the Apple Numeric Toolkit. Apple's maths routines generated 
NANs with documented payloads for certain errors, e.g:

* NAN(1)  invalid sqrt
* NAN(2)  invalid addition such as INF + -INF
* NAN(34) invalid argument to inverse trig functions

In a complex computation, it was sometimes useful to see why a NAN was 
generated. Alas, when Apple moved their maths routines into hardware, 
the MC68881 coprocessor always generated NANs with payload 255, and that 
useful debugging information was lost.

30+ years later, and we cannot easily, reliably or portably use NAN 
payloads. Most people don't care. If we offerred them a dozen or a 
thousand distinct sentinels for all the various kinds of missing data, 
how many people would use them and how many would just stick to plain 
old None?


> If the motivation for this operator is chained lookups, how about adding a
> feature to the operator module, first? It seems natural to add a
> keyword-only argument to `attrgetter`, and it's a lighter touch than
> implementing a new operator. If use becomes widespread, that gives more
> weight to PEP 505.

I agree that this is a nice way forward, and a useful function in its 
own right. The only thing is that I would argue for a different colour 
of the bikeshed:

def getattr_chain(obj, *attrs, default):
# like obj.a.b.c.d
# if any attribute is missing, 
# raises if default is not given
# otherwise returns default

getattr is far more commonly used than attrgetter.


-- 
Steve
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/AAHVRF7WXIZRUGBHOVBEB7NEZAYHHL26/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Inada Naoki]

On Thu, Oct 21, 2021 at 6:38 AM Christopher Barker  wrote:
>
> Thanks to the SC for such a thoughtful note. I really like where this is 
> going.
>
> One thought.
>
> On Wed, Oct 20, 2021 at 6:21 AM Thomas Wouters  wrote:
>>
>> Is the performance of PEP 649 and PEP 563 similar enough that we can 
>> outright discount it as a concern? Does anyone actually care about the 
>> overhead of type annotations anymore? Are there other options to alleviate 
>> this potential issue (like a process-wide switch to turn off annotations)?
>
> Annotations are used at runtime by at least one std lib module: dataclasses, 
> and who knows how many third party libs. So that may not be practical.
>

This is similar to docstring. Some tools using docstring (e.g. docopt)
prevent using -OO option.
Although some library (e.g. SQLAlchemy) has huge docstring, we can not
use -OO if a time set of module depends on docstring or assertion.

So I think we need some mechanizm to disable optimization like
dropping assertions, docstrings, and annotations per module.

-- 
Inada Naoki  
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DMJOQ6JXDQSPRZPOLL4FRADIMP5EZTF6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 505 (None-aware operators) for Python 3.11

[Piotr Waszkiewicz]

On Wed, Oct 20, 2021 at 9:39 PM Michael Selik  wrote:

>
>
> On Wed, Oct 20, 2021 at 9:18 AM Piotr Waszkiewicz 
> wrote:
>
>> Do you think about something along those lines?
>> ```
>> phone = book.publisher.owner.phone except AttributeError: None
>> ```
>>
>
> Yes, that seems reasonable.
>

Nice, I think I would also be able to get used to that notation. It's good
to know that there are others supporting the PEP 463, maybe it'd be
possible to propose it once again.


>
>
>> I don't mind this syntax but it would have to be supported by static type
>> checkers and IDEs. And currently something like this is not:
>> ```
>> try:
>> phone = book.publisher.owner.phone
>> except AttributeError:
>> phone = None
>> ```
>>
>> mypy complains:
>> ```
>> error: Item "None" of "Optional[Publisher]" has no attribute "owner"
>> ```
>>
>
> That sounds like a feature request for mypy. Would creating a new operator
> make it easier to implement analysis of that situation would mypy? My guess
> is not. Checking the AST to see if there's a try/except AttributeError
> sounds comparable to checking for the use of a none-aware operator. I'm
> completely ignorant of how mypy does its analysis, so that's just a wild
> guess.
>

I'm not sure, just wanting to point out that the `AttributeError` syntax is
not completely equivalent to the maybe-dot operator here. This example
would actually hide a real AttributeError problem, e.g. if the `publisher`
didn't have an owner field.
So I guess there may be a need to introduce a new exception type, e.g.
`NoneAttributeAccess` before mypy can safely allow any attribute access in
such situation.


>
> If PEP 505 is accepted, it would need support in the `operator` module.
>>> Might as well design that aspect of the implementation now.
>>>
>>
>> I'm sorry but I don't know if I understand that sentence correctly. You
>> mean we would have to add an "explicit" function that behaves like a
>> maybe-dot operator? Is it actually a requirement when adding new operators?
>>
>
> The documentation of the `operator` module says, "The operator module
> exports a set of efficient functions corresponding to the intrinsic
> operators of Python." It feels like there's an implicit "all" in there. The
> table of correspondences looks exhaustive. I haven't noticed any exceptions.
>
>
> https://docs.python.org/3/library/operator.html#mapping-operators-to-functions
>

Thank you very much, I wasn't aware of that module before. Will look into
that.

I don't want to prolong this conversation too much, as I feel like I get
your point and agree with it to some (rather great) extent. That doesn't
change my attitude towards this PEP 505 proposal though, as I feel that if
the general consensus would be towards accepting this change it will bring
some quality of life improvements in a usual day-to-day work, when dealing
with not-so-ideal code.
I'd be also interested in seeing PEP 463 being resurrected as it looks like
there are some folks here interested in restarting the discussion about it.

Thank you very much for the fruitful discussion and broadening my knowledge.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/IO5SREDWU3TJFEEXZ4ULD6D7PTGL4MNE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 505 (None-aware operators) for Python 3.11

[Michael Selik]

On Wed, Oct 20, 2021 at 9:18 AM Piotr Waszkiewicz 
wrote:

> Do you think about something along those lines?
> ```
> phone = book.publisher.owner.phone except AttributeError: None
> ```
>

Yes, that seems reasonable.


> I don't mind this syntax but it would have to be supported by static type
> checkers and IDEs. And currently something like this is not:
> ```
> try:
> phone = book.publisher.owner.phone
> except AttributeError:
> phone = None
> ```
>
> mypy complains:
> ```
> error: Item "None" of "Optional[Publisher]" has no attribute "owner"
> ```
>

That sounds like a feature request for mypy. Would creating a new operator
make it easier to implement analysis of that situation would mypy? My guess
is not. Checking the AST to see if there's a try/except AttributeError
sounds comparable to checking for the use of a none-aware operator. I'm
completely ignorant of how mypy does its analysis, so that's just a wild
guess.

If PEP 505 is accepted, it would need support in the `operator` module.
>> Might as well design that aspect of the implementation now.
>>
>
> I'm sorry but I don't know if I understand that sentence correctly. You
> mean we would have to add an "explicit" function that behaves like a
> maybe-dot operator? Is it actually a requirement when adding new operators?
>

The documentation of the `operator` module says, "The operator module
exports a set of efficient functions corresponding to the intrinsic
operators of Python." It feels like there's an implicit "all" in there. The
table of correspondences looks exhaustive. I haven't noticed any exceptions.

https://docs.python.org/3/library/operator.html#mapping-operators-to-functions
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/46H5MYVVMQMWS4DVJYVFGKWXS3CNYTJC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Christopher Barker]

Thanks to the SC for such a thoughtful note. I really like where this is
going.

One thought.

On Wed, Oct 20, 2021 at 6:21 AM Thomas Wouters  wrote:

>
>1.
>
>Is the performance of PEP 649 and PEP 563 similar enough that we can
>outright discount it as a concern? Does anyone actually care about the
>overhead of type annotations anymore? Are there other options to alleviate
>this potential issue (like a process-wide switch to turn off annotations)?
>
>
Annotations are used at runtime by at least one std lib module:
dataclasses, and who knows how many third party libs. So that may not be
practical.

-CHB
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/4C3G5M2OUCFHNLHJ6HHBQWLYOOT3SKXD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 505 (None-aware operators) for Python 3.11

[Piotr Waszkiewicz]

Hi,

On Wed, Oct 20, 2021 at 5:44 PM Michael Selik  wrote:

> On Wed, Oct 20, 2021 at 1:16 AM Piotr Waszkiewicz 
> wrote:
>
>> On Wed, Oct 20, 2021 at 2:33 AM Michael Selik  wrote:
>>
>>> In case it saves anyone a couple clicks:
>>> https://www.python.org/dev/peps/pep-0463/
>>> I also prefer more syntactic help with exceptions, rather than more
>>> syntax emphasizing None's uniqueness.
>>>
>>
>> Me too, but could you provide me an example where try-except approach is
>> more readable when trying to chain attribute lookups (like in the database
>> book-publisher-owner example I have provided before).
>>
>
> I'd echo the others' examples, taking inspiration from PEP 463.
>

Do you think about something along those lines?
```
phone = book.publisher.owner.phone except AttributeError: None
```

I don't mind this syntax but it would have to be supported by static type
checkers and IDEs. And currently something like this is not:
```
try:
phone = book.publisher.owner.phone
except AttributeError:
phone = None
```

mypy complains:
```
error: Item "None" of "Optional[Publisher]" has no attribute "owner"
```


>
>
>> If the motivation for this operator is chained lookups, how about adding
>>> a feature to the operator module, first? It seems natural to add a
>>> keyword-only argument to `attrgetter`, and it's a lighter touch than
>>> implementing a new operator. If use becomes widespread, that gives more
>>> weight to PEP 505.
>>>
>>> def attrgetter(*attrs, none_aware=False)
>>>
>>> https://docs.python.org/3/library/operator.html#operator.attrgetter
>>>
>>
>> I remember using inhouse solution like this at a certain workplace - a
>> method accepting list of string arguments and an object, returning the
>> value being the result of chained attribute access. And it worked all
>> right. The problem I have with such approaches is that the name of the
>> attrs are passed as strings.
>>
>
> I understand the preference for attributes over strings, but many of the
> none-aware examples use keys and indices. If JSON is the main culprit for
> deeply nested structures, then you're already using strings and not
> attributes. Adding features to `operator` wouldn't preclude accepting PEP
> 505, so why not get started with a less controversial change that provides
> much of the value?
>

I have nothing against introducing such a new feature to the `operator`
apart from this one problem mentioned before (using strings which are not
properly detected by IDE), and I agree that could be a good start.
I've seen chained-attributes-lookups solutions in quite a few places
already and I think that there would actually be people benefiting from
such addition.

Although I must admit that personally I don't see many benefits of using
strings for attribute lookups due to typing and IDE issues mentioned
before. Even for JSON data, in my own projects I tend to write dataclasses
wrapping parsed dict in order to benefit from IDE tooltips.


>
> If PEP 505 is accepted, it would need support in the `operator` module.
> Might as well design that aspect of the implementation now.
>

I'm sorry but I don't know if I understand that sentence correctly. You
mean we would have to add an "explicit" function that behaves like a
maybe-dot operator?
Is it actually a requirement when adding new operators?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5K36MMO43TWKAXGGNM257MY356WLZ45M/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Sebastian Rittau]

I'm sorry, I sent my last mail too early. Here are the rest of my thoughts.

Am 20.10.21 um 15:18 schrieb Thomas Wouters:


Keeping the future import and stringified annotations around is 
certainly an option, but we’re worried about the cost of the 
implementation, the support cost, and the confusion for users 
(specifically, it is a future import that will never become the 
future). If we do keep them, how long would we keep them around? 
Should we warn about their use? If we warn about the future import, is 
the noise and confusion this generates going to be worth it? If we 
don't warn about them, how will we ever be able to turn them off?


Personally, I think that stringified annotations should be deprecated 
and eventually be removed. This opens the design space to use quotes for 
different purposes, for example getting rid of the cumbersome need to 
use Literal for literals.


One thing we’re thinking of specifically for the future import, and 
for other deprecations in Python, is to revisit the deprecation and 
warning policy. We think it’s pretty clear that the policy we have 
right now doesn’t exactly work. We used to have noisy 
DeprecationWarnings, which were confusing to end users when they were 
not in direct control of the code. We now have silent-by-default 
DeprecationWarnings, where the expectation is that test frameworks 
surface these warnings. This avoids the problem of end users being 
confused, but leaves the problem of the code’s dependencies triggering 
the warning, and thus still warns users (developers) not necessarily 
in a position to fix the problem, which in turn leads to them 
silencing the warning and moving on. We need a better way to reach the 
users in a position to update the code.



+1


1.

If we do need a warning, how loud, and how long should it be
around? At the end of the deprecation period, should the future
import be an error, or simply be ignored?

The future import should remain active and should continue to work as it 
does now (without warning) as long as there are Python versions 
supported that have not implemented PEP 649, i.e. Python 3.10. 
Otherwise, existing code that provides supports for these older versions 
(especially libraries) have to regress their typing support. For 
example, a library supporting 3.8+ could have the following code:


from __future__ import annotations

def foo(x: int | str): pass

If newer versions stop supporting the future import or warn about it, 
the library would have to go back to using typing.Union here. Other 
constructs would even be impossible to use.


After that, I would recommend to start the normal "warn"/"remove" cycle 
for the future import. Don't keep it around if it's doing nothing.


 - Sebastian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RHSQFTTCTPIH4X3XCQBYXFSAO3I6CTBN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Type annotations, PEP 649 and PEP 563

[Sebastian Rittau]

Am 20.10.21 um 15:18 schrieb Thomas Wouters:


By and large, the SC views PEP 649 as a better way forward. If PEP 563 
had never come along, it would be a fairly easy decision to accept PEP 
649. We are still inclined to accept PEP 649. That would leave the 
consideration about what to do with PEP 563 and existing from 
__future__ import annotationsdirectives. As far as we can tell, there 
are two reasons for code to want to use PEP 563: being able to 
conveniently refer to names that aren’t available until later in the 
code (i.e. forward references), and reducing the overhead of type 
annotations. If PEP 649 satisfies allof the objectives of PEP 563, is 
there a reason to keep supporting PEP 563’s stringified annotations?  
Are there any practical, real uses of stringified annotations that 
would not be served by PEP 649's deferred annotations?



These are my reasons for using from __future__ import annotations:

 * Forward references and cyclic imports (with if TYPE_CHECKING: from
   foo import Bar) while avoiding quotes.
 * Using typing features from future Python versions. This becomes less
   important the more typing stabilizes.

 - Sebastian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KZK7Q4LT7CDBUDSAPQH4236AA5KABSGM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 505 (None-aware operators) for Python 3.11

[Michael Selik]

On Wed, Oct 20, 2021 at 1:16 AM Piotr Waszkiewicz 
wrote:

> On Wed, Oct 20, 2021 at 2:33 AM Michael Selik  wrote:
>
>> In case it saves anyone a couple clicks:
>> https://www.python.org/dev/peps/pep-0463/
>> I also prefer more syntactic help with exceptions, rather than more
>> syntax emphasizing None's uniqueness.
>>
>
> Me too, but could you provide me an example where try-except approach is
> more readable when trying to chain attribute lookups (like in the database
> book-publisher-owner example I have provided before).
>

I'd echo the others' examples, taking inspiration from PEP 463.


> If the motivation for this operator is chained lookups, how about adding a
>> feature to the operator module, first? It seems natural to add a
>> keyword-only argument to `attrgetter`, and it's a lighter touch than
>> implementing a new operator. If use becomes widespread, that gives more
>> weight to PEP 505.
>>
>> def attrgetter(*attrs, none_aware=False)
>>
>> https://docs.python.org/3/library/operator.html#operator.attrgetter
>>
>
> I remember using inhouse solution like this at a certain workplace - a
> method accepting list of string arguments and an object, returning the
> value being the result of chained attribute access. And it worked all
> right. The problem I have with such approaches is that the name of the
> attrs are passed as strings.
>

I understand the preference for attributes over strings, but many of the
none-aware examples use keys and indices. If JSON is the main culprit for
deeply nested structures, then you're already using strings and not
attributes. Adding features to `operator` wouldn't preclude accepting PEP
505, so why not get started with a less controversial change that provides
much of the value?

If PEP 505 is accepted, it would need support in the `operator` module.
Might as well design that aspect of the implementation now.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/WIKNWUS2W7LOIHAM3FU4FY2SFQRD3K7D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 505 (None-aware operators) for Python 3.11

[Piotr Waszkiewicz]

Hi,

On Wed, Oct 20, 2021 at 2:33 AM Michael Selik  wrote:

> In case it saves anyone a couple clicks:
> https://www.python.org/dev/peps/pep-0463/
>
> I also prefer more syntactic help with exceptions, rather than more syntax
> emphasizing None's uniqueness.
>

Me too, but could you provide me an example where try-except approach is
more readable when trying to chain attribute lookups (like in the database
book-publisher-owner example I have provided before).


>
> None and its ilk often conflate too many qualities. For example, is it
> missing because it doesn't exist, it never existed, or because we never
> received a value, despite knowing it must exist?
>

I think that a code where those three qualities happen all at once is in
fact a bad code design. But the truth is that None has been and will be
used to denote each of those qualities when writing a code because of its
ease of use in a normal workflow, where time constraints are tight.


> The languages SAS and R support at least 27 varieties of NA, allowing
> un-tagged, and tagged with the letters A-Z to help someone create
> distinctions between different kinds of nothingness. IEEE-754 allows about
> 16 million possible NaNs, which I believe was intended to allow floating
> point instructions to pass error messages along.
>
> If the motivation for this operator is chained lookups, how about adding a
> feature to the operator module, first? It seems natural to add a
> keyword-only argument to `attrgetter`, and it's a lighter touch than
> implementing a new operator. If use becomes widespread, that gives more
> weight to PEP 505.
>
> def attrgetter(*attrs, none_aware=False)
>
> https://docs.python.org/3/library/operator.html#operator.attrgetter
>
> Apologies if attrgetter has already been discussed. I didn't see mention
> of it in PEP 505.
>

I remember using inhouse solution like this at a certain workplace - a
method accepting list of string arguments and an object, returning the
value being the result of chained attribute access. And it worked all right.
The problem I have with such approaches is that the name of the attrs are
passed as strings. This makes it less practical in day-to-day situations
when writing code. Automated class refactors, available in most of the IDEs
also seem to not be able to support such field renames, whereas attribute
lookup via `.` is properly detected. The same goes with context help
available in most IDEs - you'll get no useful information when writing a
string (IDE does not know if you are trying to write a name of the class'
field), whereas when using the dot this works. And it'll probably be
implemented for maybe-dot operator in no time looking at other languages'
support.


> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/R5NO5H5BCEIGEUCLPRE7WW5AEG5MRX3Q/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/7YUEB6KNXARSN6ULNULF7S7NTWJ73PW3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Type annotations, PEP 649 and PEP 563

[Thomas Wouters]

(For visibility, posted both to python-dev and Discourse.)

Over the last couple of months, ever since delaying PEP 563’s default
change in 3.10
,
the Steering Council has been discussing and deliberating over PEP 563
(Postponed Evaluation of Annotations)
, PEP 649 (Deferred Evaluation
Of Annotations Using Descriptors)
, and type annotations in
general. We haven’t made a decision yet, but we want to give everyone an
update on where we’re at, what we’re thinking and why, see if there’s
consensus on any of this, and perhaps restart the discussion around the
options.

First off, as Barry already mentioned in a different thread
,
the SC does not want to see type annotations as separate from the Python
language. We don’t think it would be good to have the syntax or the
semantics diverge, primarily because we don’t think users would see them as
separate. Any divergence would be hard to explain in documentation, hard to
reason about when reading code, and hard to delineate, to describe what is
allowed where. There’s a lot of nuance in this position (it doesn’t
necessarily mean that all valid syntax for typing uses has to have sensible
semantics for non-typing uses), but we’ll be working on something to
clarify that more, later.

We also believe that the runtime uses of type annotations, which PEP 563
didn’t really anticipate, are valid uses that Python should support. If
function annotations and type annotations had evolved differently, such as
being strings from the start, PEP 563 might have been sufficient. It’s
clear runtime uses of type annotations serve a real, sensible purpose, and
Python benefits from supporting them.

By and large, the SC views PEP 649 as a better way forward. If PEP 563 had
never come along, it would be a fairly easy decision to accept PEP 649. We
are still inclined to accept PEP 649. That would leave the consideration
about what to do with PEP 563 and existing from __future__ import
annotations directives. As far as we can tell, there are two reasons for
code to want to use PEP 563: being able to conveniently refer to names that
aren’t available until later in the code (i.e. forward references), and
reducing the overhead of type annotations. If PEP 649 satisfies all of the
objectives of PEP 563, is there a reason to keep supporting PEP 563’s
stringified annotations?  Are there any practical, real uses of stringified
annotations that would not be served by PEP 649's deferred annotations?

If we no longer need to support PEP 563, can we simply make from __future__
import annotations enable PEP 649? We still may want a new future import
for PEP 649 as a transitory measure, but we could make PEP 563's future
import mean the same thing, without actually stringifying annotations. (The
reason to do this would be to reduce the combinatorial growth of the
support matrix, and to simplify the implementation of the parser.) This
would affect code that expects annotations to always be strings, but such
code would have to be poking directly at function objects (the
__annotations__ attribute), instead of using the advertised ways of getting
at annotations (like typing.get_type_hints()). This question in particular
is one in which the SC isn't yet of one mind.

Keeping the future import and stringified annotations around is certainly
an option, but we’re worried about the cost of the implementation, the
support cost, and the confusion for users (specifically, it is a future
import that will never become the future). If we do keep them, how long
would we keep them around? Should we warn about their use? If we warn about
the future import, is the noise and confusion this generates going to be
worth it? If we don't warn about them, how will we ever be able to turn
them off?

One thing we’re thinking of specifically for the future import, and for
other deprecations in Python, is to revisit the deprecation and warning
policy. We think it’s pretty clear that the policy we have right now
doesn’t exactly work. We used to have noisy DeprecationWarnings, which were
confusing to end users when they were not in direct control of the code. We
now have silent-by-default DeprecationWarnings, where the expectation is
that test frameworks surface these warnings. This avoids the problem of end
users being confused, but leaves the problem of the code’s dependencies
triggering the warning, and thus still warns users (developers) not
necessarily in a position to fix the problem, which in turn leads to them
silencing the warning and moving on. We need a better way to reach the
users in a position to update the code.

One idea is to rely on linters and IDEs to provide this signal, possibly
with a clear upgrade path for the code (e.g. a 

[Python-Dev] Re: compiled python3.10 is unable to find _ssl

[Christian Heimes]

On 20/10/2021 09.43, Robin Becker wrote:

On 19/10/2021 16:45, Christian Heimes wrote:
We use the standard AX_CHECK_OPENSSL() m4 macro from autoconf-archive 
to detect OpenSSL. The macro uses pkg-config to detect OpenSSL. It 
doesn't check for specific version, though. We don't want to prevent 
people with 
outdated OpenSSL or LibreSSL from building Python without ssl support.


Christian
I would not suggest that, but if the test at the end of the configure 
had said it can compile against openssl, but that a wrong or 
insufficient information would prevent building _ssl then a lot of 
cycles could be saved.


I suppose that might be impossible as the test might need to actually 
try and build _ssl.so.


That's an excellent idea!

I can add an autoconf test that checks the most important and newish 
API. The check won't catch all problems, but it will detect most common 
cases of missing APIs in headers and libraries.


PR: https://github.com/python/cpython/pull/29088

Christian

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/DNXIGO5FNCEQH527UK2SFLHTI6THLVQX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 670: Convert macros to functions in the Python C API

[Petr Viktorin]

On 20. 10. 21 3:15, Victor Stinner wrote:

Extra info that I didn't put in the PEP to keep the PEP short.

Since Python 3.8, multiple macros have already been converted,
including Py_INCREF() and Py_TYPE() which are very commonly used and
so matter for Python performance.

Macros converted to static inline functions:

* Py_INCREF(), Py_DECREF(), Py_XINCREF(), Py_XDECREF(): Python 3.8
* PyObject_INIT(), PyObject_INIT_VAR(): Python 3.8
* Private functions: _PyObject_GC_TRACK(), _PyObject_GC_UNTRACK(),
_Py_Dealloc(): Python 3.8
* Py_REFCNT(): Python 3.10
* Py_TYPE(), Py_SIZE(): Python 3.11

Macros converted to regular functions in Python 3.9:

* PyIndex_Check()
* PyObject_CheckBuffer()
* PyObject_GET_WEAKREFS_LISTPTR()
* PyObject_IS_GC()
* PyObject_NEW(): alias to PyObject_New()
* PyObject_NEW_VAR(): alias to PyObjectVar_New()

To keep best performances on Python built without LTO, fast private
variants were added as static inline functions to the internal C API:

* _PyIndex_Check()
* _PyObject_IS_GC()
* _PyType_HasFeature()
* _PyType_IS_GC()

--

Many of these changes have been made to prepare the C API to make
these structure opaque:

* PyObject: https://bugs.python.org/issue39573
* PyTypeObject: https://bugs.python.org/issue40170

Don't access structure members at the ABI level, but abstract them
through a function call.

Some functions are still static inline functions (and so still access
structure members at the ABI level), since the performance impact of
converting them to regular functions was not measured yet.


I think this info should be in the PEP.

If the PEP is rejected, would all these previous changes need to be 
reverted? Or just the ones done in 3.11?


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FQB2Z3A757SUTOCMAWB3BFKTP5ISQJWS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: compiled python3.10 is unable to find _ssl

[Robin Becker]

On 19/10/2021 16:45, Christian Heimes wrote:
We use the standard AX_CHECK_OPENSSL() m4 macro from autoconf-archive to detect OpenSSL. The macro uses pkg-config to 
detect OpenSSL. It doesn't check for specific version, though. We don't want to prevent people with 
outdated OpenSSL or LibreSSL from building Python without ssl support.


Christian
I would not suggest that, but if the test at the end of the configure had said it can compile against openssl, but that 
a wrong or insufficient information would prevent building _ssl then a lot of cycles could be saved.


I suppose that might be impossible as the test might need to actually try and 
build _ssl.so.
--
Robin Becker
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/EOQEXMWZFD32BXRHQZ32ULI4NYFTLFJN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 505 (None-aware operators) for Python 3.11

[Rob Cliffe via Python-Dev]

Data point: I find all the examples in PEP 505 less readable using the 
proposed new operators.
Trying to explain why: The syntax feels *too* compact (Perl-like?) - 
when reading it, every time you see a None-aware operator (*if* you 
notice it), you have to jerk to a halt and say, "Whoa!  What's going on 
here?".
I have some sympathy with this use case of exploring a nested-dict (JSON 
derived?) structure, but this can be written as


try:
    config.get("handler").get("parameters").get("y")
except Attribute Error:
    # Handle missing value.

or something similar (I haven't tested it).  Even using the new 
operators, in a realistic case you would probably have to test if the 
result is None and take different action accordingly.

Best wishes
Rob Cliffe

On 18/10/2021 19:26, Guido van Rossum wrote:
On Mon, Oct 18, 2021 at 9:35 AM Paul Bryan > wrote:


NoneType is just another type, and in type checking scenarios
should be expressed with `Optional[type]` or more preferably in
the future `type | None`; `None` is not a non-value. Assuming what
I just wrote is true, I don't get what the basis of this thread
is; what am I missing?


To me the thread isn't about type checking. It is about APIs that are 
built into the language that special-case None, in particular 
dict.get(). In certain cases, encountered commonly in certain styles 
of coding (data science? web programming?), users encounter data that 
is structured as dicts nested multiple levels. This is often out of 
the user's control, as the dict is returned by reading a JSON value 
whose structure is controlled by some other framework (often not 
specific to Python).


For example, if we have a config structure like this:

config = {
  "timeout": 0.1,
  "handler: {
    "timeout-override": 0.4,
    "method-name": "plot",
    "parameters": {
  "x": 10,
  "y": "auto",
    }
  }
}

where the convention is that keys at any level may be omitted 
altogether and config itself may be NOne, then to safely access the 
value of config["handler"]["parameters"]["y"] we would have to write


y = None  # Default
if config is not None:
  handler = config.get("handler")
  if handler is not None:
    parameters = handler.get("parameters")
    if parameters is not None:
  y = parameters.get("y")

This kind of pattern (and all the various other ways of writing it, 
e.g. using the walrus or passing {} as the second argument to 
dict.get()) can be *very* common if that's the kind of data you're 
given and that's the kind of app you have to write, and you can't 
control the format of the data.


Using ?. this can be written as

y = config?.get("handler")?.get("parameters")?.get("y")

More examples are in PEP 505 itself, see 
https://www.python.org/dev/peps/pep-0505/#examples 



--
--Guido van Rossum (python.org/~guido )
/Pronouns: he/him //(why is my pronoun here?)/ 



___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GKOYEMS7RFHTRPDJ23RAHBBNTFDXKGFJ/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/5PQUBVNHL2FHO5WRAYNU3NCULWKO4FTN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: PEP 505 (None-aware operators) for Python 3.11

[Rob Cliffe via Python-Dev]

This is very reminiscent of the (rejected) PEP 463, Exception-catching 
expressions (which I still hope will be resurrected some day).  It would 
allow you to write

    y = (config["handler"]["parameters"]["y"] except KeyError: None)
(possibly the parentheses might not be required) which IMO is even 
better; `except` makes the intent clearer than `with`.

Best wishes
Rob Cliffe

On 20/10/2021 03:05, h.vetin...@gmx.com wrote:

Baptiste Carvello wrote:

y = config["handler"]["parameters"]["y"] with KeyError as None

I love the look of this! While it doesn't address everything that PEP505 does, 
that's IMO a good thing, because - as other people mentioned already - None 
does too many things already.

On another note, the whole discussion reminds me of wg21.link/p0798, which is 
about adding something quite similar to C++ (was accepted into C++23), and 
contains a decent overview of how other languages solve the same thing.

Even though the approach there is probably not applicable (as all python types are 
implicitly `Optional` in the sense of "can be None", i.e. that API would have 
to be added all the way down to `object`), it's still ironic that C++'s `and_then` looks 
more pythonic than what's proposed here.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6JD5VGTVE2TVHUMU7OWIYT7QJEKAGRI7/
Code of Conduct: http://python.org/psf/codeofconduct/


___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/FXOAXI72DN355PD3RKQOMAELA5XHOGKN/
Code of Conduct: http://python.org/psf/codeofconduct/