[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-12 Thread Chris Angelico
On Wed, Jan 13, 2021 at 6:11 PM Ethan Furman wrote: > > On 1/12/21 10:37 PM, Chris Angelico wrote: > > On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano wrote: > >> > >> On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote: > >> > >>> That'd leave open > >>> the option for "foo() if x

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Paul Sokolovsky
Hello, On Wed, 13 Jan 2021 05:04:36 - "Jim J. Jewett" wrote: > Paul Sokolovsky wrote: > > Ok, let's take "module attribute" as an example. Why do you think > > there's anything wrong with this code: > > == > > import config > > from .types import * > > if config.SUPPORT_BIGINT: > >

[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-12 Thread Ethan Furman
On 1/12/21 10:37 PM, Chris Angelico wrote: On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano wrote: On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote: That'd leave open the option for "foo() if x else foo()" to be optimized down to just "foo()", although I don't think that

[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-12 Thread Chris Angelico
On Wed, Jan 13, 2021 at 5:05 PM Steven D'Aprano wrote: > > On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote: > > > That'd leave open > > the option for "foo() if x else foo()" to be optimized down to just > > "foo()", although I don't think that particular one is needed. > > That

[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-12 Thread Steven D'Aprano
On Wed, Jan 13, 2021 at 04:47:06AM +1100, Chris Angelico wrote: > That'd leave open > the option for "foo() if x else foo()" to be optimized down to just > "foo()", although I don't think that particular one is needed. That would be an unsafe optimization. Not all objets are representable as

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-12 Thread Greg Ewing
On 13/01/21 3:31 pm, Larry Hastings wrote: Let's say we put those behind a from __future__ import.  Now we're gonna write library code that examines annotations. A user passes in a class and asks us to examine its annotations.  The old semantics might be active on it, or the new ones.  How

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Jim J. Jewett
Paul Sokolovsky wrote: > Ok, let's take "module attribute" as an example. Why do you think > there's anything wrong with this code: > == > import config > from .types import * > if config.SUPPORT_BIGINT: > var: bigint = 1 > else: > var: int64 = 1 "Wrong" is too strong, but it would be

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-12 Thread Paul Bryan via Python-Dev
On Tue, 2021-01-12 at 20:09 -0800, Guido van Rossum wrote: > On Tue, Jan 12, 2021 at 8:00 PM Brett Cannon > wrote: > > > > > * It turns a None annotation into type(None).  Which means now > > > you > > > can't tell the difference between "None" and "type(None)". > > > > > Huh, I wasn't aware of

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-12 Thread Guido van Rossum
On Tue, Jan 12, 2021 at 8:00 PM Brett Cannon wrote: > > >>- It turns a None annotation into type(None). Which means now you >>can't tell the difference between "None" and "type(None)". >> >> Huh, I wasn't aware of that. > This has tripped up many people. Maybe we should just bite the

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-12 Thread Brett Cannon
On Tue, Jan 12, 2021 at 6:31 PM Larry Hastings wrote: > > On 1/12/21 5:28 PM, Brett Cannon wrote: > > The other thing to keep in mind is we are talking about every module, > class, and function getting 64 bytes ... which I bet isn't that much. > > Actually it's only every module and class.

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-12 Thread Guido van Rossum
On Tue, Jan 12, 2021 at 6:35 PM Larry Hastings wrote: > > On 1/12/21 5:28 PM, Brett Cannon wrote: > > The other thing to keep in mind is we are talking about every module, > class, and function getting 64 bytes ... which I bet isn't that much. > > Actually it's only every module and class.

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-12 Thread Larry Hastings
On 1/12/21 5:28 PM, Brett Cannon wrote: The other thing to keep in mind is we are talking about every module, class, and function getting 64 bytes ... which I bet isn't that much. Actually it's only every module and class.  Functions don't have this problem because they've always stored

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Inada Naoki
On Wed, Jan 13, 2021 at 1:47 AM Larry Hastings wrote: > > On 1/11/21 5:33 PM, Inada Naoki wrote: > > Note that PEP 563 semantics allows more efficient implementation. > Annotation is just a single constant tuple, not a dict. > We already have the efficient implementation for Python 3.10. > > The

[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-12 Thread Brett Cannon
On Tue, Jan 12, 2021 at 9:51 AM Chris Angelico wrote: > On Wed, Jan 13, 2021 at 4:24 AM Richard Damon > wrote: > > > > On 1/12/21 10:53 AM, Mark Shannon wrote: > > > Hi everyone, > > > > > > Should the optimizer eliminate tests that it can prove have no effect > > > on the control flow of the

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-12 Thread Brett Cannon
On Mon, Jan 11, 2021 at 5:57 PM Larry Hastings wrote: > > On 1/11/21 5:05 PM, Greg Ewing wrote: > > On 12/01/21 6:22 am, Larry Hastings wrote: > > * The language will set __annotations__ to a dict if the object has > >annotations, or None if it has no annotations. > > > That sounds

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Larry Hastings
On 1/12/21 3:53 PM, Greg Ewing wrote: On 13/01/21 5:47 am, Larry Hastings wrote: instead of the compiler storing a code object as the annotations argument to MAKE_FUNCTION, store a tuple containing the fields you'd need to /recreate/ the code object at runtime--bytecode, lnotab, names,

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-12 Thread Senthil Kumaran
On Wed, Jan 13, 2021 at 02:53:30AM +0300, Ivan Pozdeev via Python-Dev wrote: > > I support keeping same Sphinx version across all the supported python > > versions. > > This is not a sustainable route since this way, there's no way to change the > version at all. > By supported, I mean the

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-12 Thread Ivan Pozdeev via Python-Dev
On 13.01.2021 2:47, Senthil Kumaran wrote: On Wed, Jan 13, 2021 at 12:28:42AM +0100, Victor Stinner wrote: The alternative is to keep Sphinx 2 support, use strip_signature_backslash and don't use :no-trim-doctest-flags: ? +1. :no-trim-doctest-flags: was introduced to python docs only

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Greg Ewing
On 13/01/21 5:47 am, Larry Hastings wrote: instead of the compiler storing a code object as the annotations argument to MAKE_FUNCTION, store a tuple containing the fields you'd need to /recreate/ the code object at runtime--bytecode, lnotab, names, consts, etc. Would that tuple be

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-12 Thread Miro Hrončok
On 13. 01. 21 0:28, Victor Stinner wrote: I looked at Sphinx and Python versions of Debian, Ubuntu and Fedora: https://bugs.python.org/issue42843#msg384963 In my list, there is only Debian Buster (stable) which doesn't have Sphinx 3 yet. It uses Python 3.7 and so would not be affected by Python

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Greg Ewing
On 13/01/21 6:54 am, Larry Hastings wrote: Note that this only works in the current version of the PEP / prototype, where annotations are strictly evaluated in module scope.  If we start supporting closures, those need "cell" objects, which IIUC can't be marshalled. The cells belong to the

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-12 Thread Senthil Kumaran
On Wed, Jan 13, 2021 at 12:28:42AM +0100, Victor Stinner wrote: > The alternative is to keep Sphinx 2 support, use > strip_signature_backslash and don't use :no-trim-doctest-flags: ? +1. :no-trim-doctest-flags: was introduced to python docs only recently, so not using that is a reasonable

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-12 Thread Victor Stinner
Since documentation changes are backported to 3.8 and 3.9 stable branches, if we increase the minimum required Sphinx version in master, I would prefer to also increase it in 3.8 and 3.9 branches. I would prefer to not have to check manually if a doc backport PR is still compatible with Sphinx 2

[Python-Dev] Re: Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-12 Thread Senthil Kumaran
On Tue, Jan 12, 2021 at 08:38:17PM +, Julien Palard via Python-Dev wrote: > - Some functions declarations are lacking a backslash, like >print(*objects, sep=' ', end='n', ... > > Which is bad. Wouldn't this a bug with Sphinx? Why would that be special cased with a flag

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Guido van Rossum
On Tue, Jan 12, 2021 at 11:31 AM Jim J. Jewett wrote: > Larry Hastings wrote: > > The control-flow exclusion is for /module//attribute/ or /class > > attribute/ annotations: > > class C: > >if random.random() > 0.5: > > my_attr:int=3 > >else: > > my_attr2:float=3.5 > > That

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Larry Hastings
On 1/12/21 12:16 PM, Paul Bryan wrote: On Tue, 2021-01-12 at 09:54 -0800, Larry Hastings wrote: Note that this only works in the current version of the PEP / prototype, where annotations are strictly evaluated in module scope.  If we start supporting closures, those need "cell" objects,

[Python-Dev] Bumping minimum Sphinx version to 3.2 for cpython 3.10?

2021-01-12 Thread Julien Palard via Python-Dev
During the development of cpython 3.10, Sphinx was bumped to 3.2.1. Problem is Sphinx 3 have some incompatibilities with Sphinx 2, some that we could work around, some are bit harder, so we may need to bump `needs_sphinx = '3.2'` (currently it is 1.8). I found two incompatibilities: - We're

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Paul Sokolovsky
Hello, On Mon, 11 Jan 2021 13:44:45 -0800 Larry Hastings wrote: > The control-flow exclusion is for /module//attribute/ or /class > attribute/ annotations: > > class C: >   if random.random() > 0.5: >     my_attr:int=3 >   else: >     my_attr2:float=3.5 Ok, let's take

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Paul Bryan via Python-Dev
On Tue, 2021-01-12 at 09:54 -0800, Larry Hastings wrote: > Note that this only works in the current version of the PEP / > prototype, where annotations are strictly evaluated in module scope.  > If we start supporting closures, those need "cell" objects, which > IIUC can't be marshalled. Since the

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Petr Viktorin
On 1/12/21 8:23 PM, Neil Schemenauer wrote: On 2021-01-12, Pablo Galindo Salgado wrote: One worry that I have in general with this move is the usage of _PyType_GetModuleByDef to get the type object from the module definition. This normally involves getting a TLS in every instance creation,

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Neil Schemenauer
On 2021-01-12, Petr Viktorin wrote: > Unfortunately, it's not just the creation that needs to be changed. > You also need to decref Foo_Type somewhere. Add the type to the module dict? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Larry Hastings
On 1/12/21 11:26 AM, Jim J. Jewett wrote: If I understand correctly, the problem is that you can't store multiple alternative annotations on my_attr. Therefore: class C: my_attr:(int if random.random > 0.5 else float) should be OK, because there is only a single annotation.

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Neil Schemenauer
On 2021-01-12, Pablo Galindo Salgado wrote: > One worry that I have in general with this move is the usage of > _PyType_GetModuleByDef to get the type object from the module > definition. This normally involves getting a TLS in every instance > creation, which can impact notably performance for

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Jim J. Jewett
Larry Hastings wrote: > The control-flow exclusion is for /module//attribute/ or /class > attribute/ annotations: > class C: >   if random.random() > 0.5: >     my_attr:int=3 >   else: >     my_attr2:float=3.5 That very example would be helpful in the FAQ, though I understand if you're

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Nelson, Karl E. via Python-Dev
Having used the heap types extensively for JPype, I believe that converting all types too heap types would be a great benefit. There are still minor rough spots in which a static type can do things that heap types cannot (such as you can derive a type which is marked final when it is static

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Antoine Pitrou
On Tue, 12 Jan 2021 18:48:39 + Pablo Galindo Salgado wrote: > One worry that I have in general with this move > is the usage of _PyType_GetModuleByDef to get the type object > from the module definition. This normally involves getting a TLS in every > instance creation, > which can impact

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Petr Viktorin
On 1/12/21 7:48 PM, Pablo Galindo Salgado wrote: One worry that I have in general with this move is the usage of _PyType_GetModuleByDef to get the type object from the module definition. This normally involves getting a TLS in every instance creation, Not TLS, it's walking the MRO. which

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Petr Viktorin
On 1/12/21 7:16 PM, Neil Schemenauer wrote: On 2021-01-12, Victor Stinner wrote: It seems like a safer approach is to continue the work on bpo-40077: "Convert static types to PyType_FromSpec()". I agree that trying to convert static types is a good idea. Another possible bonus might be that

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Pablo Galindo Salgado
One worry that I have in general with this move is the usage of _PyType_GetModuleByDef to get the type object from the module definition. This normally involves getting a TLS in every instance creation, which can impact notably performance for some perf-sensitive types or types that are created a

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Neil Schemenauer
On 2021-01-12, Victor Stinner wrote: > It seems like a safer approach is to continue the work on > bpo-40077: "Convert static types to PyType_FromSpec()". I agree that trying to convert static types is a good idea. Another possible bonus might be that we can gain some performance by integrating

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Guido van Rossum
Given all the effort that get_type_hints() puts into evaluating those strings it seems important to spell out explicitly that they're not special. (IIRC they *are* special in PEP 563.) On Tue, Jan 12, 2021 at 8:56 AM Larry Hastings wrote: > > Yes, PEP 649 is completely agnostic about what

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Larry Hastings
On 1/12/21 8:47 AM, Larry Hastings wrote: It seems possible to create a hybrid of these two approaches! Here's my idea: instead of the compiler storing a code object as the annotations argument to MAKE_FUNCTION, store a tuple containing the fields you'd need to /recreate/ the code object at

[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-12 Thread Chris Angelico
On Wed, Jan 13, 2021 at 4:24 AM Richard Damon wrote: > > On 1/12/21 10:53 AM, Mark Shannon wrote: > > Hi everyone, > > > > Should the optimizer eliminate tests that it can prove have no effect > > on the control flow of the program, even if that may eliminate some > > side effects in __bool__()?

[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-12 Thread Richard Damon
On 1/12/21 10:53 AM, Mark Shannon wrote: > Hi everyone, > > Should the optimizer eliminate tests that it can prove have no effect > on the control flow of the program, even if that may eliminate some > side effects in __bool__()? > > For several years we have converted > >     if a and b: >    

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Larry Hastings
Yes, PEP 649 is completely agnostic about what values you put in as annotations.  You can put in strings, complex objects, expressions--whatever you put in, you get back out later. I'm happy to add some text to the PEP if this needs clarifying; I just thought it was obvious. Cheers,

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Larry Hastings
On 1/11/21 5:33 PM, Inada Naoki wrote: Note that PEP 563 semantics allows more efficient implementation. Annotation is just a single constant tuple, not a dict. We already have the efficient implementation for Python 3.10. The efficient implementation in 3.10 can share tuples. If there are

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

2021-01-12 Thread Larry Hastings
On 1/11/21 5:37 PM, Larry Hastings wrote: I'll have to let people with large code bases speak up about this, but my understanding is that most people would prefer Python to use less memory.  On my 64-bit Linux machine, a code object is 136 bytes, its empty __dict__ is 64 bytes, [...] Oops:

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Petr Viktorin
On 1/12/21 4:34 PM, Antoine Pitrou wrote: On Tue, 12 Jan 2021 15:22:36 +0100 Petr Viktorin wrote: On 1/11/21 5:26 PM, Victor Stinner wrote: Hi, There are multiple PEPs covering heap types. The latest one refers to other PEPs: PEP 630 "Isolating Extension Modules" by Petr Viktorin.

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Petr Viktorin
On 1/12/21 4:09 PM, Victor Stinner wrote: On Tue, Jan 12, 2021 at 3:28 PM Petr Viktorin wrote: If a type is immutable and has no references to heap-allocated objects, it could stay as a static type. The issue is that very many types don't fit that. For example, if some method needs to raise

[Python-Dev] Re: 3.10 change (?) for __bool__

2021-01-12 Thread Mark Shannon
Hi everyone, Should the optimizer eliminate tests that it can prove have no effect on the control flow of the program, even if that may eliminate some side effects in __bool__()? For several years we have converted if a and b: ... to if a: if b: ...

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Antoine Pitrou
On Tue, 12 Jan 2021 15:22:36 +0100 Petr Viktorin wrote: > On 1/11/21 5:26 PM, Victor Stinner wrote: > > Hi, > > > > There are multiple PEPs covering heap types. The latest one refers to > > other PEPs: PEP 630 "Isolating Extension Modules" by Petr Viktorin. > >

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Victor Stinner
On Tue, Jan 12, 2021 at 3:28 PM Petr Viktorin wrote: > If a type is immutable and has no references to heap-allocated objects, > it could stay as a static type. > The issue is that very many types don't fit that. For example, if some > method needs to raise a module-specific exception, that's a

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread encukou
Simon Cross wrote: > We've run into some minor issues with the limitations in PyType_Slot > (https://docs.python.org/3/c-api/type.html#c.PyType_Slot.PyType_Slot.slot) > but we are working around them for the moment. > It would be useful to have some sense of where PyType_FromSpec is > headed --

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

2021-01-12 Thread Petr Viktorin
On 1/11/21 5:26 PM, Victor Stinner wrote: Hi, There are multiple PEPs covering heap types. The latest one refers to other PEPs: PEP 630 "Isolating Extension Modules" by Petr Viktorin. https://www.python.org/dev/peps/pep-0630/#motivation The use case is to embed multiple Python instances

[Python-Dev] Re: Let's Fix Class Annotations -- And Maybe Annotations Generally

2021-01-12 Thread Serhiy Storchaka
12.01.21 03:21, Larry Hastings пише: > I forgot about __slots__!  Yup, it's optional, and you can even delete > it, though after the class is defined I'm not sure how much difference > that makes. It affects pickling if __slotnames__ is not set yet. The latter is set when you pickle or copy an