[Python-Dev] Proto-PEP part 4: The wonderful third option

[Larry Hastings]

Sorry, folks, but I've been busy the last few days--the Language Summit 
is Wednesday, and I had to pack and get myself to SLC for PyCon,   
I'll circle back and read the messages on the existing threads 
tomorrow.  But for now I wanted to post "the wonderful third option" for 
forward class definitions we've been batting around for a couple of days.


The fundamental tension in the proposal: we want to /allocate/ the 
object at "forward class" time so that everyone can take a reference to 
it, but we don't want to /initialize/ the class (e.g. run the class 
body) until "continue class" time.  However, the class might have a 
metaclass with a custom __new__, which would be responsible for 
allocating the object, and that isn't run until after the "class body".  
How do we allocate the class object early while still supporting custom 
metaclass.__new__ calls?


So here's the wonderful third idea.  I'm going to change the syntax and 
semantics a little, again because we were batting them around quite a 
bit, so I'm going to just show you our current thinking.


The general shape of it is the same.  First, we have some sort of 
forward declaration of the class.  I'm going to spell it like this:


   forward class C

just for clarity in the discussion.  Note that this spelling is also viable:

   class C

That is, a "class" statement without parentheses or a colon. (This is 
analogous to how C++ does forward declarations of classes, and it was 
survivable for them.)  Another viable spelling:


   C = ForwardClass()

This spelling is nice because it doesn't add new syntax.  But maybe it's 
less obvious what is going on from a user's perspective.


Whichever spelling we use here, the key idea is that C is bound to a 
"ForwardClass" object.  A "ForwardClass" object is /not/ a class, it's a 
forward declaration of a class.  (I suspect ForwardClass is similar to a 
typing.ForwardRef, though I've never worked with those so I couldn't say 
for sure.)  Anyway, all it really has is a name, and the promise that it 
might get turned into a class someday.  To be explicit about it, 
"isinstance(C, type)" is False.


I'm also going to call instances of ForwardClass "immutable".  C won't 
be immutable forever, but for now you're not permitted to set or change 
attributes of C.



Next we have the "continue" class statement.  I'm going to spell it like 
this:


   continue class C(BaseClass, ..., metaclass=MyMetaclass):
    # class body goes here
    ...

I'll mention other possible spellings later.  The first change I'll 
point out here: we've moved the base classes and the metaclass from the 
"forward" statement to the "continue" statement.  Technically we could 
put them either place if we really cared to.  But moving them here seems 
better, for reasons you'll see in a minute.


Other than that, this "continue class" statement is similar to what I 
(we) proposed before.  For example, here C is an expression, not a name.


Now comes the one thing that we might call a "trick".  The trick: when 
we allocate the ForwardClass instance C, we make it as big as a class 
object can ever get.  (Mark Shannon assures me this is simply "heap 
type", and he knows far more about CPython internals than I ever will.)  
Then, when we get to the "continue class" statement, we convince 
metaclass.__new__ call to reuse this memory, and preserve the reference 
count, but to change the type of the object to "type" (or 
what-have-you).  C has now been changed from a "ForwardClass" object 
into a real type.  (Which almost certainly means C is now mutable.)


These semantics let us preserve the entire existing class creation 
mechanism.  We can call all the same externally-visible steps in the 
same externally-visible order.  We don't add any new dunder methods, we 
don't remove any dunder methods, we don't expose a new dunder attribute 
for users to experiment with.


What mechanism do we use to achieve this?  metaclass.__new__ always has 
to do one of these two things to create the class object: either it 
calls "super().__new__", or what we usually call "three-argument type".  
In both cases, it passes through the **kwargs that it received into the 
super().__new__ call or the three-argument type call.  So the "continue 
class C" statement will internally add a new kwarg: "__forward__ = C".  
If super().__new__ or three-argument type get this kwarg, they won't 
allocate a new object, they'll reuse C.  They'll preserve the current 
reference count, but otherwise overwrite C with all the juicy vitamins 
and healthy minerals packed into a Python class object.


So, technically, this means we could spell the "continue class" step 
like so:


   class C(BaseClass, ..., metaclass=MyMetaClass, __forward__=C):
    ...

Which means that, combined with the "C = ForwardClass()" statement 
above, we could theoretically implement this idea without changing the 
syntax of the language.  And since we already don't have to change the 
underlying 

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Joao S. O. Bueno]

On Tue, Apr 26, 2022 at 12:33 AM Mehdi2277  wrote:

> The forward class annotations would not need need to be processed at
> runtime and could be no-ops.
>
> forward class A:
>   x: int
>   y: list[int]
>
>

So, as I"ve written before, since there seems to be no way out of Writing
Everything Twice
what about:

```
from typing import Protocol

class AProto(Protocol):
 x: int
 y: list[int]

 def distance(self, x: int, y: int) -> list[float]:
 ...

class B:
 value: AProto

class A(AProto):  #  inheriting from the protocol is optional, as per
PEP-544
x: int
y: list[int]

def distance(self, x: int, y: int) -> list[float]:
 # code for distance goes here
 ...
```

The best advantage of this particular bikeshed is that it makes
full use of Guido's Time Machine and is readily usable from Python 3.8
onwards!


A.__annotations__ could be empty. For a more complete example you could
> have,
>
> forward class A:
>   value: B # This annotation is solely for type checker and is not
> actually saved at runtime.
>
> class B:
>   value: A # Real annotation that is saved.
>
> continue class A:
>   value: B # Real annotation that is saved.
>
> The rule would then be that any annotation inside forward class is
> equivalent to a noop. Continue class would still need to have same
> annotation repeated to actually set it at runtime.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/O3HMCIWUM6NGTSCGL7HFRBSDT7A4KFHP/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/3CIYJ43YQ5SCGO7M4LU2P6YCWNDV64DC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proposal to deprecate mailcap

[Brett Cannon]

After talking about this in the SC today, we agreed to deprecate mailcap
under the auspices of PEP 594:
https://github.com/python/peps/commit/701999a91dc5f976c00d5bde1510226ebd9c7822
.

On Thu, Apr 14, 2022 at 11:44 AM Brett Cannon  wrote:

> A CVE has been opened against mailcap (see
> https://github.com/python/cpython/issues/68966 for details). I'm not
> aware of anyone trying to maintain the module and Victor did a search
> online and didn't find any use of the module in the top 5000 projects on
> PyPI (see the issue). The module is also under 300 lines of Python code
> that only  (https://github.com/python/cpython/blob/main/Lib/mailcap.py),
> so vendoring wouldn't be burdensome.
>
> As such, I'm proposing we deprecate mailcap in 3.11 and remove it in 3.13.
> Any explicit objections?
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/I32FOGSHZWPQYFWHXVR7BGQ3IHRD7VJY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Mehdi2277]

The forward class annotations would not need need to be processed at runtime 
and could be no-ops. 

forward class A:
  x: int
  y: list[int]

A.__annotations__ could be empty. For a more complete example you could have,

forward class A:
  value: B # This annotation is solely for type checker and is not actually 
saved at runtime.

class B:
  value: A # Real annotation that is saved.

continue class A:
  value: B # Real annotation that is saved.

The rule would then be that any annotation inside forward class is equivalent 
to a noop. Continue class would still need to have same annotation repeated to 
actually set it at runtime.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/O3HMCIWUM6NGTSCGL7HFRBSDT7A4KFHP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Mehdi2277]

The problem comes solely from runtime introspection of type annotations. Static 
typing does not need this nor do any exceptional cases occur. From 
mypy/pyright/other type checker perspective there is not an issue to solve 
here. dataclasses, pydantic, cattrs, click, and other libraries that inspect 
type annotations at runtime is where cyclic or undefined nature of some 
annotations is causing edge cases. For most users of typing world they 
primarily think of former and not the latter.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/GCNP5YVHC4EY4XLXQ3F5MLMCGM5FPDAQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Rob Cliffe via Python-Dev]

One reason I dislike this whole proposal is that I can see forward 
declarations (FDs) ending up in code that doesn't need them.  This could 
happen if
    (a) The FDs were needed at some point, but then the type 
declarations were taken out.  This could happen with someone modifying 
their own code, or lifting from someone else's code.
    (b) A relative newbie could see FDs in someone else's code and 
assume that they were necessary, or at least desirable, and put 
unnecessary FDs in their own code in imitation.

Result: Code clutter and confusion.
AIUI the problem to be solved is solely related to typing.  Is it not 
possible to solve it purely in the "typing world", rather than letting 
it leak out and "infect" something basic in the core language like how 
classes are declared?  By "core language" I guess I mean "Python without 
typing".

Best wishes
Rob Cliffe
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/F36RCN5GLRIKZJD44LGUFNJ2YQEBUMMZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Eric V. Smith]

On 4/25/2022 9:13 PM, Mehdi2277 wrote:

We could have forward part include all of method signatures/attributes which is 
what a type checker needs. In theory we could do,

forward class Foo:
   x: int
   y: list[str]
   
   def __init__(self, ...)


   def process(self, data: list[float]) -> float: ...

and then later do continue class. If we're willing to write a full header 
equivalent of c++ then I think static typing side will work. It'll be a lot 
more verbose then normal that I'd probably pick other solutions, but I it 
should make it tractable for a type checker.


How would this address the use case presented in the original post?

class A:
    value: B

class B:
    value: A

Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2CPE3IB7TUJKEAY633LS3WM7PRQ7R6L6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Mehdi2277]

We could have forward part include all of method signatures/attributes which is 
what a type checker needs. In theory we could do,

forward class Foo:
  x: int
  y: list[str]
  
  def __init__(self, ...)

  def process(self, data: list[float]) -> float: ...

and then later do continue class. If we're willing to write a full header 
equivalent of c++ then I think static typing side will work. It'll be a lot 
more verbose then normal that I'd probably pick other solutions, but I it 
should make it tractable for a type checker.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/YP6Y7XE7HSKAEQM4KUDSLBBHOEIKE3UT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Eric V. Smith]

On 4/25/2022 1:10 PM, Jim J. Jewett wrote:

(That said, my personal opinion is that this is pretty heavyweight for very 
little gain; why not just create a placeholder class that static analysis tools 
are supposed to recognize as  likely-to-be-replaced later?  And why not just 
use strings giving the expected eventual class name?  It isn't as though the 
analysis can verify whether something actually meets the full intended contract 
before they've also parsed the continuation.)


The experience with PEP 563 shows that string annotations make things 
much more difficult for dataclasses and other users of runtime type 
annotations. The point of PEP 649 and the forward declaration of classes 
is to make it possible to remove string annotations entirely.


Eric

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZO7M5NCQKR7EPHRVUALRSH2RCKMT5YX7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Chris Angelico]

On Tue, 26 Apr 2022 at 10:05, Greg Ewing  wrote:
>
> On 23/04/22 5:44 pm, Chris Angelico wrote:
> > On Sat, 23 Apr 2022 at 15:32, Larry Hastings  wrote:
> >>
> >> Still, it's not the intent of my PEP to condone or facilitate 
> >> monkeypatching.
> >>
> >
> > The only difference is that you call it something
> > different.
>
> To me, monkeypatching means modifying the definition of something
> after the fact in a way that it wasn't designed for.
>
> Here, the modification is fully intended, so it's not monkeypatching.

That's exactly what I mean though: if the only difference between
"monkeypatching" and "not monkeypatching" is whether it was intended,
then the only difference is what you call it.

class Spam: pass

def frobnicate(self): print("Hah")
Spam.frobnicate = frobnicate

Is this monkeypatching? Does it stop being monkeypatching if I put a
comment in the Spam class saying "the frobnicate method will be added
afterwards"?

The ONLY difference here is what you call it. Maybe we shouldn't be
scared of mutating mutable objects?

> Also, as long as you can only 'continue' a class that was
> previously declared 'forward', and only do it once, and can't
> use a forward class until it has been continued, it doesn't
> make monkeypatching any easier than it currently is.
>

Yes, and IMO that's an unfortunate loss of generality for very little
benefit. You can patch in the dunder and then continue any class,
which basically just means you can use this for any monkeypatching you
like - all you have to do is fiddle with a bookkeeping directive. You
have to remove the "please don't monkey around" sign and then you can
monkey around all you like.

So what's the value of the sign?

(There are other aspects to the proposal, like how it interacts with
metaclasses. But for this specific part, I don't see it as beneficial
to restrict the feature. For instance, the dunder could effectively
mean "next time this class gets continued, run its metaclass", but
other than that, continuing any class could be permitted.)

ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/ZSF4IT5XOEH2RHN67KSGWW3L6FBADQB4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Greg Ewing]

On 23/04/22 6:41 pm, Mehdi2277 wrote:
If it's allowed that a forward class may be continued in a different module I do not see how type checker like mypy/pyright could handle that. Classes are generally viewed as closed and fully defined within type checker. 


The way this kind of thing works in a statically typed language is that
the "forward" part of the definition contains all of the interface
information, so the type checker doesn't need to see the "continue"
part -- that only contains implementation.

What's being proposed here is to split the *interface* between two
class statements, and possibly different modules. That's very different.

The nearest thing I can think of in a statically typed language is the
"extension methods" of C#. The way it works there is if you want to
use any extension methods, you have to import both the module defining
the class and the module defining the extension methods.

Maybe here the client code should import the class from the module
containing the "continue class", and that module should import the
one containing the "forward class"? Would that still work to resolve
circularities?

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/BOO3TJ5V3LOXC2MMNFBUWIEGCNMZLKET/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Greg Ewing]

On 23/04/22 5:44 pm, Chris Angelico wrote:

On Sat, 23 Apr 2022 at 15:32, Larry Hastings  wrote:


Still, it's not the intent of my PEP to condone or facilitate monkeypatching.



The only difference is that you call it something
different.


To me, monkeypatching means modifying the definition of something
after the fact in a way that it wasn't designed for.

Here, the modification is fully intended, so it's not monkeypatching.

Also, as long as you can only 'continue' a class that was
previously declared 'forward', and only do it once, and can't
use a forward class until it has been continued, it doesn't
make monkeypatching any easier than it currently is.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/V6VXIX4SWIRMN5GJRCN35M3OMKXQDWEF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Greg Ewing]

On 23/04/22 10:10 pm, Terry Reedy wrote:
'forward class' for an incomplete class is not at all clear to me.  It 
is not clear to me which part of speech you intend it to be: noun, verb, 
adjective, or adverb.  You must have some experience with 'forward' in a 
different context that makes it clearer to you.


Pascal has a "forward" keyword[1] for declaring functions, and the
term is often used for the same concept in other languages even if
it's spelled differently. So I think it would be fine here,
although "def" would also work if you want to reuse an existing
keyword.

[1] Actually it's a context-sensitive pseudo-keyword, which is
an unusual thing to find in Pascal; maybe Wirth added it as an
afterthought?

--
Greg

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/6M4MEEMNAFM6H2ST75XVNNUUSO5MY7MY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Greg Ewing]

On 23/04/22 7:22 pm, Steven D'Aprano wrote:

He said that could be
used as a very primitive form of separation of interface and
implementation, by putting the `forward class` in one module and the
`continue` in another.


But that only works if there is some way to define the interface
of the class's public attributes in the "forward" part without
also supplying their definitions. If I remember rightly the
"forward" part was only going to include the class name and
base classes.

--
Greg
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/SIK3RXO4ESGQLF5KOB6OEROH2MWTWBS6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Python multithreading without the GIL

[Guido van Rossum]

On Mon, Apr 25, 2022 at 2:33 PM Brett Cannon  wrote:

>
>
> On Sat, Apr 23, 2022 at 8:31 AM  wrote:
>
>> Hello all,
>>
>> I am very excited about a future multithreaded Python. I managed to
>> postpone some rewrites in the company I work for Rust/Go, precisely because
>> of the potential to have a Python solution in the medium term.
>>
>> I was wondering. Is Sam Gross' nogil merge being seriously considered by
>> the core Python team?
>>
>
> Yes, although we have no timeline as to when we will make a decision about
> whether we will accept it or not.
>

We haven't even discussed a *process* for how to decide. OTOH, in two days
at the Language Summit at PyCon, Sam will give a presentation to the core
devs present (which is far from all of us, alas).


> The last update we had on the work was Sam was upstreaming the performance
> improvements he made that were not nogil-specific. The nogil work was also
> being updated for the `main` branch. Once that's all done we will probably
> start a serious discussion as to whether we want to accept it.
>

It's possible that I've missed those code reviews, but I haven't seen a
single PR from Sam, nor have there been any messages from him in this forum
or in any other forums I'm monitoring. I'm hoping that the Language Summit
will change this, but I suspect that there aren't that many perf
improvements in Sam's work that are easily separated from the nogil work.
(To be sure, Christian Heimes seems to have made progress with introducing
mimalloc, which is one of Sam's dependencies, but AFAIK that work hasn't
been finished yet.)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/2LGLMS4JQTB2AFZIS25MBITJZ4TQ2WFF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Brett Cannon]

On Sat, Apr 23, 2022 at 2:39 PM Rob Cliffe via Python-Dev <
python-dev@python.org> wrote:

> UGH!
>

I understand you don't like the proposal, but do please try to keep it
civil. It's totally okay to say you don't like something, but I don't think
anyone would take kindly to someone coming up to them in person and going,
"UGH!" in their face about an idea they had.

-Brett


>
> I thought there was a general understanding that when typing was added
> to Python, there would be no impact, or at least minimal impact, on
> people who didn't use it.  (Raises hand.)
> Now we see an(other) instance of intention creep.
> Rob Cliffe
>
> On 23/04/2022 02:13, Larry Hastings wrote:
> >
> >
> > This document is a loose proto-PEP for a new "forward class" /
> > "continue class" syntax.  Keep in mind, the formatting is a mess. If I
> > wind up submitting it as a real PEP I'll be sure to clean it up first.
> >
> >
> > /arry
> >
> > --
> >
> >
> > PEP : Forward declaration of classes
> >
> > Overview
> > 
> >
> > Python currently has one statement to define a class, the `class`
> > statement:
> >
> > ```Python
> > class X():
> > # class body goes here
> > def __init__(self, key):
> > self.key = key
> > ```
> >
> > This single statement declares the class, including its bases and
> > metaclass,
> > and also defines the contents of the class in the "class body".
> >
> > This PEP proposes an additional syntax for declaring a class which splits
> > this work across two statements:
> > * The first statement is `forward class`, which declares the class and
> > binds
> >   the class object.
> > * The second statement is `continue class`, which defines the contents
> >   of the class in the "class body".
> >
> > To be clear: `forward class` creates the official, actual class object.
> > Code that wants to take a reference to the class object may take
> > references
> > to the `forward class` declared class, and interact with it as normal.
> > However, a class created by `forward class` can't be *instantiated*
> > until after the matching `continue class` statement finishes.
> >
> > Defining class `X` from the previous example using this new syntax
> > would read
> > as follows:
> >
> > ```
> > forward class X()
> >
> > continue class X:
> > # class body goes here
> > def __init__(self, key):
> > self.key = key
> > ```
> >
> > This PEP does not propose altering or removing the traditional `class`
> > statement;
> > it would continue to work as before.
> >
> >
> > Rationale
> > -
> >
> > Python programmers have had a minor problem with classes for years:
> > there's
> > no way to have early-bound circular dependencies between objects. If A
> > depends on B, and B depends on A, there's no linear order that allows
> > you to cleanly declare both.
> >
> > Most of the time, the dependencies were in late-binding code, e.g. A
> > refers
> > to B inside a method.  So this was rarely an actual problem at
> > runtime.  When
> > this problem did arise, in code run at definition-time, it was usually
> > only
> > a minor headache and could be easily worked around.
> >
> > But the explosion of static type analysis in Python, particularly with
> > the `typing` module and the `mypy` tool, has made circular
> > definition-time
> > dependencies between classes commonplace--and much harder to solve.
> > Here's
> > one simple example:
> >
> > ```Python
> > class A:
> > value: B
> >
> > class B:
> > value: A
> > ```
> >
> > An attribute of `B` is defined using a type annotation of `A`, and an
> > attribute of `A` is defined using a type annotation of `B`. There's
> > no order to these two definitions that works; either `A` isn't defined
> > yet, or `B` isn't defined yet.
> >
> > Various workarounds and solutions have been proposed to solve this
> > problem,
> > including two PEPs: PEP 563 (automatic stringized annotations) and PEP
> > 649
> > (delayed evaluation of annotations using functions).
> > But nothing so far has been both satisfying and complete; either it
> > is wordy and clumsy to use (manually stringizing annotations), or it
> > added restrictions and caused massive code breakage for runtime use of
> > annotations (PEP 563), or simply didn't solve every problem (PEP 649).
> > This proposed  `forward class` / `continue class` syntax should permit
> > solving *every* forward-reference and circular-reference problem faced
> > in Python, using an elegant and Pythonic new syntax.
> >
> > As a side benefit, `forward class` and `continue class` syntax enables
> > rudimentary separation of "interface" from "implementation", at least for
> > classes.  A user seeking to "hide" the implementation details of their
> > code could put their class definitions in one module, and the
> > implementations of those classes in a different module.
> >
> > This new syntax is not intended to replace the traditional 

[Python-Dev] Re: Python multithreading without the GIL

[Brett Cannon]

On Sat, Apr 23, 2022 at 8:31 AM  wrote:

> Hello all,
>
> I am very excited about a future multithreaded Python. I managed to
> postpone some rewrites in the company I work for Rust/Go, precisely because
> of the potential to have a Python solution in the medium term.
>
> I was wondering. Is Sam Gross' nogil merge being seriously considered by
> the core Python team?
>

Yes, although we have no timeline as to when we will make a decision about
whether we will accept it or not. The last update we had on the work was
Sam was upstreaming the performance improvements he made that were not
nogil-specific. The nogil work was also being updated for the `main`
branch. Once that's all done we will probably start a serious discussion as
to whether we want to accept it.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/KCATAW2AXWFBGYCVN3IQPIKU2QN2GAA3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes

[Jim J. Jewett]

There is an important difference between monkeypatching in general, vs 
monkey-patching an object that was explicitly marked and documented as 
expecting a monkeypatch.

(That said, my personal opinion is that this is pretty heavyweight for very 
little gain; why not just create a placeholder class that static analysis tools 
are supposed to recognize as  likely-to-be-replaced later?  And why not just 
use strings giving the expected eventual class name?  It isn't as though the 
analysis can verify whether something actually meets the full intended contract 
before they've also parsed the continuation.)
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/JCPKY36RLN5WEFET34EHM4SC6STIJIUC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: [Typing-sig] Almost accepting PEP 681 – Data Class Transforms

[Erik De Bonte via Python-Dev]

  *   Erik, could you propose a change to the PEP text?

I just created https://github.com/python/peps/pull/2555 to address these issues.

-Erik
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/W4NJNTXJIH75QKOBTRIYYNSQYN5R4ZRK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Proto-PEP part 2: Alternate implementation proposal for "forward class" using a proxy object

[Petr Viktorin]

On 23. 04. 22 3:15, Larry Hastings wrote:


Here's one alternate idea for how to implement the "forward class" syntax.

The entire point of the "forward class" statement is that it creates
the real actual class object.  But what if it wasn't actually the
"real" class object?  What if it was only a proxy for the real object?

In this scenario, the syntax of "forward object" remains the same.
You define the class's bases and metaclass.  But all "forward class"
does is create a simple, lightweight class proxy object.  This object
has a few built-in dunder values, __name__ etc.  It also allows you
to set attributes, so let's assume (for now) it calls
metaclass.__prepare__ and uses the returned "dict-like object" as
the class proxy object __dict__.

"continue class" internally performs all the rest of the
class-creation machinery.  (Everything except __prepare__, as we
already called that.)  The first step is metaclass.__new__, which
returns the real class object.  "continue class" takes that
object and calls a method on the class proxy object that says
"here's your real class object".  From that moment on, the proxy
becomes a pass-through for the "real" class object, and nobody
ever sees a reference to the "real" class object ever again.
Every interaction with the class proxy object is passed through
to the underlying class object.  __getattribute__ calls on the
proxy look up the attribute in the underlying class object.  If
the object returned is a bound method object, it rebinds that
callable with the class proxy instead, so that the "self" passed
in to methods is the proxy object.  Both base_cls.__init_subclass__
and cls.__init__ see the proxy object during class creation.  As far
as Python user code is concerned, the class proxy *is* the class,
in every way, important or not.


Sadly, I think that if you try to implement this you'll discover a 
fractal of little issues, each requiring another hack to solve.
(But it seems the “main” forward/continue class proposal suffered a 
similar fate, hasn't it?)


I can see lots of possible issues in interaction with C code (which 
Python user code would call).


What would the __class__ attribute hold?
What would C's ob_type be?
What would be in the MROs?
How would Exception subclasses work? (taking `except` as an example of 
something that uses the real inheritance chain rather than 
__isinstance__/__getattribute__ magic)





The upside: this moves all class object creation code into "continue
class" call.  We don't have to replace __new__ with two new calls.

The downside: a dinky overhead to every interaction with a "forward
class" class object and with instances of a "forward class" class
object.


A huge concern: how does this interact with metaclasses implemented
in C?  If you make a method call on a proxy class object, and that
calls a C function from the metaclass, we'd presumably have to pass
in the "real class object", not the proxy class object.  Which means
references to the real class object could leak out somewhere, and
now we have a real-class-object vs proxy-class-object identity crisis.
Is this a real concern?


A possible concern: what if metaclass.__new__ keeps a reference to
the object it created?  Now we have two objects with an identity
crisis.  I don't know if people ever do that.  Fingers crossed that
they don't.  Or maybe we add a new dunder method:

     @special_cased_staticmethod
     metaclass.__bind_proxy__(metaclass, proxy, cls)

This tells the metaclass "bind cls to this proxy object", so
metaclasses that care can update their database or whatever.
The default implementation uses the appropriate mechanism,
whatever it is.

One additional probably-bad idea: in the case where it's just a
normal "class" statement, and we're not binding it to a proxy,
should we call this?

     metaclass.__bind_proxy__(metaclass, None, cls)

The idea there being "if you register the class objects you create,
do the registration in __bind_proxy__, it's always called, and you'll
always know the canonical object in there".  I'm guessing probably not,
in which case we tell metaclasses that track the class objects we
create "go ahead and track the object you return from __new__, but
be prepared to update your tracking info in case we call __bind_proxy__
on you".


A small but awfully complicated wrinkle here: what do we do if the
metaclass implements __del__?  Obviously, we have to call __del__
with the "real" class object, so it can be destroyed properly.
But __del__ might resurrect that object, which means someone took a 
reference to it.




One final note.  Given that, in this scenario, all real class creation
happens in "continue class", we could move the bases and metaclass
declaration down to the "continue class" statement.  The resulting
syntax would look like:

     forward class X

     ...

     continue class X(base1, base2, metaclass=AmazingMeta, 
rocket="booster")


Is that better? worse? doesn't matter?  I don't have an intuition about
it 

[Python-Dev] Re: C API: Move PEP 523 "Adding a frame evaluation API to CPython" private C API to the internal C API

[Victor Stinner]

On Fri, Apr 22, 2022 at 2:52 PM Fabio Zadrozny  wrote:
> Humm, now I'm a bit worried... the approach the debugger is using gets the 
> PyFrameObject that's about to be executed and changes the 
> PyFrameObject.f_code just before the execution so that the new code is 
> executed instead.

You can already modify _PyInterpreterFrame.f_code using the internal C API.

> From what you're saying the PyFrameObject isn't really used anymore 
> (apparently it's substituted by a _PyInterpreterFrame?)... in this case, will 
> this approach still let the debugger patch the code object in the frame 
> before it's actually executed?

There is no public C API to modify the "f_code" attribute of a
PyFrameObject. There is only PyFrame_GetCode() *getter*.

Victor
-- 
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/python-dev@python.org/message/RTBUDAEPTZBTAHEO5LV77MHEH7URJP3J/
Code of Conduct: http://python.org/psf/codeofconduct/