[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Wed, 27 Apr 2022 at 16:04, Greg Ewing wrote: > > On 27/04/22 1:26 pm, Chris Angelico wrote: > > On Wed, 27 Apr 2022 at 11:18, Greg Ewing > > wrote: > >> > >> The proposed feature is analogous to forward declaring a > >> struct in C. Would you call what C does monkeypatching? > >> > > No, because C doesn't have first-class types, much less mutable ones. > > The purpose of a forward declaration, in any language, is to > allow a name to be used before it's fully defined. Mutability > or first-classness of types has nothing to do with it. The ONLY thing you can do in C without the full structure definition is to declare a pointer to it. It has all the functionality of a void pointer (ie basically none), but you know it's a pointer to this undefined type. Python simply doesn't have an equivalent, since there is no concept of typed pointers, but the nearest equivalent would be making it legal in type hints. Python DOES, however, have first-class types. They are real objects and can be examined at run-time. This proposed feature isn't analogous to C's structs, since C types its variables rather than its objects. > > According to this proposal, it is entirely possible to continue a > > class more than once, with just some fiddling with dunders. > > No, it's not. As has been explained upthread, stuff is done > when finalising a class that can't be repeated. >From the OP: On Sat, 23 Apr 2022 at 11:16, Larry Hastings wrote: > Semantics of forward-declared class objects > ... > However, the user isn't permitted to instantiate a forward-declared > class object until after the corresponding `continue class X`. > We ensure this with a new dunder attribute, `__forward__`, > which if present tells the Python runtime that this is a > forward-declared class object. The `continue class` statement > would delete this attribute from the object, after which it > could be instantiated. > > (Users could work around this constraint, or even delete `__forward__` > if they so chose--again, the "consenting adults" rule applies.) > ... > `continue class` may only be run on a class once. > (As Eric V. Smith pointed out in response to an early version of > this proposal, allowing multiple "continue" declarations on the > same class would lead directly to language-condoned monkey-patching.) So unless I'm horrifically misreading this, the only thing stopping you from doing multiple continues is this attribute which says that it's a forward-declared class. Removing this would allow you to work around this constraint, which exists solely to avoid the perception of "language-condoned monkey-patching". The proposal explicitly lays out that a forward-declared class IS a class, and can be used in a number of basic ways (but can't be instantiated). The "continue class" statement removes this flag, but you can mess with that flag and re-continue a class, or instantiate a non-completed class. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7CVJPKQ53K2BOQ4KOOXWLZ7LOX4G2NQH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 27/04/22 1:26 pm, Chris Angelico wrote: On Wed, 27 Apr 2022 at 11:18, Greg Ewing wrote: The proposed feature is analogous to forward declaring a struct in C. Would you call what C does monkeypatching? No, because C doesn't have first-class types, much less mutable ones. The purpose of a forward declaration, in any language, is to allow a name to be used before it's fully defined. Mutability or first-classness of types has nothing to do with it. According to this proposal, it is entirely possible to continue a class more than once, with just some fiddling with dunders. No, it's not. As has been explained upthread, stuff is done when finalising a class that can't be repeated. But what you're saying is that sometimes that's monkeypatching and a bad thing, and other times it's a good thing and not monkeypatching. No, I'm saying that the proposed feature is not monkeypatching because it does not and cannot be used to modify an existing class. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SVZPAEBDIGHR2TADKCOXND4S3RPKXAMY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 27/04/22 11:07 am, Chris Angelico wrote: You're saying that it's somehow different when the original dev intends for it, and that that makes it "not monkeypatching". I dispute that, and I consider that the feature would be helpful whether the original dev meant for it or not. The forward declaration and continuation both have to be specially marked, so you *can't* use it for monkeypatching. Would subclassing therefore become legitimate if the original author intended for it, and be called "method stealing" if it were not? You seem to think I have something against monkeypatching. Personally I don't, but someone else seems to as they objected that they don't want monkeypatching to be encouraged. I was just pointing out that (1) I don't think it's the same thing as monkeypatching and (2) you can't use it to modify an existing class in an ad-hoc manner anyway. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6HOFAR3TKR62YFPUZ4OIBY5DJBNV374V/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Tue, Apr 26, 2022 at 7:24 PM Greg Ewing wrote: > On 27/04/22 2:01 am, Chris Angelico wrote: > > That would be the case if monkeypatching were illegal. Since it's not, > > wherein lies the difference? > > The proposed feature is analogous to forward declaring a > struct in C. Would you call what C does monkeypatching? > It is not analogous; it is a false analogy that obscures the issues with this proposal in Python. A C forward declaration (not to mention the full struct declaration also!) is purely for the compiler; at runtime one can have a pointer to some memory that the compiler expects to be shaped like that struct, but one can never get hold of any runtime value that is “the struct definition itself,” let alone a runtime value that is the nascent forward-declared yet-to-be-completed struct. So clearly there can be no patching of something that never exists at runtime at all. Python is quite different from C in this respect. Classes are runtime objects, and so is the “forward declared class” object. The proposal is for a class object to initially at runtime be the latter, and then later (at some point that is not well defined if the implementation is in a separate module, because global module import ordering is an unstable emergent property of all the imports in the entire codebase) may suddenly, everywhere and all at once, turn into the former. Any given module that imports the forward declared name can have no guarantee when (if ever) that object will magically transform into something that is safe to use. Whether we call it monkeypatching or not is irrelevant. Having global singleton objects change from one thing to a very different thing, at an unpredictable point in time, as a side effect of someone somewhere importing some other module, causes very specific problems in being able to locally reason about code. I think it is more useful to discuss the specific behavior and its consequences than what it is called. Carl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CQ7TAV6TWGEG2HLVY7T46U6JCPESRACR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Wed, 27 Apr 2022 at 11:18, Greg Ewing wrote: > > On 27/04/22 2:01 am, Chris Angelico wrote: > > That would be the case if monkeypatching were illegal. Since it's not, > > wherein lies the difference? > > The proposed feature is analogous to forward declaring a > struct in C. Would you call what C does monkeypatching? > No, because C doesn't have first-class types, much less mutable ones. It doesn't make sense to compare them. The proposed feature is an alternative way to add attributes to an existing class object, which we normally call monkeypatching, but you're saying that sometimes it isn't. I'm still confused as to why you consider there to be a difference. According to this proposal, it is entirely possible to continue a class more than once, with just some fiddling with dunders. But what you're saying is that sometimes that's monkeypatching and a bad thing, and other times it's a good thing and not monkeypatching. I say that it's always monkeypatching, and always a good thing. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5UYWIKP5PUHDRAE6APVHMBWIAVQYZTH3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 27/04/22 2:01 am, Chris Angelico wrote: That would be the case if monkeypatching were illegal. Since it's not, wherein lies the difference? The proposed feature is analogous to forward declaring a struct in C. Would you call what C does monkeypatching? -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5DRJG3CGCLHJSXBD6ILPHROX6LBIDCHK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Wed, 27 Apr 2022 at 08:06, Greg Ewing wrote: > > On 27/04/22 1:04 am, Joao S. O. Bueno wrote: > > MonkeyPatching in Python is not illegal in this sense. > > I'm not suggesting it is. You're seizing on the wrong part > of the analogy. The point is that what you call something > doesn't change its nature. > I agree. So why is it monkeypatching sometimes, and not other times? Why are you distinguishing? To be clear, this is what I'm responding to: On Tue, 26 Apr 2022 at 10:05, Greg Ewing wrote: > To me, monkeypatching means modifying the definition of something > after the fact in a way that it wasn't designed for. > > Here, the modification is fully intended, so it's not monkeypatching. You're saying that it's somehow different when the original dev intends for it, and that that makes it "not monkeypatching". I dispute that, and I consider that the feature would be helpful whether the original dev meant for it or not. Consider what Python would be like if every class had to declare whether it was allowed to be subclassed or not. Would subclassing therefore become legitimate if the original author intended for it, and be called "method stealing" if it were not? Would that materially improve the language? I have frequently subclassed something without the "consent" of the original class's author, because it is a useful feature. Python is a "consenting adults" language. If you do something the original author didn't intend, you might find yourself running into backward compatibility issues, but you shouldn't have the language block you unnecessarily. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2BGUHWOUNJTVETANBVJPCVPQZ43IVA57/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 27/04/22 1:04 am, Joao S. O. Bueno wrote: MonkeyPatching in Python is not illegal in this sense. I'm not suggesting it is. You're seizing on the wrong part of the analogy. The point is that what you call something doesn't change its nature. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WFVAWIKBQF72KW2IHOERP2DIKD5ECO7N/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
Apr 26, 2022 20:32:55 Eric V. Smith : How would runtime consumers of annotations use this? -- Eric On Apr 26, 2022, at 12:05 PM, Thomas Kehrenberg wrote: If the problem is mostly type annotations, then another potential solution would be to make use of .pyi files, which are not hamstrung by circular definitions. The idea would be that type checkers would merge the annotations from .pyi files into the annotations in the corresponding .py file. So: a.py: from b import B class A: value: B b.py: class B: value = None b.pyi: from typing import Optional from a import A class B: value: Optional[A] = ... The pyi files would kind of act like header files that are used in other languages. It would mean that type checkers need to check the .pyi files against the code in the .py files to verify that they're consistent with one another. -thomas ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VWPWN5KWTRPP6VS4PEHJA4SRVMUDU5WR/ Code of Conduct: http://python.org/psf/codeofconduct/ They wouldn't. But I thought PEP 649 solves the runtime problems, and that the remaining problems are with static typing of circular definitions. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LHHE7HPSKNT6UAQGSZPCMU5XJATAARRB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
> On Apr 26, 2022, at 3:05 PM, Thomas Kehrenberg wrote: > > > Apr 26, 2022 20:32:55 Eric V. Smith : > >> How would runtime consumers of annotations use this? >> >> -- >> Eric >> On Apr 26, 2022, at 12:05 PM, Thomas Kehrenberg wrote: >>> >>> If the problem is mostly type annotations, then another potential >>> solution would be to make use of .pyi files, which are not hamstrung by >>> circular definitions. The idea would be that type checkers would merge >>> the annotations from .pyi files into the annotations in the >>> corresponding .py file. >>> >>> So: >>> >>> a.py: >>> >>>from b import B >>> >>>class A: >>>value: B >>> >>> b.py: >>> >>>class B: >>>value = None >>> >>> b.pyi: >>> >>>from typing import Optional >>>from a import A >>> >>>class B: >>>value: Optional[A] = ... >>> >>> The pyi files would kind of act like header files that are used in other >>> languages. It would mean that type checkers need to check the .pyi >>> files against the code in the .py files to verify that they're >>> consistent with one another. >>> >>> -thomas >>> ___ >>> Python-Dev mailing list -- python-dev@python.org >>> To unsubscribe send an email to python-dev-le...@python.org >>> https://mail.python.org/mailman3/lists/python-dev.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-dev@python.org/message/VWPWN5KWTRPP6VS4PEHJA4SRVMUDU5WR/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ > They wouldn't. But I thought PEP 649 solves the runtime problems, > and that the remaining problems are with static typing > of circular definitions. If the class needs access to its own type annotations at runtime (for example, if it’s a dataclass), then the circular reference problem still exists with PEP 649. That’s among the cases we’re trying to resolve. Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SV37AN7HYQNAVIE2BMJ5C5AC77FENTCS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
How would runtime consumers of annotations use this? -- Eric > On Apr 26, 2022, at 12:05 PM, Thomas Kehrenberg wrote: > > If the problem is mostly type annotations, then another potential > solution would be to make use of .pyi files, which are not hamstrung by > circular definitions. The idea would be that type checkers would merge > the annotations from .pyi files into the annotations in the > corresponding .py file. > > So: > > a.py: > >from b import B > >class A: >value: B > > b.py: > >class B: >value = None > > b.pyi: > >from typing import Optional >from a import A > >class B: >value: Optional[A] = ... > > The pyi files would kind of act like header files that are used in other > languages. It would mean that type checkers need to check the .pyi > files against the code in the .py files to verify that they're > consistent with one another. > > -thomas > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VWPWN5KWTRPP6VS4PEHJA4SRVMUDU5WR/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2VNYLOHCXWBUYTWRGRJC6C4LXQVF45MD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
If the problem is mostly type annotations, then another potential solution would be to make use of .pyi files, which are not hamstrung by circular definitions. The idea would be that type checkers would merge the annotations from .pyi files into the annotations in the corresponding .py file. So: a.py: from b import B class A: value: B b.py: class B: value = None b.pyi: from typing import Optional from a import A class B: value: Optional[A] = ... The pyi files would kind of act like header files that are used in other languages. It would mean that type checkers need to check the .pyi files against the code in the .py files to verify that they're consistent with one another. -thomas ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VWPWN5KWTRPP6VS4PEHJA4SRVMUDU5WR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Tue, 26 Apr 2022 at 22:56, Greg Ewing wrote: > > On 26/04/22 12:33 pm, Chris Angelico wrote: > > That's exactly what I mean though: if the only difference between > > "monkeypatching" and "not monkeypatching" is whether it was intended, > > then the only difference is what you call it. > > No, it's not just a matter of what you call it. > > If I lose my keys and have to break into my house, it's not > illegal. But if someone else breaks into my house without my > permission, that is illegal. It doesn't matter if the thief > *calls* it legal, there's still a difference. > That would be the case if monkeypatching were illegal. Since it's not, wherein lies the difference? ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4FZTF4TIQY7CEZCQX4FLKS2JOUWZGQ5P/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Tue, Apr 26, 2022 at 9:55 AM Greg Ewing wrote: > On 26/04/22 12:33 pm, Chris Angelico wrote: > > That's exactly what I mean though: if the only difference between > > "monkeypatching" and "not monkeypatching" is whether it was intended, > > then the only difference is what you call it. > > No, it's not just a matter of what you call it. > > If I lose my keys and have to break into my house, it's not > illegal. But if someone else breaks into my house without my > permission, that is illegal. It doesn't matter if the thief > *calls* it legal, there's still a difference. > MonkeyPatching in Python is not illegal in this sense. As was put in this thread: let's not criminalize people for mutating mutable objects. btw, wether to call certain aspects of one of the options for this proposal "monkeypatching" or not, is a discussion that IMHO is far, far beyond bike-shedding. > > -- > Greg > > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7HG35MGIE2HROG5LFIAIJH5IVGUUU32C/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 26/04/22 12:33 pm, Chris Angelico wrote: That's exactly what I mean though: if the only difference between "monkeypatching" and "not monkeypatching" is whether it was intended, then the only difference is what you call it. No, it's not just a matter of what you call it. If I lose my keys and have to break into my house, it's not illegal. But if someone else breaks into my house without my permission, that is illegal. It doesn't matter if the thief *calls* it legal, there's still a difference. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HE7Z6VDDYN7RE3YIIWUKZDYBBCQCPWEG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Tue, Apr 26, 2022 at 12:33 AM Mehdi2277 wrote: > The forward class annotations would not need need to be processed at > runtime and could be no-ops. > > forward class A: > x: int > y: list[int] > > So, as I"ve written before, since there seems to be no way out of Writing Everything Twice what about: ``` from typing import Protocol class AProto(Protocol): x: int y: list[int] def distance(self, x: int, y: int) -> list[float]: ... class B: value: AProto class A(AProto): # inheriting from the protocol is optional, as per PEP-544 x: int y: list[int] def distance(self, x: int, y: int) -> list[float]: # code for distance goes here ... ``` The best advantage of this particular bikeshed is that it makes full use of Guido's Time Machine and is readily usable from Python 3.8 onwards! A.__annotations__ could be empty. For a more complete example you could > have, > > forward class A: > value: B # This annotation is solely for type checker and is not > actually saved at runtime. > > class B: > value: A # Real annotation that is saved. > > continue class A: > value: B # Real annotation that is saved. > > The rule would then be that any annotation inside forward class is > equivalent to a noop. Continue class would still need to have same > annotation repeated to actually set it at runtime. > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/O3HMCIWUM6NGTSCGL7HFRBSDT7A4KFHP/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/3CIYJ43YQ5SCGO7M4LU2P6YCWNDV64DC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
The forward class annotations would not need need to be processed at runtime and could be no-ops. forward class A: x: int y: list[int] A.__annotations__ could be empty. For a more complete example you could have, forward class A: value: B # This annotation is solely for type checker and is not actually saved at runtime. class B: value: A # Real annotation that is saved. continue class A: value: B # Real annotation that is saved. The rule would then be that any annotation inside forward class is equivalent to a noop. Continue class would still need to have same annotation repeated to actually set it at runtime. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/O3HMCIWUM6NGTSCGL7HFRBSDT7A4KFHP/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
The problem comes solely from runtime introspection of type annotations. Static typing does not need this nor do any exceptional cases occur. From mypy/pyright/other type checker perspective there is not an issue to solve here. dataclasses, pydantic, cattrs, click, and other libraries that inspect type annotations at runtime is where cyclic or undefined nature of some annotations is causing edge cases. For most users of typing world they primarily think of former and not the latter. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GCNP5YVHC4EY4XLXQ3F5MLMCGM5FPDAQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
One reason I dislike this whole proposal is that I can see forward declarations (FDs) ending up in code that doesn't need them. This could happen if (a) The FDs were needed at some point, but then the type declarations were taken out. This could happen with someone modifying their own code, or lifting from someone else's code. (b) A relative newbie could see FDs in someone else's code and assume that they were necessary, or at least desirable, and put unnecessary FDs in their own code in imitation. Result: Code clutter and confusion. AIUI the problem to be solved is solely related to typing. Is it not possible to solve it purely in the "typing world", rather than letting it leak out and "infect" something basic in the core language like how classes are declared? By "core language" I guess I mean "Python without typing". Best wishes Rob Cliffe ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/F36RCN5GLRIKZJD44LGUFNJ2YQEBUMMZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/25/2022 9:13 PM, Mehdi2277 wrote: We could have forward part include all of method signatures/attributes which is what a type checker needs. In theory we could do, forward class Foo: x: int y: list[str] def __init__(self, ...) def process(self, data: list[float]) -> float: ... and then later do continue class. If we're willing to write a full header equivalent of c++ then I think static typing side will work. It'll be a lot more verbose then normal that I'd probably pick other solutions, but I it should make it tractable for a type checker. How would this address the use case presented in the original post? class A: value: B class B: value: A Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2CPE3IB7TUJKEAY633LS3WM7PRQ7R6L6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
We could have forward part include all of method signatures/attributes which is what a type checker needs. In theory we could do, forward class Foo: x: int y: list[str] def __init__(self, ...) def process(self, data: list[float]) -> float: ... and then later do continue class. If we're willing to write a full header equivalent of c++ then I think static typing side will work. It'll be a lot more verbose then normal that I'd probably pick other solutions, but I it should make it tractable for a type checker. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YP6Y7XE7HSKAEQM4KUDSLBBHOEIKE3UT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/25/2022 1:10 PM, Jim J. Jewett wrote: (That said, my personal opinion is that this is pretty heavyweight for very little gain; why not just create a placeholder class that static analysis tools are supposed to recognize as likely-to-be-replaced later? And why not just use strings giving the expected eventual class name? It isn't as though the analysis can verify whether something actually meets the full intended contract before they've also parsed the continuation.) The experience with PEP 563 shows that string annotations make things much more difficult for dataclasses and other users of runtime type annotations. The point of PEP 649 and the forward declaration of classes is to make it possible to remove string annotations entirely. Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZO7M5NCQKR7EPHRVUALRSH2RCKMT5YX7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Tue, 26 Apr 2022 at 10:05, Greg Ewing wrote:
>
> On 23/04/22 5:44 pm, Chris Angelico wrote:
> > On Sat, 23 Apr 2022 at 15:32, Larry Hastings wrote:
> >>
> >> Still, it's not the intent of my PEP to condone or facilitate
> >> monkeypatching.
> >>
> >
> > The only difference is that you call it something
> > different.
>
> To me, monkeypatching means modifying the definition of something
> after the fact in a way that it wasn't designed for.
>
> Here, the modification is fully intended, so it's not monkeypatching.
That's exactly what I mean though: if the only difference between
"monkeypatching" and "not monkeypatching" is whether it was intended,
then the only difference is what you call it.
class Spam: pass
def frobnicate(self): print("Hah")
Spam.frobnicate = frobnicate
Is this monkeypatching? Does it stop being monkeypatching if I put a
comment in the Spam class saying "the frobnicate method will be added
afterwards"?
The ONLY difference here is what you call it. Maybe we shouldn't be
scared of mutating mutable objects?
> Also, as long as you can only 'continue' a class that was
> previously declared 'forward', and only do it once, and can't
> use a forward class until it has been continued, it doesn't
> make monkeypatching any easier than it currently is.
>
Yes, and IMO that's an unfortunate loss of generality for very little
benefit. You can patch in the dunder and then continue any class,
which basically just means you can use this for any monkeypatching you
like - all you have to do is fiddle with a bookkeeping directive. You
have to remove the "please don't monkey around" sign and then you can
monkey around all you like.
So what's the value of the sign?
(There are other aspects to the proposal, like how it interacts with
metaclasses. But for this specific part, I don't see it as beneficial
to restrict the feature. For instance, the dunder could effectively
mean "next time this class gets continued, run its metaclass", but
other than that, continuing any class could be permitted.)
ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/ZSF4IT5XOEH2RHN67KSGWW3L6FBADQB4/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 23/04/22 6:41 pm, Mehdi2277 wrote: If it's allowed that a forward class may be continued in a different module I do not see how type checker like mypy/pyright could handle that. Classes are generally viewed as closed and fully defined within type checker. The way this kind of thing works in a statically typed language is that the "forward" part of the definition contains all of the interface information, so the type checker doesn't need to see the "continue" part -- that only contains implementation. What's being proposed here is to split the *interface* between two class statements, and possibly different modules. That's very different. The nearest thing I can think of in a statically typed language is the "extension methods" of C#. The way it works there is if you want to use any extension methods, you have to import both the module defining the class and the module defining the extension methods. Maybe here the client code should import the class from the module containing the "continue class", and that module should import the one containing the "forward class"? Would that still work to resolve circularities? -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/BOO3TJ5V3LOXC2MMNFBUWIEGCNMZLKET/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 23/04/22 5:44 pm, Chris Angelico wrote: On Sat, 23 Apr 2022 at 15:32, Larry Hastings wrote: Still, it's not the intent of my PEP to condone or facilitate monkeypatching. The only difference is that you call it something different. To me, monkeypatching means modifying the definition of something after the fact in a way that it wasn't designed for. Here, the modification is fully intended, so it's not monkeypatching. Also, as long as you can only 'continue' a class that was previously declared 'forward', and only do it once, and can't use a forward class until it has been continued, it doesn't make monkeypatching any easier than it currently is. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/V6VXIX4SWIRMN5GJRCN35M3OMKXQDWEF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 23/04/22 10:10 pm, Terry Reedy wrote: 'forward class' for an incomplete class is not at all clear to me. It is not clear to me which part of speech you intend it to be: noun, verb, adjective, or adverb. You must have some experience with 'forward' in a different context that makes it clearer to you. Pascal has a "forward" keyword[1] for declaring functions, and the term is often used for the same concept in other languages even if it's spelled differently. So I think it would be fine here, although "def" would also work if you want to reuse an existing keyword. [1] Actually it's a context-sensitive pseudo-keyword, which is an unusual thing to find in Pascal; maybe Wirth added it as an afterthought? -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6M4MEEMNAFM6H2ST75XVNNUUSO5MY7MY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 23/04/22 7:22 pm, Steven D'Aprano wrote: He said that could be used as a very primitive form of separation of interface and implementation, by putting the `forward class` in one module and the `continue` in another. But that only works if there is some way to define the interface of the class's public attributes in the "forward" part without also supplying their definitions. If I remember rightly the "forward" part was only going to include the class name and base classes. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SIK3RXO4ESGQLF5KOB6OEROH2MWTWBS6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sat, Apr 23, 2022 at 2:39 PM Rob Cliffe via Python-Dev < python-dev@python.org> wrote: > UGH! > I understand you don't like the proposal, but do please try to keep it civil. It's totally okay to say you don't like something, but I don't think anyone would take kindly to someone coming up to them in person and going, "UGH!" in their face about an idea they had. -Brett > > I thought there was a general understanding that when typing was added > to Python, there would be no impact, or at least minimal impact, on > people who didn't use it. (Raises hand.) > Now we see an(other) instance of intention creep. > Rob Cliffe > > On 23/04/2022 02:13, Larry Hastings wrote: > > > > > > This document is a loose proto-PEP for a new "forward class" / > > "continue class" syntax. Keep in mind, the formatting is a mess. If I > > wind up submitting it as a real PEP I'll be sure to clean it up first. > > > > > > /arry > > > > -- > > > > > > PEP : Forward declaration of classes > > > > Overview > > > > > > Python currently has one statement to define a class, the `class` > > statement: > > > > ```Python > > class X(): > > # class body goes here > > def __init__(self, key): > > self.key = key > > ``` > > > > This single statement declares the class, including its bases and > > metaclass, > > and also defines the contents of the class in the "class body". > > > > This PEP proposes an additional syntax for declaring a class which splits > > this work across two statements: > > * The first statement is `forward class`, which declares the class and > > binds > > the class object. > > * The second statement is `continue class`, which defines the contents > > of the class in the "class body". > > > > To be clear: `forward class` creates the official, actual class object. > > Code that wants to take a reference to the class object may take > > references > > to the `forward class` declared class, and interact with it as normal. > > However, a class created by `forward class` can't be *instantiated* > > until after the matching `continue class` statement finishes. > > > > Defining class `X` from the previous example using this new syntax > > would read > > as follows: > > > > ``` > > forward class X() > > > > continue class X: > > # class body goes here > > def __init__(self, key): > > self.key = key > > ``` > > > > This PEP does not propose altering or removing the traditional `class` > > statement; > > it would continue to work as before. > > > > > > Rationale > > - > > > > Python programmers have had a minor problem with classes for years: > > there's > > no way to have early-bound circular dependencies between objects. If A > > depends on B, and B depends on A, there's no linear order that allows > > you to cleanly declare both. > > > > Most of the time, the dependencies were in late-binding code, e.g. A > > refers > > to B inside a method. So this was rarely an actual problem at > > runtime. When > > this problem did arise, in code run at definition-time, it was usually > > only > > a minor headache and could be easily worked around. > > > > But the explosion of static type analysis in Python, particularly with > > the `typing` module and the `mypy` tool, has made circular > > definition-time > > dependencies between classes commonplace--and much harder to solve. > > Here's > > one simple example: > > > > ```Python > > class A: > > value: B > > > > class B: > > value: A > > ``` > > > > An attribute of `B` is defined using a type annotation of `A`, and an > > attribute of `A` is defined using a type annotation of `B`. There's > > no order to these two definitions that works; either `A` isn't defined > > yet, or `B` isn't defined yet. > > > > Various workarounds and solutions have been proposed to solve this > > problem, > > including two PEPs: PEP 563 (automatic stringized annotations) and PEP > > 649 > > (delayed evaluation of annotations using functions). > > But nothing so far has been both satisfying and complete; either it > > is wordy and clumsy to use (manually stringizing annotations), or it > > added restrictions and caused massive code breakage for runtime use of > > annotations (PEP 563), or simply didn't solve every problem (PEP 649). > > This proposed `forward class` / `continue class` syntax should permit > > solving *every* forward-reference and circular-reference problem faced > > in Python, using an elegant and Pythonic new syntax. > > > > As a side benefit, `forward class` and `continue class` syntax enables > > rudimentary separation of "interface" from "implementation", at least for > > classes. A user seeking to "hide" the implementation details of their > > code could put their class definitions in one module, and the > > implementations of those classes in a different module. > > > > This new syntax is not intended to replace the traditional
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
There is an important difference between monkeypatching in general, vs monkey-patching an object that was explicitly marked and documented as expecting a monkeypatch. (That said, my personal opinion is that this is pretty heavyweight for very little gain; why not just create a placeholder class that static analysis tools are supposed to recognize as likely-to-be-replaced later? And why not just use strings giving the expected eventual class name? It isn't as though the analysis can verify whether something actually meets the full intended contract before they've also parsed the continuation.) ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JCPKY36RLN5WEFET34EHM4SC6STIJIUC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
I guess Carl's messages put an end to this proposal as it is. >From "the language side", I can think of nothing short of actually transforming _in place_ (like it is possible with instances of ordinary classes when you assign then a new "__class__") the "forward referenced object" into the complete classes. This is not "unfeasable", but would be hackish as never before done, and there are practical problems: the final size of the class object itself cannot be known before the final committing of the class-body namespace into the final-namespace (slots, and inheritance of built-in classes with different layouts, etc...) Then, it looks like it only would be remotely practical for a static type checker if the class continuation is on the same file as its header. (I think I did not even consider they being in separate files - of course the name-rebinding would not work in that case) On the other hand, if it is practical to declare as header the complete interface of a class, with the intent of putting the actual methods with code later on, that is already possible, with typing.protocols ; whoever would take the time to split a class in two due to annotations, forward referencing, interface declaration separated from implementation, can just as well declare a protocol, as it exists today, I am not sure if typing.Protocol works nicely with inheritance, in the sense that it could inherit from other protocols or concrete classes, and present everything inherited as part of its interface, if it does not, then _that_ is the improvement needed so that what this proto-pep would allow could be done with them. best regards, js -><- On Sun, Apr 24, 2022 at 1:33 PM Carl Meyer wrote: > On Sun, Apr 24, 2022 at 10:20 AM Joao S. O. Bueno > wrote: > > > > I am not worried about the bikeshed part of which syntax to use - > > and more worried with the possible breaking of a lot of stuff, unless > > we work with creation of a non-identical "forward object" that is > > rebound, as in plain name binding, when the second part > > is declared. I've stated that amidst my ramblings, > > but Nick Coghlan managed to keep it suscint at > > > https://mail.python.org/archives/list/python-dev@python.org/message/DMITVTUIQKJW6RYVOPQXHD54VSYE7QHA/ > > I don't think name rebinding works. That means that if we have > `forward class Bar` in module `foo` and `continue class Bar: ...` in > module `foo.impl`, if module `baz` does `from foo import Bar`, it will > forever have either the forward reference object or the real class, > and which one it has is entirely unpredictable (depends on import > ordering accidents of the rest of the codebase.) If `baz` happens to > be imported before `foo.impl`, the name `Bar` in the `baz` namespace > will never be resolved to the real class, and isn't resolvable to the > real class without some outside intervention. > > > """ > > Something worth considering: whether forward references need to be > > *transparent* at runtime. If they can just be regular ForwardRef objects > > then much of the runtime complexity will go away (at the cost of some > > potentially confusing identity check failures between forward references > > and the actual class objects). > > > > ForwardRef's constructor could also potentially be enhanced to accept a > > "resolve" callable, and a "resolve()" method added to its public API, > > although the extra complexity that would bring may not be worth it. > > """ > > I'm not sure how this `resolve()` method is even possible under the > proposed syntax. If `forward class Bar` and `continue class Bar` are > in different modules, then how can `forward class Bar` (which must > create the "forward reference" object) possibly know which module > `continue class Bar: ...` exists in? How can it know how to resolve > itself? > > Carl > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/US6OOS7OJMEZKQQC456LDRQKC2HRP72P/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sun, Apr 24, 2022 at 10:20 AM Joao S. O. Bueno wrote: > > I am not worried about the bikeshed part of which syntax to use - > and more worried with the possible breaking of a lot of stuff, unless > we work with creation of a non-identical "forward object" that is > rebound, as in plain name binding, when the second part > is declared. I've stated that amidst my ramblings, > but Nick Coghlan managed to keep it suscint at > https://mail.python.org/archives/list/python-dev@python.org/message/DMITVTUIQKJW6RYVOPQXHD54VSYE7QHA/ I don't think name rebinding works. That means that if we have `forward class Bar` in module `foo` and `continue class Bar: ...` in module `foo.impl`, if module `baz` does `from foo import Bar`, it will forever have either the forward reference object or the real class, and which one it has is entirely unpredictable (depends on import ordering accidents of the rest of the codebase.) If `baz` happens to be imported before `foo.impl`, the name `Bar` in the `baz` namespace will never be resolved to the real class, and isn't resolvable to the real class without some outside intervention. > """ > Something worth considering: whether forward references need to be > *transparent* at runtime. If they can just be regular ForwardRef objects > then much of the runtime complexity will go away (at the cost of some > potentially confusing identity check failures between forward references > and the actual class objects). > > ForwardRef's constructor could also potentially be enhanced to accept a > "resolve" callable, and a "resolve()" method added to its public API, > although the extra complexity that would bring may not be worth it. > """ I'm not sure how this `resolve()` method is even possible under the proposed syntax. If `forward class Bar` and `continue class Bar` are in different modules, then how can `forward class Bar` (which must create the "forward reference" object) possibly know which module `continue class Bar: ...` exists in? How can it know how to resolve itself? Carl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WLZRZIMPRST52UMINB5VB57TOIQVTYQH/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
I am not worried about the bikeshed part of which syntax to use - and more worried with the possible breaking of a lot of stuff, unless we work with creation of a non-identical "forward object" that is rebound, as in plain name binding, when the second part is declared. I've stated that amidst my ramblings, but Nick Coghlan managed to keep it suscint at https://mail.python.org/archives/list/python-dev@python.org/message/DMITVTUIQKJW6RYVOPQXHD54VSYE7QHA/ """ Something worth considering: whether forward references need to be *transparent* at runtime. If they can just be regular ForwardRef objects then much of the runtime complexity will go away (at the cost of some potentially confusing identity check failures between forward references and the actual class objects). ForwardRef's constructor could also potentially be enhanced to accept a "resolve" callable, and a "resolve()" method added to its public API, although the extra complexity that would bring may not be worth it. Cheers, Nick. """ For that matter, syntax wise, it could just be: break class XX() ... continue class XX: ... It gains at once no backwards incompatibility, and reuse concepts that are already somewhat "grouped together". On Sun, Apr 24, 2022 at 11:25 AM Eric V. Smith wrote: > On 4/24/2022 5:42 AM, Stephen J. Turnbull wrote: > > What's the use case for arbitrary expressions vs. a (possibly > > qualified) name? A class factory that produces forward declarations? > > Do you have a use case in mind? > > It's: > > x.py: > > --8< > forward class A() > --8< > > x_impl.py > > --8< > import X > > continue class X.A: > # class body here > --8< > > It needs to be an expression because it's not defining a name, it > referring to an existing name. You could use "from X import A" here and > avoid a dotted expression, but it still needs to be an expression > referring to an existing "forward class". Even if you restrict it to not > having dots, it's logically an expression, not a name binding. > > Eric > > > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VV53MWDTYQCHBCAQGXLZ4SFCKUEHPQ2K/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WBXIZLNKWTJGAQLTFJOOPFJ7VCD42U6D/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
Hi Larry, On Sat, Apr 23, 2022 at 1:53 AM Larry Hastings wrote: > But rather than speculate further, perhaps someone who works on one of the > static type analysis checkers will join the discussion and render an informed > opinion about how easy or hard it would be to support "forward class" and > "continue class". I work on a Python static type checker. I think a major issue with this proposal is that (in the separate-modules case) it requires monkey-patching as an import side effect, which is quite hard for both humans and static analysis tools to reason effectively about. Imagine we have a module `foo` that contains `forward class Bar`, a separate module `foo.impl` that contains `continue class Bar: ...`, and then a module `baz` that contains `import foo`. What type of object is `foo.Bar` during the import of `baz`? Will it work for the module body of `baz` to create a singleton instance `my_bar = foo.Bar()`? The answer is that we have no idea. `foo.Bar` might be a non-instantiable "forward class declaration" (or proxy object, in your second variation), or it might be a fully-constituted class. Which one it is depends on accidents of import order anywhere else in the codebase. If any other module happens to have imported `foo.impl` before `baz` is imported, then `foo.Bar` will be the full class. If nothing else has imported `foo.impl`, then it will be a non-instantiable declaration/proxy. This question of import order potentially involves any other module in the codebase, and the only way to reliably answer it is to run the entire program; neither a static type checker nor a reader of the code can reliably answer it in the general case. It will be very easy to write a module `baz` that does `import foo; my_bar = foo.Bar()` and have it semi-accidentally work initially, then later break mysteriously due to a change in imports in a seemingly unrelated part of the codebase, which causes `baz` to now be imported before `foo.impl` is imported, instead of after. There is another big problem for static type checkers with this hypothetical module `baz` that only imports `foo`. The type checker cannot know the shape of the full class `Bar` unless it sees the right `continue Bar: ...` statement. When analyzing `baz`, it can't just go wandering the filesystem aimlessly in hopes of encountering some module with `continue Bar: ...` in it, and hope that's the right one. (Even worse considering it might be `continue snodgrass: ...` or anything else instead.) So this means a type checker must require that any module that imports `Bar` MUST itself import `foo.impl` so the type checker has a chance of understanding what `Bar` actually is. This highlights an important difference between this proposal and languages with real forward declarations. In, say, C++, a forward declaration of a function or class contains the full interface of the function or class, i.e. everything a type checker (or human reader) would need to know in order to know how it can use the function or class. In this proposal, that is not true; lots of critical information about the _interface_ of the class (what methods and attributes does it have, what are the signatures of its methods?) are not knowable without also seeing the "implementation." This proposal does not actually forward declare a class interface; all it declares is the existence of the class (and its inheritance hierarchy.) That's not sufficient information for a type checker or a human reader to make use of the class. Taken together, this means that every single `import foo` in the codebase would have to be accompanied by an `import foo.impl` right next to it. In some cases (if `foo.Bar` is not used in module-level code and we are working around a cycle) it might be safe for the `import foo.impl` to be within an `if TYPE_CHECKING:` block; otherwise it would need to be a real runtime import. But it must always be there. So every single `import foo` in the codebase must now become two or three lines rather than one. There are of course other well-known problems with import-time side effects. All the imports of `foo.impl` in the codebase would exist only for their side effect of "completing" Bar, not because anyone actually uses a name defined in `foo.impl`. Linters would flag these imports as unused, requiring extra cruft to silence the linter. Even worse, these imports would tend to appear unused to human readers, who might remove them and be confused why that breaks the program. All of these import side-effect problems can be resolved by dis-allowing module separation and requiring `forward class` and `continue class` to appear in the same module. But then the proposal no longer helps with resolving inter-module cycles, only intra-module ones. Because of these issues (and others that have been mentioned), I don't think this proposal is a good solution to forward references. I think PEP 649, with some tricks that I've mentioned elsewhere to allow introspecting annotations
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/24/2022 5:42 AM, Stephen J. Turnbull wrote: What's the use case for arbitrary expressions vs. a (possibly qualified) name? A class factory that produces forward declarations? Do you have a use case in mind? It's: x.py: --8< forward class A() --8< x_impl.py --8< import X continue class X.A: # class body here --8< It needs to be an expression because it's not defining a name, it referring to an existing name. You could use "from X import A" here and avoid a dotted expression, but it still needs to be an expression referring to an existing "forward class". Even if you restrict it to not having dots, it's logically an expression, not a name binding. Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VV53MWDTYQCHBCAQGXLZ4SFCKUEHPQ2K/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
Larry Hastings writes: > On 4/22/22 19:36, Terry Reedy wrote: > > How about a 'regular' class statement with a special marker of some > > sort. Example: 'body=None'. > > It's plausible. I take it "body=None" would mean the declaration would > not be permitted to have a colon and a class body. So now we have two > forms of the "class" statement, and which syntax you're using is > controlled by a named parameter argument. > > I think this "body=None" argument changing the syntax of the "class" > statement is clumsy. How about another predefined keyword in class definitions? class ForwardDeclared(*bases, metaclass=Meta, declaration=True): pass AIUI, in your scheme you can't have a body in a forward declaration, so in this one you could error on a non-pass body, or just ignore it. I don't see how you can avoid the 'continue class' statement, if you want the actual argument to be an arbitrary expression that evaluates to an incomplete class. Then the symmetry of the 'forward class' statement is appealing. If you restrict it to be a name, though, you could use class SomeModule.ForwardDeclared(declaration=False): # body goes here I think. What's the use case for arbitrary expressions vs. a (possibly qualified) name? A class factory that produces forward declarations? Do you have a use case in mind? I dunno, but all this seems really complicated, unless you go the proxy object route (which is basically what we have now, except that str is extremely limited as a proxy, you could call it a PINO :-). Sure, there are risks if the temporary thing produced by forward class' escapes, but if both the "thing" and the proxy are limited enough, there will be little incentive to do that. Consenting adults. Just add the FAFO warning label. Add features when they're called for. A separate question: There are going to be metaclasses that define __new__ whose instances cannot be used in forward declarations because the class object can't be created until the class body is seen, and therefore cannot be split into __new_forward__ and __new_continue__, I suppose. Is that true? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SQB63XJWE67NVOT4XT6I3RSU4HS73AGL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sat, Apr 23, 2022 at 7:10 PM Paul Moore wrote: > On Sat, 23 Apr 2022 at 22:42, Rob Cliffe via Python-Dev > wrote: > > > > UGH! > > > > I thought there was a general understanding that when typing was added > > to Python, there would be no impact, or at least minimal impact, on > > people who didn't use it. (Raises hand.) > > Now we see an(other) instance of intention creep. > > To be fair, none of this is needed unless you want to add type > annotations to your code. So it's still perfectly possible to ignore > all of this (which is what I am currently doing). > > What I am concerned about is when users of libraries I write start to > claim that I "need" to add this sort of stuff to my code, so that they > can type check their code that uses mine, and/or they can get tooltips > for my APIs in their IDEs. That's where I think the biggest issue with > a proposal like this arises - the *social* pressure on people to adopt > typing, and all the complexities it adds. But again, that's not > something that's specific to this proposal, it's inherent in the whole > question of whether people add type annotations at all. > > So I'm -1 on this proposal, but just because I fear I may be forced to > use it when I don't need to, rather than because I think it's a bad > idea per se. > > Paul > I stand with Paul here - But as a "heavy user" of metaclasses, as far as that is possible. (less in production code or "non toy projects", but a lot in understanding what goes, answering questions and teaching people on how they should best use (or skip using) a metaclass. I'd say this breaks too much of the current working of class creation for, possibly, too little. And - on a second thought: Was not this the kind of thing that PEP 563 was supposed to fix? Pep 563 is accepted now, I think one may just use a forward declaration in any annotation context: it will be lazily evaluated and the target class will have been created. So, while I understand the problem with forward references, which have been historically workaround with the usage of strings (even long before even Python 3.0, in Django's Models) I believe this is not a good approach So first part: == I'd be comfortable with the statements and blocks as they are if the "forward class" statement would not mess with the metaclass and class creation at all: just create a place-holder the type checkers could then use to find the class later on the file, or later on the code, and then use that class. Has this been thought of? Is there any reason why that would not work, better than it will put some burden on static-type checkers, as opposed to fundamentally modify the way classes are built so that any library featuring any metaclass will have to be divided in incompatible releases "before this pep" and "after this pep"? (and with the harsh inconvenience of needing to have one a part of the code which is often the most complicated 100% rewritten) Second part: = The proposal as it is does not preserve everything that is possible with the current metaclass "__new__", besides complicating things. Moreover, there _are_ similar steps on "breaking metaclass.`__new__` " that would actually be useful in customizing the way classes are created, without sacrificing compatibility (in a sense the custom "metaclass.__new__" would still be called exactly as it is today). If there is any chance this thing will move on, I'd like to detail these ideas (it would be longish, similar in size with the proto-pep text) - and maybe there is a way to reconcile compatibility, without the proposal of splitting "__new__" in two 100% incompatible methods. I will leave the actual amendments to this part of the proposal (which would affect all the inner workings of the metaclasses as exposed) to another message. But I will leave one first, huge, problem here that: ```Python def __new_forward__(metaclass, name, bases, namespace, **kwargs): def __new_continue__(metaclass, cls, **kwargs): ``` These two, as they are, do not allow for some of the things that are possible today: let's supose I want, in a custom metaclass `__new__` method inspect the namespace and creates a "__slots__" declaration based on what already _is_ on the namespace: in "__new_forward__" the namespace is filled in a non deterministic way and should not be inspected , and in "__new_continue__" it had already been processed by (what currently is) "type.__new__" and baked into the proxy-map inside the "cls". I the specific case of creating "__slots__" in a similar way "@dataclass" does, the oportunity is gone. I won't mention the suggestion in the text that "class transformers that would create slots are free to re-create the class object upon its first instantiation" is _rather_ bogus. I can imagine a metaclass "__call__" method that could do that, but then, the classes the instances would be actually using would rather become either have to be kept as an attribute in the
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sat, 23 Apr 2022 at 22:42, Rob Cliffe via Python-Dev wrote: > > UGH! > > I thought there was a general understanding that when typing was added > to Python, there would be no impact, or at least minimal impact, on > people who didn't use it. (Raises hand.) > Now we see an(other) instance of intention creep. To be fair, none of this is needed unless you want to add type annotations to your code. So it's still perfectly possible to ignore all of this (which is what I am currently doing). What I am concerned about is when users of libraries I write start to claim that I "need" to add this sort of stuff to my code, so that they can type check their code that uses mine, and/or they can get tooltips for my APIs in their IDEs. That's where I think the biggest issue with a proposal like this arises - the *social* pressure on people to adopt typing, and all the complexities it adds. But again, that's not something that's specific to this proposal, it's inherent in the whole question of whether people add type annotations at all. So I'm -1 on this proposal, but just because I fear I may be forced to use it when I don't need to, rather than because I think it's a bad idea per se. Paul PS To be open here, I do actually like type annotations in straightforward situations - they have located some bugs in my code for me, and tooltips in VS code *are* nice. What I don't like is not being able to stick to simple stuff and not bother annotating the complicated bits, or being pushed into over-strict annotations because it's too hard (or verbose) to express dynamic, duck-typed constraints. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MOO3FUV3XKSKEGRIAOXSIVBMGPGFXYLV/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
UGH! I thought there was a general understanding that when typing was added to Python, there would be no impact, or at least minimal impact, on people who didn't use it. (Raises hand.) Now we see an(other) instance of intention creep. Rob Cliffe On 23/04/2022 02:13, Larry Hastings wrote: This document is a loose proto-PEP for a new "forward class" / "continue class" syntax. Keep in mind, the formatting is a mess. If I wind up submitting it as a real PEP I'll be sure to clean it up first. /arry -- PEP : Forward declaration of classes Overview Python currently has one statement to define a class, the `class` statement: ```Python class X(): # class body goes here def __init__(self, key): self.key = key ``` This single statement declares the class, including its bases and metaclass, and also defines the contents of the class in the "class body". This PEP proposes an additional syntax for declaring a class which splits this work across two statements: * The first statement is `forward class`, which declares the class and binds the class object. * The second statement is `continue class`, which defines the contents of the class in the "class body". To be clear: `forward class` creates the official, actual class object. Code that wants to take a reference to the class object may take references to the `forward class` declared class, and interact with it as normal. However, a class created by `forward class` can't be *instantiated* until after the matching `continue class` statement finishes. Defining class `X` from the previous example using this new syntax would read as follows: ``` forward class X() continue class X: # class body goes here def __init__(self, key): self.key = key ``` This PEP does not propose altering or removing the traditional `class` statement; it would continue to work as before. Rationale - Python programmers have had a minor problem with classes for years: there's no way to have early-bound circular dependencies between objects. If A depends on B, and B depends on A, there's no linear order that allows you to cleanly declare both. Most of the time, the dependencies were in late-binding code, e.g. A refers to B inside a method. So this was rarely an actual problem at runtime. When this problem did arise, in code run at definition-time, it was usually only a minor headache and could be easily worked around. But the explosion of static type analysis in Python, particularly with the `typing` module and the `mypy` tool, has made circular definition-time dependencies between classes commonplace--and much harder to solve. Here's one simple example: ```Python class A: value: B class B: value: A ``` An attribute of `B` is defined using a type annotation of `A`, and an attribute of `A` is defined using a type annotation of `B`. There's no order to these two definitions that works; either `A` isn't defined yet, or `B` isn't defined yet. Various workarounds and solutions have been proposed to solve this problem, including two PEPs: PEP 563 (automatic stringized annotations) and PEP 649 (delayed evaluation of annotations using functions). But nothing so far has been both satisfying and complete; either it is wordy and clumsy to use (manually stringizing annotations), or it added restrictions and caused massive code breakage for runtime use of annotations (PEP 563), or simply didn't solve every problem (PEP 649). This proposed `forward class` / `continue class` syntax should permit solving *every* forward-reference and circular-reference problem faced in Python, using an elegant and Pythonic new syntax. As a side benefit, `forward class` and `continue class` syntax enables rudimentary separation of "interface" from "implementation", at least for classes. A user seeking to "hide" the implementation details of their code could put their class definitions in one module, and the implementations of those classes in a different module. This new syntax is not intended to replace the traditional `class` declaration syntax in Python. If this PEP were accepted, the `class` statement would still be the preferred mechanism for creating classes in Python; `forward class` should only be used when it confers some specific benefit. Syntax -- The `forward class` statement is the same as the `class` statement, except it doesn't end with a colon and is not followed by an indented block. Without any base classes or metaclass, the `forward class` statement is as follows: ``` forward class X ``` This would declare class `X`. If `X` needs base classes or metaclass, the corresponding `forward class` statement would be as follows, rendered in a sort of "function prototype" manner: ``` forward class X(*bases, metaclass=object, **kwargs) ``` The `continue class` statement is similar to
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/23/22 08:57, Eric V. Smith wrote: On 4/23/2022 9:55 AM, Jelle Zijlstra wrote: However, it doesn't solve the problem for base classes. For example, str is conceptually defined as `class str(Sequence["str"]):`. A forward reference can't make `str` defined when the bases are evaluated, because bases are resolved at the `forward class` stage. Larry's second email "Proto-PEP part 2: Alternate implementation proposal for "forward class" using a proxy object" discusses a possibility to move the bases and metaclasses to the "continue class" stage. It also has the advantage of not changing the behavior of __new__, and I think is in general easier to reason about. Let me expound on Eric's statement for a bit. Moving the base classes and metaclass to the "continue" statement might permit the self-referential "str" definition suggested above: forward class str ... continue class str(Sequence[str]): ... Though I suspect this isn't viable, or at least not today. I'm willing to bet a self-referential definition like this would result in an infinite loop when calculating the MRO. I don't have a strong sense of whether it'd be better to have the base classes and metaclass defined with the "forward" declaration or the "continue" declaration. The analogous syntax in C++ has the base classes defined with the "continue" class. Actually, that reminds me of something I should have mentioned in the proto-PEP. If the "class proxy" version is viable and desirable, and we considered moving the base classes and metaclass down to the "continue" statement, that theoretically means we could drop the "forward" and "continue" keywords entirely. I prefer them, simply because it makes it so explicit what you're reading. But the equivalent syntax in C++ doesn't bother with extra keywords for either the "forward" declaration or the "continue" declaration, and people seem to like it fine. Using that variant of the syntax, the toy example from the PEP would read as follows: class A class B: value: A class A: value: B If you're speed-reading here and wondering "wait, how is this not ambiguous?", note that in the first line of the example, the "class" statement has no colon. Also, it would never have parentheses or decorators. The forward declaration of a class would always be just "class", followed by a name, followed by a newline (or comment). //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/GTOJIZZXON3KND37WSATL3E7Q4KON6YO/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/23/22 06:55, Jelle Zijlstra wrote: So to reiterate, your proposal would be to write this as: forward class B: pass class A: value: B continue class B: value: A Not quite; the "forward class" statement doesn't have a colon or a class body. This would be written as: forward class B class A: value: B continue class B: value: A While the current workaround is: class A: value: "B" class B: value: "A" In this example, with two toy classes in one file, it shouldn't be necessary to quote the annotation in B. So all you need is the quotes around the first annotation: class A: value: "B" class B: value: A I don't think I would write the "forward class" version if I had the choice. It's clunkier and requires more refactoring if I change my mind about whether the `value` attribute should exist. In this toy example, it adds an extra line. Describing that as "clunky" is a matter of opinion; I disagree and think it's fine. But the real difference is when it comes to larger codebases. If classes "A" and "B" are referenced dozens or even hundreds of times, you'd have to add quote marks around every annotation that references one (both?). Manual stringization of large codebases was sufficiently disliked as to have brought about the creation and acceptance of PEP 563. Judicious use of the "forward class" statement should obviate most (all?) the manual stringizing in these codebases. //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TVCB6ITGHO5QI5GYFNMEVEZKE6K24UDB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/23/2022 9:55 AM, Jelle Zijlstra wrote: However, it doesn't solve the problem for base classes. For example, str is conceptually defined as `class str(Sequence["str"]):`. A forward reference can't make `str` defined when the bases are evaluated, because bases are resolved at the `forward class` stage. Larry's second email "Proto-PEP part 2: Alternate implementation proposal for "forward class" using a proxy object" discusses a possibility to move the bases and metaclasses to the "continue class" stage. It also has the advantage of not changing the behavior of __new__, and I think is in general easier to reason about. He and I have discussed this approach, but neither of have looked at in enough detail to know if the implementation is possible. Some of the concerns are noted in that email. Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/MODT7FKWYNDOR4D7LZRW2UOQB5OQSSJ3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/23/2022 3:28 AM, Tobias HT wrote: On the other hand, there's something I've been seeing around. I don't know if it was introduced by Mypy or something, but its the use of declaration files. I think they are saved as pyi files. They just have the declaration of a python object, be it class or variable. What if we just found a way of reusing that instead? As they currently exist, stub files (.pyi files) don't contain enough information. In particular, they don't have the metaclass information. This could be changed, but at that point you basically have the "forward" declaration, but it's hidden away where the interpreter can't see it. Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZNZS7Q2MYGGDW5DJ4LOAQ2FOOULKQV3S/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
This seems like it would solve a huge problem that I've often faced myself, but by introducing something that might later on cause problems I anticipate. Like some people suggested earlier, introducing new keyword in python is not a good solution, I feel it just adds to the already existing bulk and increases complexity. Also Like some else mentioned again, this solution doesn't seem pythonic. I don't know what they meant by that, but in hindsight, it does really not feel pythonic. On the other hand, there's something I've been seeing around. I don't know if it was introduced by Mypy or something, but its the use of declaration files. I think they are saved as pyi files. They just have the declaration of a python object, be it class or variable. What if we just found a way of reusing that instead? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IW223KRYW7TX6UTTDMMGYQQWHEGWV32O/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
I don't think this proposal is a good solution for the problems of static
typing users.
El vie, 22 abr 2022 a las 18:16, Larry Hastings ()
escribió:
> But the explosion of static type analysis in Python, particularly with
> the `typing` module and the `mypy` tool, has made circular definition-time
> dependencies between classes commonplace--and much harder to solve. Here's
> one simple example:
>
> ```Python
> class A:
> value: B
>
> class B:
> value: A
> ```
>
> So to reiterate, your proposal would be to write this as:
forward class B:
pass
class A:
value: B
continue class B:
value: A
While the current workaround is:
class A:
value: "B"
class B:
value: "A"
I don't think I would write the "forward class" version if I had the
choice. It's clunkier and requires more refactoring if I change my mind
about whether the `value` attribute should exist.
I'd prefer if we found some elegant way to get the natural way to write
this code to work, which is the way you wrote it in your post (with no
explicit declarations or stringifications). Carl Meyer's idea at
https://github.com/larryhastings/co_annotations/issues/2#issuecomment-1092432875
is a very promising approach that should allow us to do that.
I also agree with Mehdi2277's concern that this feature would be difficult
for static type checkers to fully implement, because class declaration and
implementation may be widely separated. To be sure, static checkers can put
in additional restrictions, like requiring declaration and implementation
to be in the same module, but it would be unfortunate if a feature designed
to help users of static typing actually makes it harder to adopt it.
-
That said, one nice thing about the proposal is that it can help with
forward references outside of annotations, a problem which neither PEP 563
nor 649 currently solves. Those include type aliases (MyAlias = Sequence["A
| B"]) and TypeVar bounds (Typevar("AT", bound="A | B")). However, it
doesn't solve the problem for base classes. For example, str is
conceptually defined as `class str(Sequence["str"]):`. A forward reference
can't make `str` defined when the bases are evaluated, because bases are
resolved at the `forward class` stage.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/F7L4SU3QIQLPOYHJF5RNJLROF4VMJTMK/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
I should have said "numpy_forward", not "numpy.forward". I changed my mind at the last second as I was writing that email, and momentarily forgot that when you import x.y you implicitly import x. /arry On Sat, Apr 23, 2022, 01:53 Larry Hastings wrote: > > On 4/23/22 01:14, Steven D'Aprano wrote: > > On Sat, Apr 23, 2022 at 12:46:37AM -0700, Larry Hastings wrote: > > > But rather than speculate further, perhaps someone who works on one of > the static type analysis checkers will join the discussion and render an > informed opinion about how easy or hard it would be to support "forward > class" and "continue class". > > > No offense Larry, but since this proto-PEP is designed to help the > typing community (I guess...) shouldn't you have done that before > approaching Python-Dev with the proposal? > > The perfect is the enemy of the good. Like I said, I wanted to get this > out there before the Language Summit, and I just ran out of time. I think > there's also some sort of typing summit next week? I'm not really plugged > in to the static type analysis world--I don't use it in any of my projects. > > > Wouldn't that be a massively breaking change? Anyone who does: > > from numpy import ndarray > > will get the forward-declared class object instead of the fully > initialised class object, leading to all sorts of action-at-a-distance > bugs. > > I wasn't recommending The Famous numpy Project do this exact thing, it was > an abstract example using the name "numpy". I didn't think this was a real > example anyway, as I was assuming that most people who import numpy don't > do so in an "if TYPE_CHECKING:" block. > > Separating the forward class declaration from the continue class > implementation in the actual "numpy" module itself is probably not in the > cards for a while, if ever. But perhaps numpy could do this: > > import numpy.forward > if TYPE_CHECKING: > import numpy > > In this case, the "numpy" module would also internally "import > numpy.forward", and would contain the "continue class" statements for the > forward-declared classes in "numpy.forward". > > There are lots of ways to solve problems with the flexibility afforded by > the proposed "forward class" / "continue class" syntax. Perhaps in the > future you'll suggest some of them! > > > */arry* > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QHHI3D2T7MK2Y4SCRJ33BO433R4MKZYU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/23/22 03:10, Terry Reedy wrote: On 4/22/2022 11:16 PM, Larry Hastings wrote: So I still prefer "forward class". I don't think it's as clear as "forward class" 'forward class' for an incomplete class is not at all clear to me. It is not clear to me which part of speech you intend it to be: noun, verb, adjective, or adverb. You must have some experience with 'forward' in a different context that makes it clearer to you. It's a reference to the term "forward declaration": https://en.wikipedia.org/wiki/Forward_declaration //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VEX3ONWBKIIDF2VR5OPANSKEIDFV3AP3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/22/2022 11:16 PM, Larry Hastings wrote: So I still prefer "forward class". I don't think it's as clear as "forward class" 'forward class' for an incomplete class is not at all clear to me. It is not clear to me which part of speech you intend it to be: noun, verb, adjective, or adverb. You must have some experience with 'forward' in a different context that makes it clearer to you. -- Terry Jan Reedy ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4BB7UPSE26RFYUMX4PZOJHHQYKCYAYAR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/23/22 01:14, Steven D'Aprano wrote: On Sat, Apr 23, 2022 at 12:46:37AM -0700, Larry Hastings wrote: But rather than speculate further, perhaps someone who works on one of the static type analysis checkers will join the discussion and render an informed opinion about how easy or hard it would be to support "forward class" and "continue class". No offense Larry, but since this proto-PEP is designed to help the typing community (I guess...) shouldn't you have done that before approaching Python-Dev with the proposal? The perfect is the enemy of the good. Like I said, I wanted to get this out there before the Language Summit, and I just ran out of time. I think there's also some sort of typing summit next week? I'm not really plugged in to the static type analysis world--I don't use it in any of my projects. Wouldn't that be a massively breaking change? Anyone who does: from numpy import ndarray will get the forward-declared class object instead of the fully initialised class object, leading to all sorts of action-at-a-distance bugs. I wasn't recommending The Famous numpy Project do this exact thing, it was an abstract example using the name "numpy". I didn't think this was a real example anyway, as I was assuming that most people who import numpy don't do so in an "if TYPE_CHECKING:" block. Separating the forward class declaration from the continue class implementation in the actual "numpy" module itself is probably not in the cards for a while, if ever. But perhaps numpy could do this: import numpy.forward if TYPE_CHECKING: import numpy In this case, the "numpy" module would also internally "import numpy.forward", and would contain the "continue class" statements for the forward-declared classes in "numpy.forward". There are lots of ways to solve problems with the flexibility afforded by the proposed "forward class" / "continue class" syntax. Perhaps in the future you'll suggest some of them! //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Y2PSUKYJFCVUUWBBV5Z3HNZBVRVS6SZS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/23/22 00:53, Steven D'Aprano wrote: It's a "forward-declared class object". It's the real class object, but it hasn't been fully initialized yet, and won't be until the "continue class" statement. The only thing that makes it not fully initialised is that it has a bozo bit dunder "__forward__" instructing the interpreter to disallow instantiation. Yes? If I take that class object created by `forward class X`, and delete the dunder, there is no difference between it and other regular classes. Am I correct? No, there are several differences. * It still has the "dict-like object" returned by metaclass.__prepare__, rather than its final dict. * Its class body hasn't been called yet, so it likely doesn't have any of its important dunder methods. * It hasn't had its BaseClass.__init_subclass__ called yet. * It hasn't had its metaclass.__init__ called yet. The "forward-declared class object" is in a not-yet-fully initialized state, and is not ready for use as a class. From my perspective, the "__forward__" attribute is an internal implementation detail, and something that user code should strictly leave alone. But if it's considered too dangerous to expose to users, we could hide it in the class object and not expose it to users. I'm not convinced that's the right call; I think the Consenting Adults rule still applies. Python lets you do crazy things like assigning to __class__, and resurrecting objects from inside their __del__; manually removing __forward__ seems like it falls into the same category. It's not recommended, and we might go so far as to say doing that results in undefined behavior. But Python shouldn't stand in your way if you really think you need to do it for some reason. //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2XDQGNBYQMN6M7Y5JNZA26JX5XS5AYIF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sat, Apr 23, 2022 at 12:46:37AM -0700, Larry Hastings wrote: > But rather than speculate further, perhaps someone who works on one of > the static type analysis checkers will join the discussion and render an > informed opinion about how easy or hard it would be to support "forward > class" and "continue class". No offense Larry, but since this proto-PEP is designed to help the typing community (I guess...) shouldn't you have done that before approaching Python-Dev with the proposal? I might have missed it, but I don't think I've seen this discussed on the typing-sig mailing list. Or have I misunderstood? If typing is not driving this, what is? > >One other edge case here is how would you forward declare an annotation > >for a type that like this, > > > >if TYPE_CHECKING: > > import numpy > > > >def f(x: numpy.ndarray) -> None: > > ... > > > >forward declaring ndarray here would not make numpy.ndarray available. > > In this case, adding forward declarations for classes in "numpy" would > be up to the "numpy" module. One approach might look more like this: > >import numpy # contains forward declarations >if TYPE_CHECKING: > import numpy.impl Wouldn't that be a massively breaking change? Anyone who does: from numpy import ndarray will get the forward-declared class object instead of the fully initialised class object, leading to all sorts of action-at-a-distance bugs. from numpy import ndarry a = ndarray([1, 2]) # Fails, because its a FDCO import some_random_module # which happens to import numpy.impl # which finishes the initialisation of the class object a = ndarray([1, 2]) # and this now magically works! -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5NPG3AG2GYDY7S3RBZD3XTUQDD7WQEQU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Fri, Apr 22, 2022 at 10:09:33PM -0700, Larry Hastings wrote: [Larry] > >>To be clear: `forward class` creates the official, actual class object. > >>Code that wants to take a reference to the class object may take > >>references > >>to the `forward class` declared class, and interact with it as normal. > >>However, a class created by `forward class` can't be *instantiated* > >>until after the matching `continue class` statement finishes. [Steve (me)] > >Since the "forward class" is a real class, > > It's a "forward-declared class object". It's the real class object, but > it hasn't been fully initialized yet, and won't be until the "continue > class" statement. The only thing that makes it not fully initialised is that it has a bozo bit dunder "__forward__" instructing the interpreter to disallow instantiation. Yes? If I take that class object created by `forward class X`, and delete the dunder, there is no difference between it and other regular classes. Am I correct? So your reasoning is circular: you give it a dunder marking it as a "forward-declared class object" to prevent it from being instantiated, but the only reason it can't be instantiated is that it has the dunder. I won't respond to the rest of your post until you have clarified the above, in case I have misunderstood. -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/B7BZDPTWQTZ67PO7AB2BPOBJRDKAFCDZ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/22/22 23:41, Mehdi2277 wrote: My main question for this approach is how would this work with type checkers? It would be new syntax for Python, so type checkers would have to understand it. Is there any restriction that forward class's continuation must appear in same module? No. If it's allowed that a forward class may be continued in a different module I do not see how type checker like mypy/pyright could handle that. Classes are generally viewed as closed and fully defined within type checker. Monkey patching at runtime later is not supported. If it became official Python syntax, I suspect they'd figure out a way to support it. They might require that the expression used in the "continue class" statement map to the original "forward class" declaration, e.g. they might stipulate that they don't support this: forward class X random_name = X continue class random_name: ... But rather than speculate further, perhaps someone who works on one of the static type analysis checkers will join the discussion and render an informed opinion about how easy or hard it would be to support "forward class" and "continue class". One other edge case here is how would you forward declare an annotation for a type that like this, if TYPE_CHECKING: import numpy def f(x: numpy.ndarray) -> None: ... forward declaring ndarray here would not make numpy.ndarray available. In this case, adding forward declarations for classes in "numpy" would be up to the "numpy" module. One approach might look more like this: import numpy # contains forward declarations if TYPE_CHECKING: import numpy.impl Though numpy presumably couldn't do this while they still supported older versions of Python. That's one downside of using new syntax--you can't use it until you stop support for old versions of Python that predate it. Would you forward declare modules? Is that allowed? I haven't proposed any syntax for forward-declaring modules, only classes. I'm confused in general how if TYPE_CHECKING issue is handled by this approach. Usually class being imported in those blocks is defined normally (without continue class) somewhere else. My proposal should mate well with "if TYPE_CHECKING". You would define your forward classes in a module that does get imported, but leave the continue classes in a separate module that is only imported "if TYPE_CHECKING", as per my example with "numpy" above. //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/A7MW3B7JYM2LT4VUCAE7OUHJSYEX76G7/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sat, Apr 23, 2022 at 06:41:23AM -, Mehdi2277 wrote: > My main question for this approach is how would this work with type > checkers? Is there any restriction that forward class's continuation > must appear in same module? If it's allowed that a forward class may > be continued in a different module I do not see how type checker like > mypy/pyright could handle that. Larry said that the name that follows `continue class` is an expression, so that something like this is allowed: import mymodule continue class mymodule.X: def method(self): pass so yes, you can continue classes in other modules. He said that could be used as a very primitive form of separation of interface and implementation, by putting the `forward class` in one module and the `continue` in another. -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/N3YIF2FJARSE73M6VJX6UNU776FS6QC4/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
My main question for this approach is how would this work with type checkers? Is there any restriction that forward class's continuation must appear in same module? If it's allowed that a forward class may be continued in a different module I do not see how type checker like mypy/pyright could handle that. Classes are generally viewed as closed and fully defined within type checker. Monkey patching at runtime later is not supported. Libraries/techniques that rely on extending a class later tend to be sources of type errors/poor type checking coverage. ABC.register for dynamically declaring subclass relationship is similarly not supported by checkers. So without constraint that forward class must be continued within same module I think it's unlikely any type checker would support this technique. As main motivation is related to type annotations I think usage should be supported by type checkers. One other edge case here is how would you forward declare an annotation for a type that like this, if TYPE_CHECKING: import numpy def f(x: numpy.ndarray) -> None: ... forward declaring ndarray here would not make numpy.ndarray available. Would you forward declare modules? Is that allowed? Looking at sourcegraph I do see a some module imports in TYPE_CHECKING blocks for purpose of using them as annotations. I'm confused in general how if TYPE_CHECKING issue is handled by this approach. Usually class being imported in those blocks is defined normally (without continue class) somewhere else. My current leaning lies towards pep 649 + smart eval strategy by get_type_hints that Carl proposed. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/XNYKAEI6TSZU2EKQEM6AXDZOMVD3CPLS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sat, 23 Apr 2022 at 15:32, Larry Hastings wrote: > > > On 4/22/22 22:03, Chris Angelico wrote: > > Anyhow, [a forward-defined class object is] a class, with some special > features (notably that you can't instantiate it). > > Yes. Specifically, here's my intention for "forward-defined class objects": > you can examine some generic dunder values (__name__, __mro__), and you can > take references to it. You can't instantiate it or meaningfully examine its > contents, because it hasn't been fully initialized yet. > > > It seems odd that you define a blessed way of monkeypatching a class, > but then demand that it can only be done once unless you mess with > dunders. Why not just allow multiple continuations? > > I think monkeypatching is bad, and I'm trying to avoid Python condoning it. > > On that note, the intent of my proposal is that "continue class" is not > viewed as "monkeypatching" the class, it's the second step in defining the > class. > > I considered attempting to prevent the user modifying the "forward-declared > class object". But a) that just seemed like an arms race with the user--"oh > yeah? well watch THIS!" and b) I thought the Consenting Adults rule applied. > > Still, it's not the intent of my PEP to condone or facilitate monkeypatching. > I guess perception is everything. This really *is* monkeypatching; you have an object, and that object will be mutated, and those mutations are defined by adding functionality to the class. It's not fundamentally different from X.__dict__.update() with a pile of new methods etc. The only difference is that you call it something different. Is that sufficient justification? I'm unsure. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/DIEGQM5ARFGIYPWG34CQODAENKNBVUET/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/22/22 22:03, Chris Angelico wrote: Anyhow, [a forward-defined class object is] a class, with some special features (notably that you can't instantiate it). Yes. Specifically, here's my intention for "forward-defined class objects": you can examine some generic dunder values (__name__, __mro__), and you can take references to it. You can't instantiate it or meaningfully examine its contents, because it hasn't been fully initialized yet. It seems odd that you define a blessed way of monkeypatching a class, but then demand that it can only be done once unless you mess with dunders. Why not just allow multiple continuations? I think monkeypatching is bad, and I'm trying to avoid Python condoning it. On that note, the intent of my proposal is that "continue class" is not viewed as "monkeypatching" the class, it's the second step in defining the class. I considered attempting to prevent the user modifying the "forward-declared class object". But a) that just seemed like an arms race with the user--"oh yeah? well watch THIS!" and b) I thought the Consenting Adults rule applied. Still, it's not the intent of my PEP to condone or facilitate monkeypatching. //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CXRMKKMKIHGGPAW3F4W4QCNRDIBHFLTB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/22/22 20:58, Steven D'Aprano wrote: On Fri, Apr 22, 2022 at 06:13:57PM -0700, Larry Hastings wrote: This PEP proposes an additional syntax for declaring a class which splits this work across two statements: * The first statement is `forward class`, which declares the class and binds the class object. * The second statement is `continue class`, which defines the contents of the class in the "class body". To be clear: `forward class` creates the official, actual class object. Code that wants to take a reference to the class object may take references to the `forward class` declared class, and interact with it as normal. However, a class created by `forward class` can't be *instantiated* until after the matching `continue class` statement finishes. Since the "forward class" is a real class, It's a "forward-declared class object". It's the real class object, but it hasn't been fully initialized yet, and won't be until the "continue class" statement. it doesn't need any new syntax to create it. Just use plain old regular class syntax. class X(object): """Doc string""" attribute = 42 And now we have our X class, ready to use in annotations. To add to it, or "continue class" in your terms, we can already do this: X.value = "Hello World" But if X has a metaclass that defines __new__ , "value" won't be defined yet, so metaclass.__new__ won't be able to react to and possibly modify it. Similarly for metaclass.__init__ and BaseClass.__init_subclass__. So, while your suggested technique doesn't "break" class creation per se, it prevents the user from benefiting from metaclasses and base classes using these advanced techniques. Counter proposal: `continue class expression:` evaluates expression to an existing class object (or raises an exception) and introduces a block. The block is executed inside that class' namespace, as the `class` keyword does, except the class already exists. If "continue class" is run on an already-created class, this breaks the functionality of __prepare__, which creates the namespace used during class body execution and is thrown away afterwards. The "dict-like object" returned by __prepare__ will have been thrown away by the time "continue class" is executed. Also, again, this means that the contents added to the class in the "continue class" block won't be visible to metaclass.__new__, metaclass.__init__, and BaseClass.__init_subclass__. Also, we would want some way of preventing the user from running "continue class" multiple times on the same class--else we accidentally condone monkey-patching in Python, which we don't want to do. Isn't this case solved by either forward references: class A: value: "B" or by either of PEP 563 or PEP 649? It is, but: a) manual stringizing was rejected by the community as too tiresome and too error-prone (the syntax of the string isn't checked until you run your static type analysis tool). Also, if you need the actual Python value at runtime, you need to eval() it, which causes a lot of headaches. b) PEP 649 doesn't solve this only-slightly-more-advanced case: @dataclass class A: value: B @dataclass class B: value: A as the dataclass decorator examines the contents of the class, including its annotations. c) PEP 563 has the same "what if you need the actual Python value at runtime" problem as manual stringization, which I believe is why the SC has delayed its becoming default behavior. Perhaps my example for b) would be a better example for the PEP. That could become: class A: pass class B: value: A # This is fine, A exists. A.value: B # And here B exists, so this is fine too. No new syntax is needed. This is already legal. It's legal, but it doesn't set the annotation of "value" on A. Perhaps this is just a bug and could be fixed. (TBH I'm not sure what the intended semantics of this statement are, or where that annotation ends up currently. I couldn't find it in A.__annotations__ or the module's __annotations__. Is it just forgotten?) I assert this approach will be undesirable to Python programmers. This makes for two very-different feeling approaches to defining the members of a class. One of the goals of my PEP was to preserve the existing "feel" of Python as much as possible. Also, as previously mentioned, your technique prevents "A.value", and all other attributes and methods set using this technique, from being visible to metaclass.__new__, metaclass.__init__, and BaseClass.__init_subclass__. This proposed `forward class` / `continue class` syntax should permit solving *every* forward-reference and circular-reference problem faced in Python, I think that's overselling the concept. Okay, perhaps I should have said "the forward-reference and circular-reference problems of class definitions" or something like that. I'll adjust the
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sat, 23 Apr 2022 at 12:50, Larry Hastings wrote: > > > On 4/22/22 19:17, Chris Angelico wrote: > > I'm unsure about the forward class. How is it different from subclassing an > ABC? > > They're just different objects. A subclass of an ABC is either itself > another abstract base class, which will never be instantiatable, or a > non-abstract class, which is immediately instantiatable. A forward-declared > class object is not currently instantiatable, and is not fully defined, but > will become fully defined and instantiatable after the matching "continue > class" statement. > Ah okay, I think I had the idea that an ABC could demand certain methods, but I think I'm mismatching semantics when it comes to half-implementing something. Anyhow, it's a class, with some special features (notably that you can't instantiate it). > > What happens if you try to continue a non-forward class? > > From the proto-PEP: > > Executing a `continue class` statement with a class defined by the `class` > statement raises a `ValueError` exception. > > And also: > > It's expected that knowledgeable users will be able to trick Python into > executing `continue class` on the same class multiple times by interfering > with "dunder" attributes. The same tricks may also permit users to trick > Python into executing `continue class` on a class defined by the `class` > statement. This is undefined and unsupported behavior, but Python will not > prevent it. > Huh. I skimmed back and forth looking, and did a search, and didn't find that. My bad. It seems odd that you define a blessed way of monkeypatching a class, but then demand that it can only be done once unless you mess with dunders. Why not just allow multiple continuations? And if multiple continuations are permitted, there's really no difference between a forward class and a regular class. It'd be a "non-instantiable class", which is an aspect which can be removed by reopening the class, but otherwise it's just like any other class. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/AELFL53RTT2GXOZ6NA7BFEJUE2BH67UR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Fri, Apr 22, 2022 at 06:13:57PM -0700, Larry Hastings wrote: > This PEP proposes an additional syntax for declaring a class which splits > this work across two statements: > * The first statement is `forward class`, which declares the class and binds > the class object. > * The second statement is `continue class`, which defines the contents > of the class in the "class body". > > To be clear: `forward class` creates the official, actual class object. > Code that wants to take a reference to the class object may take references > to the `forward class` declared class, and interact with it as normal. > However, a class created by `forward class` can't be *instantiated* > until after the matching `continue class` statement finishes. Since the "forward class" is a real class, it doesn't need any new syntax to create it. Just use plain old regular class syntax. class X(object): """Doc string""" attribute = 42 And now we have our X class, ready to use in annotations. To add to it, or "continue class" in your terms, we can already do this: X.value = "Hello World" Methods are a bit trickier. The status quo pollutes the global namespace, and repeats the method name three times: def method(self): pass X.method = method but we can do better with just a small amount of new syntax: def X.method(self): pass This has been proposed before, and rejected for lack of any real need. But your "forward class/continue class" suggests that the need has now appeared. So where you would write this: forward class Node() continue class Node: """Node in a linked list.""" def __init__(self, value, next:Optional[Node]): self.value = value self.next = next we could instead do this: class Node(): """Node in a linked list.""" def Node.__init__(self, value, next:Optional[Node]): self.value = value self.next = next The only downside -- although perhaps that's an upside -- is that we save an indent level, which may make it marginally harder to recognise which function defs are being added into a class, and which are not. If that really is an issue, then we could keep the "continue class ..." syntax solely to give us that block structure. Counter proposal: `continue class expression:` evaluates expression to an existing class object (or raises an exception) and introduces a block. The block is executed inside that class' namespace, as the `class` keyword does, except the class already exists. But we don't need the "forward class". Just create a class! Any methods, docstrings, attributes etc which aren't part of a forward reference cycle can go in the initial class creation, and only those which are part of a cycle need to be added afterwards. Here is your circular class example: > ```Python > class A: > value: B > > class B: > value: A > ``` Isn't this case solved by either forward references: class A: value: "B" or by either of PEP 563 or PEP 649? I don't think this is a compelling example. But let's continue anyway. That could become: class A: pass class B: value: A # This is fine, A exists. A.value: B # And here B exists, so this is fine too. No new syntax is needed. This is already legal. We only need minimal new syntax to allow methods to be defined outside of their class body: def A.method(self): ... with no need to overload `continue` for a second meaning. Or we could use the `continue A` syntax, but either way, the `forward A` syntax seems to be unneeded. > But nothing so far has been both satisfying and complete; either it > is wordy and clumsy to use (manually stringizing annotations), or it > added restrictions and caused massive code breakage for runtime use of > annotations (PEP 563), or simply didn't solve every problem (PEP 649). But doesn't PEP 649 solve problems which this proposal does not? Delaying the evaluation of annotations is more general than merely the "forward reference" problem. > This proposed `forward class` / `continue class` syntax should permit > solving *every* forward-reference and circular-reference problem faced > in Python, I think that's overselling the concept. How about forward references for function defaults? def func(arg=func(0)): if arg == 0: return "something" ... Obviously there are other ways to solve that, but the point is, your forward/continue class proposal is not one of them! > using an elegant and Pythonic new syntax. That's a matter of opinion. Personally, I think that *declarations* as in `forward MyClass` are not especially Pythonic, although at least your declaration also creates a real class object. But having two ways to create a class (three if we count type()) is not especially Pythonic either. > As a side benefit, `forward class` and `continue class`
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/22/22 19:36, Terry Reedy wrote:
On 4/22/2022 9:13 PM, Larry Hastings wrote:
forward class X()
New keywords are a nuisance. And the proposed implementation seems
too complex.
My proposed implementation seemed necessary to handle the complexity of
the problem. I would welcome a simpler solution that also worked for
all the same use cases.
How about a 'regular' class statement with a special marker of some
sort. Example: 'body=None'.
It's plausible. I take it "body=None" would mean the declaration would
not be permitted to have a colon and a class body. So now we have two
forms of the "class" statement, and which syntax you're using is
controlled by a named parameter argument.
I think this "body=None" argument changing the syntax of the "class"
statement is clumsy. It lacks the visibility and clarity of "forward
class"; the keyword here makes it pretty obvious that this is not a
conventional class declaration. So I still prefer "forward class".
In my PEP I proposed an alternate syntax for "forward class": "def
class", which has the feature that it doesn't require adding a new
keyword. But again, I don't think it's as clear as "forward class", and
I think clarity is vital here.
Either __new__ or __init__ could raise XError("Cannot instantiate
until this is continued.", so no special instantiation code would
needed and X could be a real class, with a special limitation.
Yes, my proposal already suggests that __new__ raise an exception.
That's not the hard part of the problem.
The problem with X being a "real class" is that creating a "real class"
means running all the class creation code, and a lot of the class
creation code is designed with the assumption that the namespace has
already been filled by executing the class body.
For example, Enum in enum.py relies on EnumMeta, which defines __new__,
which examines the already-initialized namespace of the class you want
to create. If you propose a "body=None" class be a "real class object",
then how do you declare a class that inherits from Enum using
"body=None"? Creating the class object will call EnumMeta.__new__,
which needs to examine the namespace, which hasn't been initialized yet.
Changing the class object creation code so we can construct a class
object in two steps, with the execution of the class body being part of
the second step, was the major sticking point--and the source of most of
the complexity of my proposal.
continue class X:
# class body goes here
def __init__(self, key):
self.key = key
'continue' is already a keyword.
I'm aware. I'm not sure why you mentioned it.
Given that X is a real class, could implementation be
X.__dict__.update(new-body-dict)
That's what the proof-of-concept does. But the proof-of-concept fails
with a lot of common use cases:
* metaclasses that override __new__ or __init__
* base classes that implement __init_subclass__
because these methods assume the namespace of the class has already been
filled in, but it doesn't get filled in until the @continue_() class
decorator. Handling these cases is why my proposal is sadly as complex
as it is, and why in practice the proof-of-concept doesn't work a lot of
the time.
//arry/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/KVDT632CHPORAA6KWJG6ZZVY4FFSZI27/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/22/22 19:17, Chris Angelico wrote: I'm unsure about the forward class. How is it different from subclassing an ABC? They're just different objects. A subclass of an ABC is either itself another abstract base class, which will never be instantiatable, or a non-abstract class, which is immediately instantiatable. A forward-declared class object is not currently instantiatable, and is not fully defined, but will become fully defined and instantiatable after the matching "continue class" statement. What happens if you try to continue a non-forward class? From the proto-PEP: Executing a `continue class` statement with a class defined by the `class` statement raises a `ValueError` exception. And also: It's expected that knowledgeable users will be able to trick Python into executing `continue class` on the same class multiple times by interfering with "dunder" attributes. The same tricks may also permit users to trick Python into executing `continue class` on a class defined by the `class` statement. This is undefined and unsupported behavior, but Python will not prevent it. //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QU2HE5AAMLHU33Z5Y4EPDF46CXO7F3NF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 4/22/2022 9:13 PM, Larry Hastings wrote:
forward class X()
New keywords are a nuisance. And the proposed implementation seems too
complex. How about a 'regular' class statement with a special marker of
some sort.
Example: 'body=None'.
Either __new__ or __init__ could raise XError("Cannot instantiate until
this is continued.", so no special instantiation code would needed and X
could be a real class, with a special limitation.
continue class X:
# class body goes here
def __init__(self, key):
self.key = key
'continue' is already a keyword. Given that X is a real class, could
implementation be
X.__dict__.update(new-body-dict)
--
Terry Jan Reedy
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/UAZ5P7N5S4OWFSVZSKK6DUEPZTQPV3GP/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Sat, 23 Apr 2022 at 11:16, Larry Hastings wrote: > This PEP proposes an additional syntax for declaring a class which splits > this work across two statements: > * The first statement is `forward class`, which declares the class and binds >the class object. > * The second statement is `continue class`, which defines the contents >of the class in the "class body". > > [chomp all the details] Hmm. The "continue class" part looks great IMO (and I have in fact implemented the decorator version, which is clunky - your version, where "continue class EXPR:" will reference any class - is way more flexible than anything a decorator can do easily), but I'm unsure about the forward class. How is it different from subclassing an ABC? You're claiming a keyword (and one which is definitely going to conflict - if this goes forward in this form, I would certainly be bikeshedding the exact choice of keyword) for something that's extremely similar to simply defining a class, and then marking it as abstract. What happens if you try to continue a non-forward class? For example: class Demo: pass continue class Demo: pass If that works fine, then I'd be inclined to downgrade "forward class" to a keyword argument (like metaclass is), eg "class Demo(abstract=True)" or "class Demo(forward=True)". But, I definitely like the "continue class" part of the idea, and would love to see that happen. Of course, it's going to be horribly abused for monkeypatching, but we could do that already anyway :) ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LDYKP7NSVUECEZPW7SP5FMZLASMGJ6TT/ Code of Conduct: http://python.org/psf/codeofconduct/
