[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 27/04/22 11:07 am, Chris Angelico wrote: You're saying that it's somehow different when the original dev intends for it, and that that makes it "not monkeypatching". I dispute that, and I consider that the feature would be helpful whether the original dev meant for it or not. The forward declaration and continuation both have to be specially marked, so you *can't* use it for monkeypatching. Would subclassing therefore become legitimate if the original author intended for it, and be called "method stealing" if it were not? You seem to think I have something against monkeypatching. Personally I don't, but someone else seems to as they objected that they don't want monkeypatching to be encouraged. I was just pointing out that (1) I don't think it's the same thing as monkeypatching and (2) you can't use it to modify an existing class in an ad-hoc manner anyway. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6HOFAR3TKR62YFPUZ4OIBY5DJBNV374V/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On 26/04/2022 20:48, Carl Meyer via Python-Dev wrote:
On Tue, Apr 26, 2022 at 1:25 PM Guido van Rossum wrote:
I also would like to hear more about the problem this is trying to solve, when
th real-world examples. (E.g. from pydantic?)
Yes please. I think these threads have jumped far too quickly into
esoteric details of implementation and syntax, without critical
analysis of whether the semantics of the proposal are in fact a good
solution to a real-world problem that someone has.
I've already outlined in a more detailed reply on the first thread why
I don't think forward declarations provide a practically useful
solution to forward reference problems for users of static typing
(every module that imports something that might be a forward reference
would have to import its implementation also, turning every one-line
import of that class into two or three lines) and causes new problems
for every user of Python due to its reliance on import side effects
causing global changes at a distance. See
https://mail.python.org/archives/list/python-dev@python.org/message/NMCS77YFM2V54PUB66AXEFTE4NXFHWPI/
for details.
Under PEP 649, forward references are a small problem confined to the
edge case of early resolution of type annotations. There are simple
and practical appropriately-scoped solutions easily available for that
small problem: providing a way to resolve type annotations at runtime
without raising NameError on not-yet-defined names. Such a facility
(whether default or opt-in) is practically useful for many users of
annotations (including dataclasses and documentation tools), which
have a need to introspect some aspects of annotations without
necessarily needing every part of the annotation to resolve. The
existence of such a facility is a reasonable special case for
annotations specifically, because annotations are fundamentally
special: they provide a description of code, rather than being only a
part of the code. (This special-ness is precisely also why they cause
more forward references in the first place.)
IMO, this forward declaration proposal takes a small problem in a
small corner of the language and turns it into a big problem for the
whole language, without even providing as nice and usable an option
for common use cases as "PEP 649 with option for lax resolution" does.
This seems like a case study in theoretical purity ("resolution of
names in annotations must not be special") running roughshod over
practicality.
Carl
Insofar as I understand the above (knowing almost nothing about typing), +1.
Best wishes
Rob Cliffe
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/DWA4AGXTBI5SU2R2T5O7KTQJ4F6DU27S/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Tue, Apr 26, 2022 at 7:24 PM Greg Ewing wrote: > On 27/04/22 2:01 am, Chris Angelico wrote: > > That would be the case if monkeypatching were illegal. Since it's not, > > wherein lies the difference? > > The proposed feature is analogous to forward declaring a > struct in C. Would you call what C does monkeypatching? > It is not analogous; it is a false analogy that obscures the issues with this proposal in Python. A C forward declaration (not to mention the full struct declaration also!) is purely for the compiler; at runtime one can have a pointer to some memory that the compiler expects to be shaped like that struct, but one can never get hold of any runtime value that is “the struct definition itself,” let alone a runtime value that is the nascent forward-declared yet-to-be-completed struct. So clearly there can be no patching of something that never exists at runtime at all. Python is quite different from C in this respect. Classes are runtime objects, and so is the “forward declared class” object. The proposal is for a class object to initially at runtime be the latter, and then later (at some point that is not well defined if the implementation is in a separate module, because global module import ordering is an unstable emergent property of all the imports in the entire codebase) may suddenly, everywhere and all at once, turn into the former. Any given module that imports the forward declared name can have no guarantee when (if ever) that object will magically transform into something that is safe to use. Whether we call it monkeypatching or not is irrelevant. Having global singleton objects change from one thing to a very different thing, at an unpredictable point in time, as a side effect of someone somewhere importing some other module, causes very specific problems in being able to locally reason about code. I think it is more useful to discuss the specific behavior and its consequences than what it is called. Carl ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CQ7TAV6TWGEG2HLVY7T46U6JCPESRACR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Reminder: PEP 479's __future__ about to become the default behavior
___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4V6I2SEDNIEDF23ZZAMCIBIQZWSLN7OS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On Mon, Apr 25, 2022 at 10:32:15PM -0700, Larry Hastings wrote: [...] > Whichever spelling we use here, the key idea is that C is bound to a > "ForwardClass" object. A "ForwardClass" object is /not/ a class, it's a > forward declaration of a class. (I suspect ForwardClass is similar to a > typing.ForwardRef, though I've never worked with those so I couldn't say > for sure.) I know that the three devs working on this plan (Larry, Barry, Mark) are extremely competent, well-respected people, but this proto-PEP really feels to me like you aren't heavy users of typing and are proposing an extremely complex, complicated, problematic "solution" which even the typing people don't want. Carl Meyer has repeatedly said that this proposal won't solve the issues that you have designed it to solve. I don't think you have responded to him either time, which is worrying. https://mail.python.org/archives/list/python-dev@python.org/message/NMCS77YFM2V54PUB66AXEFTE4NXFHWPI/ https://mail.python.org/archives/list/python-dev@python.org/message/RVQSLD435BFKEVIMY2AIA5MCJB37BPHK/ Now Carl is only one person, and not necessarily representative, but Jelle Zijlstra is also heavily involved with mypy and typeshed and has expressed serious doubts about this proposal as well. So unless somebody with solid experience in typing contradict Carl and Jelle, we ought to take their comments as definitive and trust them that these "ForwardClass" objects won't solve the problem of forward declarations (or indeed that the problem doesn't really need solving beyond the two PEPs already on the table, plus stringified types). I realise that you have invested a lot of time and energy into this proto-PEP, but if it fails to solve the forward declaration problem, and introduces problems of its own, then maybe we should not fall into the Sunk Costs fallacy by continuing to debate the best syntax and implementation for something that we don't need. -- Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7JIR2D3WUCHHXVTQUQV57XY5VXSBWAVN/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Wed, 27 Apr 2022 at 11:18, Greg Ewing wrote: > > On 27/04/22 2:01 am, Chris Angelico wrote: > > That would be the case if monkeypatching were illegal. Since it's not, > > wherein lies the difference? > > The proposed feature is analogous to forward declaring a > struct in C. Would you call what C does monkeypatching? > No, because C doesn't have first-class types, much less mutable ones. It doesn't make sense to compare them. The proposed feature is an alternative way to add attributes to an existing class object, which we normally call monkeypatching, but you're saying that sometimes it isn't. I'm still confused as to why you consider there to be a difference. According to this proposal, it is entirely possible to continue a class more than once, with just some fiddling with dunders. But what you're saying is that sometimes that's monkeypatching and a bad thing, and other times it's a good thing and not monkeypatching. I say that it's always monkeypatching, and always a good thing. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5UYWIKP5PUHDRAE6APVHMBWIAVQYZTH3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 27/04/22 2:01 am, Chris Angelico wrote: That would be the case if monkeypatching were illegal. Since it's not, wherein lies the difference? The proposed feature is analogous to forward declaring a struct in C. Would you call what C does monkeypatching? -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/5DRJG3CGCLHJSXBD6ILPHROX6LBIDCHK/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Wed, 27 Apr 2022 at 08:06, Greg Ewing wrote: > > On 27/04/22 1:04 am, Joao S. O. Bueno wrote: > > MonkeyPatching in Python is not illegal in this sense. > > I'm not suggesting it is. You're seizing on the wrong part > of the analogy. The point is that what you call something > doesn't change its nature. > I agree. So why is it monkeypatching sometimes, and not other times? Why are you distinguishing? To be clear, this is what I'm responding to: On Tue, 26 Apr 2022 at 10:05, Greg Ewing wrote: > To me, monkeypatching means modifying the definition of something > after the fact in a way that it wasn't designed for. > > Here, the modification is fully intended, so it's not monkeypatching. You're saying that it's somehow different when the original dev intends for it, and that that makes it "not monkeypatching". I dispute that, and I consider that the feature would be helpful whether the original dev meant for it or not. Consider what Python would be like if every class had to declare whether it was allowed to be subclassed or not. Would subclassing therefore become legitimate if the original author intended for it, and be called "method stealing" if it were not? Would that materially improve the language? I have frequently subclassed something without the "consent" of the original class's author, because it is a useful feature. Python is a "consenting adults" language. If you do something the original author didn't intend, you might find yourself running into backward compatibility issues, but you shouldn't have the language block you unnecessarily. ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2BGUHWOUNJTVETANBVJPCVPQZ43IVA57/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 27/04/22 1:04 am, Joao S. O. Bueno wrote: MonkeyPatching in Python is not illegal in this sense. I'm not suggesting it is. You're seizing on the wrong part of the analogy. The point is that what you call something doesn't change its nature. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/WFVAWIKBQF72KW2IHOERP2DIKD5ECO7N/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Add -P command line option to not add sys.path[0]
The only purpose of proposed -P option is to "not add sys.path[0]". There are use cases which only need that. Victor On Tue, Apr 26, 2022 at 8:37 PM Steve Dower wrote: > > On 4/26/2022 10:46 AM, Victor Stinner wrote: > > I propose adding a -P option to Python command line interface to "not > > add sys.path[0]": > > https://github.com/python/cpython/pull/31542 > > > > See the documentation in the PR for the exact behavior of this option. > > I prefer to add an environment variable, only pass the option > > explicitly on the command line. > > Another viable option might be to add an option to imply "import site", > which would work together with -I to: > * ignore environment variables (-E/-I) > * omit implicit CWD imports (-I) > * still process .pth files (-?) > * still include site-packages and user site-packages in sys.path (-?) > > It seems likely that the proposed -P would almost always be used with > -E, since if you can't control CWD then you presumably can't control > environment variables either. > > The existing ._pth functionality starts by implying -I, and allows > "import site" in the file to explicitly include site. A command-line > option matching this behaviour would be consistent. There's also already > configuration in our structures for import site, so there'd be no need > to add new fields to public APIs for the option. > > The biggest issue I see is that the obvious command line options for > "import site" are already used to imply "do not import site". But then, > -P isn't obvious either. Maybe an -X option would suffice? > > Cheers, > Steve > -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/CU75ZRF2UTWGU6OHKHQTWSXRZH52WVOF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On Tue, Apr 26, 2022 at 1:25 PM Guido van Rossum wrote:
> I also would like to hear more about the problem this is trying to solve,
> when th real-world examples. (E.g. from pydantic?)
Yes please. I think these threads have jumped far too quickly into
esoteric details of implementation and syntax, without critical
analysis of whether the semantics of the proposal are in fact a good
solution to a real-world problem that someone has.
I've already outlined in a more detailed reply on the first thread why
I don't think forward declarations provide a practically useful
solution to forward reference problems for users of static typing
(every module that imports something that might be a forward reference
would have to import its implementation also, turning every one-line
import of that class into two or three lines) and causes new problems
for every user of Python due to its reliance on import side effects
causing global changes at a distance. See
https://mail.python.org/archives/list/python-dev@python.org/message/NMCS77YFM2V54PUB66AXEFTE4NXFHWPI/
for details.
Under PEP 649, forward references are a small problem confined to the
edge case of early resolution of type annotations. There are simple
and practical appropriately-scoped solutions easily available for that
small problem: providing a way to resolve type annotations at runtime
without raising NameError on not-yet-defined names. Such a facility
(whether default or opt-in) is practically useful for many users of
annotations (including dataclasses and documentation tools), which
have a need to introspect some aspects of annotations without
necessarily needing every part of the annotation to resolve. The
existence of such a facility is a reasonable special case for
annotations specifically, because annotations are fundamentally
special: they provide a description of code, rather than being only a
part of the code. (This special-ness is precisely also why they cause
more forward references in the first place.)
IMO, this forward declaration proposal takes a small problem in a
small corner of the language and turns it into a big problem for the
whole language, without even providing as nice and usable an option
for common use cases as "PEP 649 with option for lax resolution" does.
This seems like a case study in theoretical purity ("resolution of
names in annotations must not be special") running roughshod over
practicality.
Carl
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/RVQSLD435BFKEVIMY2AIA5MCJB37BPHK/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
Apr 26, 2022 20:32:55 Eric V. Smith : How would runtime consumers of annotations use this? -- Eric On Apr 26, 2022, at 12:05 PM, Thomas Kehrenberg wrote: If the problem is mostly type annotations, then another potential solution would be to make use of .pyi files, which are not hamstrung by circular definitions. The idea would be that type checkers would merge the annotations from .pyi files into the annotations in the corresponding .py file. So: a.py: from b import B class A: value: B b.py: class B: value = None b.pyi: from typing import Optional from a import A class B: value: Optional[A] = ... The pyi files would kind of act like header files that are used in other languages. It would mean that type checkers need to check the .pyi files against the code in the .py files to verify that they're consistent with one another. -thomas ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VWPWN5KWTRPP6VS4PEHJA4SRVMUDU5WR/ Code of Conduct: http://python.org/psf/codeofconduct/ They wouldn't. But I thought PEP 649 solves the runtime problems, and that the remaining problems are with static typing of circular definitions. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/LHHE7HPSKNT6UAQGSZPCMU5XJATAARRB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
> On Apr 26, 2022, at 3:05 PM, Thomas Kehrenberg wrote: > > > Apr 26, 2022 20:32:55 Eric V. Smith : > >> How would runtime consumers of annotations use this? >> >> -- >> Eric >> On Apr 26, 2022, at 12:05 PM, Thomas Kehrenberg wrote: >>> >>> If the problem is mostly type annotations, then another potential >>> solution would be to make use of .pyi files, which are not hamstrung by >>> circular definitions. The idea would be that type checkers would merge >>> the annotations from .pyi files into the annotations in the >>> corresponding .py file. >>> >>> So: >>> >>> a.py: >>> >>>from b import B >>> >>>class A: >>>value: B >>> >>> b.py: >>> >>>class B: >>>value = None >>> >>> b.pyi: >>> >>>from typing import Optional >>>from a import A >>> >>>class B: >>>value: Optional[A] = ... >>> >>> The pyi files would kind of act like header files that are used in other >>> languages. It would mean that type checkers need to check the .pyi >>> files against the code in the .py files to verify that they're >>> consistent with one another. >>> >>> -thomas >>> ___ >>> Python-Dev mailing list -- python-dev@python.org >>> To unsubscribe send an email to python-dev-le...@python.org >>> https://mail.python.org/mailman3/lists/python-dev.python.org/ >>> Message archived at >>> https://mail.python.org/archives/list/python-dev@python.org/message/VWPWN5KWTRPP6VS4PEHJA4SRVMUDU5WR/ >>> Code of Conduct: http://python.org/psf/codeofconduct/ > They wouldn't. But I thought PEP 649 solves the runtime problems, > and that the remaining problems are with static typing > of circular definitions. If the class needs access to its own type annotations at runtime (for example, if it’s a dataclass), then the circular reference problem still exists with PEP 649. That’s among the cases we’re trying to resolve. Eric ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SV37AN7HYQNAVIE2BMJ5C5AC77FENTCS/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On Wed, 27 Apr 2022 at 05:05, Larry Hastings wrote:
>
>
> On 4/26/22 09:31, MRAB wrote:
>> Perhaps:
>>
>>class C: ...
>
> Also, your suggestion is already legal Python syntax; it creates a class with
> no attributes. So changing this existing statement to mean something else
> would potentially (and I think likely) break existing code.
>
Not sure if it quite counts as "existing code", but I do often use
this notation during development to indicate that this class will
exist, but I haven't coded it yet. (In contrast, "class C: pass"
indicates that an empty body is sufficient for this class, eg "class
SpamException(Exception): pass" which needs no further work.)
If a less subtle distinction is needed, what about "class C = None"?
That removes the expectation of a colon and body. Personally, I'm
still inclined towards the kwarg method, though ("class
C(forward=True): pass"), since it's legal syntax.
ChrisA
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/RPUZFPRESB6JIMMIPM7WLUKA6RVBAQRD/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
I am traveling and have no keyboard right now, but it looks like this thread is confusing the slots that a type gives to its *instances* and extra slots in the type object itself. Only the latter are a problem. I also would like to hear more about the problem this is trying to solve, when th real-world examples. (E.g. from pydantic?) On Tue, Apr 26, 2022 at 11:57 Larry Hastings wrote: > > On 4/25/22 23:56, Ronald Oussoren wrote: > > A problem with this trick is that you don’t know how large a class object > can get because a subclass of type might add new slots. This is currently > not possible to do in Python code (non-empty ``__slots__`` in a type > subclass is rejected at runtime), but you can do this in C code. > > Dang it! __slots__! Always there to ruin your best-laid plans. *shakes > fist at heavens* > > I admit I don't know how __slots__ is currently implemented, so I wasn't > aware of this. However! The first part of my proto-PEP already proposes > changing the implementation of __slots__, to allow adding __slots__ after > the class is created but before it's instantiated. Since this is so > late-binding, it means the slots wouldn't be allocated at the same time as > the type, so happily we'd sidestep this problem. On the other hand, this > raises the concern that we may need to change the C interface for creating > __slots__, which might break C extensions that use it. (Maybe we can find > a way to support the old API while permitting the new late-binding > behavior, though from your description of the problem I'm kind of doubtful.) > > > Cheers, > > > */arry* > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/YU3PJKPMJZNWKLZUG3JCJFGFOKGMV2GI/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido (mobile) ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/ZQ7T3IZIAUGEUJUUR7DDM72MLKWLKEAU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
> On 26 Apr 2022, at 20:52, Larry Hastings wrote:
>
>
>
> On 4/25/22 23:56, Ronald Oussoren wrote:
>> A problem with this trick is that you don’t know how large a class object
>> can get because a subclass of type might add new slots. This is currently
>> not possible to do in Python code (non-empty ``__slots__`` in a type
>> subclass is rejected at runtime), but you can do this in C code.
> Dang it! __slots__! Always there to ruin your best-laid plans. *shakes
> fist at heavens*
>
> I admit I don't know how __slots__ is currently implemented, so I wasn't
> aware of this. However! The first part of my proto-PEP already proposes
> changing the implementation of __slots__, to allow adding __slots__ after the
> class is created but before it's instantiated. Since this is so
> late-binding, it means the slots wouldn't be allocated at the same time as
> the type, so happily we'd sidestep this problem. On the other hand, this
> raises the concern that we may need to change the C interface for creating
> __slots__, which might break C extensions that use it. (Maybe we can find a
> way to support the old API while permitting the new late-binding behavior,
> though from your description of the problem I'm kind of doubtful.)
>
I used the term slots in a very loose way. In PyObjC I’m basically doing:
typedef struct {
PyHeapTypeObject base;
/* Extra C fields go here */
} PyObjCClassObject;
Those extra C fields don’t get exposed to Python, but could well be by using
getset definitions. This has worked without problems since early in the 2.x
release cycle (at least, that’s when I started doing this in PyObjC), and is
how one subclasses other types as well.
“Real” __slots__ don’t work when subclassing type() because type is a var
object. That’s “just” an implementation limitation, it should be possible to
add slots after the variable length bit (he says while wildly waving his
hands).
Ronald
—
Twitter / micro.blog: @ronaldoussoren
Blog: https://blog.ronaldoussoren.net/
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/XYK7NO4CJGP5N6CLDOI7IONOBJAZYNHK/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On 4/26/22 09:31, MRAB wrote: On 2022-04-26 06:32, Larry Hastings wrote: Note that this spelling is also viable: class C I don't like that because it looks like you've just forgotten the colon. Perhaps: class C: ... That's not a good idea. Every other place in Python where there's a statement that ends in a colon, it's followed by a nested block of code. But the point of this statement is to forward-declare C, and this statement /does not have/ a class body. Putting a colon there is misleading. Also, your suggestion is already legal Python syntax; it creates a class with no attributes. So changing this existing statement to mean something else would potentially (and I think likely) break existing code. Consider C++'s forward-declared class statement: class C; You could say about that, "I don't like that because it looks like you've just forgotten the curly braces." But we didn't forget anything, it's just new syntax for a different statement. //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/S54FAGLEKF22T3WTLTIZ37FW3BVMJQ3V/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On 4/25/22 23:56, Ronald Oussoren wrote: A problem with this trick is that you don’t know how large a class object can get because a subclass of type might add new slots. This is currently not possible to do in Python code (non-empty ``__slots__`` in a type subclass is rejected at runtime), but you can do this in C code. Dang it! __slots__! Always there to ruin your best-laid plans. *shakes fist at heavens* I admit I don't know how __slots__ is currently implemented, so I wasn't aware of this. However! The first part of my proto-PEP already proposes changing the implementation of __slots__, to allow adding __slots__ after the class is created but before it's instantiated. Since this is so late-binding, it means the slots wouldn't be allocated at the same time as the type, so happily we'd sidestep this problem. On the other hand, this raises the concern that we may need to change the C interface for creating __slots__, which might break C extensions that use it. (Maybe we can find a way to support the old API while permitting the new late-binding behavior, though from your description of the problem I'm kind of doubtful.) Cheers, //arry/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/YU3PJKPMJZNWKLZUG3JCJFGFOKGMV2GI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Add -P command line option to not add sys.path[0]
On 4/26/2022 10:46 AM, Victor Stinner wrote: I propose adding a -P option to Python command line interface to "not add sys.path[0]": https://github.com/python/cpython/pull/31542 See the documentation in the PR for the exact behavior of this option. I prefer to add an environment variable, only pass the option explicitly on the command line. Another viable option might be to add an option to imply "import site", which would work together with -I to: * ignore environment variables (-E/-I) * omit implicit CWD imports (-I) * still process .pth files (-?) * still include site-packages and user site-packages in sys.path (-?) It seems likely that the proposed -P would almost always be used with -E, since if you can't control CWD then you presumably can't control environment variables either. The existing ._pth functionality starts by implying -I, and allows "import site" in the file to explicitly include site. A command-line option matching this behaviour would be consistent. There's also already configuration in our structures for import site, so there'd be no need to add new fields to public APIs for the option. The biggest issue I see is that the obvious command line options for "import site" are already used to imply "do not import site". But then, -P isn't obvious either. Maybe an -X option would suffice? Cheers, Steve ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7XKFGCQJCWXU37SFZQA6RVAPOGOQLWDG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
How would runtime consumers of annotations use this? -- Eric > On Apr 26, 2022, at 12:05 PM, Thomas Kehrenberg wrote: > > If the problem is mostly type annotations, then another potential > solution would be to make use of .pyi files, which are not hamstrung by > circular definitions. The idea would be that type checkers would merge > the annotations from .pyi files into the annotations in the > corresponding .py file. > > So: > > a.py: > >from b import B > >class A: >value: B > > b.py: > >class B: >value = None > > b.pyi: > >from typing import Optional >from a import A > >class B: >value: Optional[A] = ... > > The pyi files would kind of act like header files that are used in other > languages. It would mean that type checkers need to check the .pyi > files against the code in the .py files to verify that they're > consistent with one another. > > -thomas > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/VWPWN5KWTRPP6VS4PEHJA4SRVMUDU5WR/ > Code of Conduct: http://python.org/psf/codeofconduct/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2VNYLOHCXWBUYTWRGRJC6C4LXQVF45MD/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On Apr 25, 2022, at 22:32, Larry Hastings wrote:
> The general shape of it is the same. First, we have some sort of forward
> declaration of the class. I'm going to spell it like this:
>
> forward class C
>
> just for clarity in the discussion. Note that this spelling is also viable:
>
> class C
>
> That is, a "class" statement without parentheses or a colon. (This is
> analogous to how C++ does forward declarations of classes, and it was
> survivable for them.) Another viable spelling:
>
> C = ForwardClass()
I like this latter one exactly because as you say it doesn’t require any syntax
changes. In the fine tradition of Python, we could certainly add syntactic
sugar later, but I like that this can be implemented without it and we can see
how the idea plays out in practice before committing to new syntax.
> This spelling is nice because it doesn't add new syntax. But maybe it's less
> obvious what is going on from a user's perspective.
>
> Whichever spelling we use here, the key idea is that C is bound to a
> "ForwardClass" object. A "ForwardClass" object is not a class, it's a
> forward declaration of a class. (I suspect ForwardClass is similar to a
> typing.ForwardRef, though I've never worked with those so I couldn't say for
> sure.)
I haven’t looked deeply at the code for ForwardRef, but I just cracked open
typing.py and noticed this:
class ForwardRef(_Final, _root=True):
"""Internal wrapper to hold a forward reference."""
__slots__ = ('__forward_arg__', '__forward_code__',
'__forward_evaluated__', '__forward_value__',
'__forward_is_argument__', '__forward_is_class__',
'__forward_module__’)
So it seems that you’re almost there already!
> So, technically, this means we could spell the "continue class" step like so:
>
> class C(BaseClass, ..., metaclass=MyMetaClass, __forward__=C):
> ...
Again, nice! Look Ma, no syntax changes.
I don’t know whether the __slots__ issue will break this idea but it is really
… wonderful!
-Barry
signature.asc
Description: Message signed with OpenPGP
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/KEZ3752H6DEZ25HJGMG3QZJYRCVJ5LCR/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Add -P command line option to not add sys.path[0]
On Tue, Apr 26, 2022 at 2:50 AM Victor Stinner wrote:
> Hi,
>
> There are 4 main ways to run Python:
>
> (1) python -m module [...]
> (2) python script.py [...]
> (3) python -c code [...]
> (4) python [...]
>
> (1) and (2) insert the directory of the module/script at sys.path[0].
> (3) and (4) insert an empty string at sys.path[0].
>
> This behavior is convenient and is maybe part of Python usability
> success: importing a module in the current directory is as easy as
> "import other_module" (load other_module.py). But it's also a threat
> to security: an attacker can override a stdlib module by creating a
> Python script with the same name than a stdlib module, like os.py or
> shutil.py.
>
> People learning Python commonly create a file with the same name than
> a stdlib module (ex: random.py) and then are clueless in face of an
> ImportError exception.
>
> Changing the default behavior was discussed multiple times. No
> consensus was reached, maybe because most users like the current
> default behavior and are not affected by corner cases (see below).
>
> I propose adding a -P option to Python command line interface to "not
> add sys.path[0]":
> https://github.com/python/cpython/pull/31542
We would use this in the Python extension for VS Code for case (1) as we
have had issues with (1) when running tools on people's behalf. People will
accidentally shadow the stdlib and then do something unexpected as an
import side-effect in their shadowing module like delete files. Not
something you want happening when you're just wanting to run Pylint.
-Brett
>
>
> See the documentation in the PR for the exact behavior of this option.
> I prefer to add an environment variable, only pass the option
> explicitly on the command line.
>
> Since Python 3.4, there is already the -I ("isolated mode") option:
> https://docs.python.org/dev/using/cmdline.html#cmdoption-I
>
> The -I option has other effects like disabling user site directories,
> it option doesn't fit use cases of the -P option.
>
> One annoying issue of the Python default behavior is that running a
> script in /usr/bin/ as root can create or override .pyc files in the
> /usr directory, even in the /usr/bin/ directory. Example of this
> surprising and annoying issue:
> https://github.com/benjaminp/six/issues/359#issuecomment-996159668
>
> The -P option can be used in #!/usr/bin/python shebang to avoid this issue.
>
> --
>
> An alternative would be to change the default behavior to not add
> sys.path[0], and add an option to opt-in for Python 3.10 behavior.
> Here are my notes about it:
> https://github.com/vstinner/misc/blob/main/cpython/pep_path0.rst
>
> What do you think?
>
> Victor
> --
> Night gathers, and now my watch begins. It shall not end until my death.
> ___
> Python-Dev mailing list -- python-dev@python.org
> To unsubscribe send an email to python-dev-le...@python.org
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-dev@python.org/message/IU5Q2AXAURFVDPRWNU3BDFVKV2QX5NOR/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/2DCKKOJNB4T3ZKXQAEFTC6ZCMXPONFPG/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] New functionality for unittest.mock side_effect
I often want a side_effect of "if called with foo, return bar" functionality.
This is really useful when the order of calls to your mock is indeterminate, so
you can't just use an iterable. What I end up doing is writing a little
function:
def f(x):
data = {
'foo': 'bar',
'baz': 'blah',
}
return data[x]
my_mock.side_effect = f
sometimes I wrap that up in a lambda, but either way it's fidgety boilerplate
which obfuscates the intent. It would be nice if side_effect could accept a
dict and provide the above functionality for free:
my_mock.side_effect = {
'foo': 'bar',
'baz': 'blah',
}
If could handle DEFAULT just like side_effect does now. And it could throw
something more useful than KeyError if the key isn't found.
I could see the argument that "but dicts are themselves iterable, so this could
break some existing mocks which depend on side_effect iterating over the dicts
keys. That would be bizarre, but possible. The fix for that would be invent a
new attribute, say, return_value_map, which does this, and say that it's an
error to set more than one of {return_value, return_value_map, side_effect}.
Anyway, if I were to take a whack at coding this up, is it something that would
be considered for a future release?
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/4A24FFKMN2PEB4GRERXUX2BUNVMZR2MA/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On 2022-04-26 06:32, Larry Hastings wrote: Sorry, folks, but I've been busy the last few days--the Language Summit is Wednesday, and I had to pack and get myself to SLC for PyCon, I'll circle back and read the messages on the existing threads tomorrow. But for now I wanted to post "the wonderful third option" for forward class definitions we've been batting around for a couple of days. The fundamental tension in the proposal: we want to /allocate/ the object at "forward class" time so that everyone can take a reference to it, but we don't want to /initialize/ the class (e.g. run the class body) until "continue class" time. However, the class might have a metaclass with a custom __new__, which would be responsible for allocating the object, and that isn't run until after the "class body". How do we allocate the class object early while still supporting custom metaclass.__new__ calls? So here's the wonderful third idea. I'm going to change the syntax and semantics a little, again because we were batting them around quite a bit, so I'm going to just show you our current thinking. The general shape of it is the same. First, we have some sort of forward declaration of the class. I'm going to spell it like this: forward class C just for clarity in the discussion. Note that this spelling is also viable: class C I don't like that because it looks like you've just forgotten the colon. Perhaps: class C: ... [snip] ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/IZAWCYPHGD2ZVB2U6TVIBM5XKGY5KQS6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Add -P command line option to not add sys.path[0]
On 4/26/22, Victor Stinner wrote: > > There are 4 main ways to run Python: > > (1) python -m module [...] > (2) python script.py [...] > (3) python -c code [...] > (4) python [...] > > (1) and (2) insert the directory of the module/script at sys.path[0]. Running a module with -m inserts the current working directory (the path, not an empty string) at sys.path[0], followed by the module directory at sys.path[1]. Only one entry is added if they're the same directory. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/63CBL373SWD7P24TMQOHCJYDP76J4NTL/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
If the problem is mostly type annotations, then another potential solution would be to make use of .pyi files, which are not hamstrung by circular definitions. The idea would be that type checkers would merge the annotations from .pyi files into the annotations in the corresponding .py file. So: a.py: from b import B class A: value: B b.py: class B: value = None b.pyi: from typing import Optional from a import A class B: value: Optional[A] = ... The pyi files would kind of act like header files that are used in other languages. It would mean that type checkers need to check the .pyi files against the code in the .py files to verify that they're consistent with one another. -thomas ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/VWPWN5KWTRPP6VS4PEHJA4SRVMUDU5WR/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
> On 26 Apr 2022, at 07:32, Larry Hastings wrote: > > […] > What could go wrong? My biggest question so far: is there such a thing as a > metaclass written in C, besides type itself? Are there metaclasses with a > __new__ that doesn't call super().__new__ or three-argument type? If there > are are metaclasses that allocate their own class objects out of raw bytes, > they'd likely sidestep this entire process. I suspect this is rare, if > indeed it has ever been done. Anyway, that'd break this mechanism, so exotic > metaclasses like these wouldn't work with "forward-declared classes". But at > least they needn't fail silently. We just need to add a guard after the call > to metaclass.__new__: if we passed in "__forward__=C" into metaclass.__new__, > and metaclass.__new__ didn't return C, we raise an exception. > There are third party metaclasses written in C, one example is PyObjC which has meta classes written in C and those meta classes create a type with additional entries in the C struct for the type. I haven’t yet tried to think about the impact of this proposal, other than the size of the type (as mentioned earlier). The PyObjC meta class constructs both the Python class and a corresponding Objective-C class in lock step. On first glance this forward class proposal should not cause any problems here other than the size of the type object. Ronald — Twitter / micro.blog: @ronaldoussoren Blog: https://blog.ronaldoussoren.net/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/33PUXWIV6OAVRFJAQWYIV7CMLDKBI2ER/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Tue, 26 Apr 2022 at 22:56, Greg Ewing wrote: > > On 26/04/22 12:33 pm, Chris Angelico wrote: > > That's exactly what I mean though: if the only difference between > > "monkeypatching" and "not monkeypatching" is whether it was intended, > > then the only difference is what you call it. > > No, it's not just a matter of what you call it. > > If I lose my keys and have to break into my house, it's not > illegal. But if someone else breaks into my house without my > permission, that is illegal. It doesn't matter if the thief > *calls* it legal, there's still a difference. > That would be the case if monkeypatching were illegal. Since it's not, wherein lies the difference? ChrisA ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/4FZTF4TIQY7CEZCQX4FLKS2JOUWZGQ5P/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
On Tue, Apr 26, 2022 at 4:04 AM wrote: > Larry Hastings wrote: > > [...] > > > > Now comes the one thing that we might call a "trick". The trick: when > > we allocate the ForwardClass instance C, we make it as big as a class > > object can ever get. (Mark Shannon assures me this is simply "heap > > type", and he knows far more about CPython internals than I ever will.) This proposal will indeed surpass almost all concerns I raised earlier, if not all. It is perfectly legal to have a custom metaclass not passing "**kwargs" to type.__new__, and I think most classes don't do it: these in general do not expect a "**kwargs" in general and will simply not run the first time the code is called, and could be updated at that point. Besides, there is no problem in keeping this compatible for "pre" and "pos' this PEP. I had thought of that possibility before typing my other answers, and the major problem of "inplace modifying' is that one can't know for sure the final size of the class due to "__slots__". Over allocating a couple hundred bytes could make for reasonable slots, but it could simply raise a runtime exception if the final call would require more slots than this maximum size - so, I don't think this is a blocking concern. As for > What could go wrong? My biggest question so far: is there such a thing as > a metaclass written in C, besides type itself? Are there metaclasses with > a __new__ that *doesn't* call super().__new__ or three-argument type? If > there are are metaclasses that allocate their own class objects out of raw > bytes, they'd likely sidestep this entire process. I suspect this is rare, > if indeed it has ever been done. Anyway, that'd break this mechanism, so > exotic metaclasses like these wouldn't work with "forward-declared > classes". But at least they needn't fail silently. We just need to add a > guard after the call to metaclass.__new__: if we passed in "__forward__=C" > into metaclass.__new__, and metaclass.__new__ didn't return C, we raise an > exception. That. There are some metaclasses. I did not check, but I do suspect even some of the builtin-types like "list" and "tuple" do bypass "type.__new__" (they fail when used with multiple inheritance along with ABCs, in that they do not check for abstractmethods). This is fixable. Third party "raw' metaclasses that re-do the object structure could simply runtime err until, _and if ever desired_, rewritten to support this feature as you put it. Even for metaclasses in pure Python, sometimes they won't resolve to an instance of the class itself (think of the behavior seen in pathlib.Path which acts as a factory to a subclass)- I see no problem in these just not supporting this feature as well. == With this in mind, the major concerns are those put by Carl Meyer on the "Part 1" thread, namely, that this might not be usable for static checking at all - https://mail.python.org/archives/list/python-dev@python.org/message/NMCS77YFM2V54PUB66AXEFTE4NXFHWPI/ And, of course, my suggestion that the problem this tries to resolve is already resolved by the use of typing.Protocol, as far as type-checking is concerned. Adding a way for a Protocol to be able to find its registered implementations and instantiate one of them when needed would solve this for "real forward referencing" as well. All in all, I still think this complicates things too much for little gain - people needing real forward references always could find a way out, since Python 2 times. And, while in my earlier e-mails I wrote that I mentioned PEP 563 could also resolve this, I really was thinking about PEP 649 - although, I think, actually, any of the 2 could solve the problem annotation wise. If needed for "real code" instead of annotations, extending Protocol so that it can find implementations, could work as well. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/2DILNMWKRO4MZBTDPYHV7K3WCGR2UOHY/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On Tue, Apr 26, 2022 at 9:55 AM Greg Ewing wrote: > On 26/04/22 12:33 pm, Chris Angelico wrote: > > That's exactly what I mean though: if the only difference between > > "monkeypatching" and "not monkeypatching" is whether it was intended, > > then the only difference is what you call it. > > No, it's not just a matter of what you call it. > > If I lose my keys and have to break into my house, it's not > illegal. But if someone else breaks into my house without my > permission, that is illegal. It doesn't matter if the thief > *calls* it legal, there's still a difference. > MonkeyPatching in Python is not illegal in this sense. As was put in this thread: let's not criminalize people for mutating mutable objects. btw, wether to call certain aspects of one of the options for this proposal "monkeypatching" or not, is a discussion that IMHO is far, far beyond bike-shedding. > > -- > Greg > > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/7HG35MGIE2HROG5LFIAIJH5IVGUUU32C/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 1: Forward declaration of classes
On 26/04/22 12:33 pm, Chris Angelico wrote: That's exactly what I mean though: if the only difference between "monkeypatching" and "not monkeypatching" is whether it was intended, then the only difference is what you call it. No, it's not just a matter of what you call it. If I lose my keys and have to break into my house, it's not illegal. But if someone else breaks into my house without my permission, that is illegal. It doesn't matter if the thief *calls* it legal, there's still a difference. -- Greg ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/HE7Z6VDDYN7RE3YIIWUKZDYBBCQCPWEG/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposal to deprecate mailcap
I didn't spot anyone else report this to mitmproxy so I raised an issue to make them aware: https://github.com/mitmproxy/mitmproxy/issues/5297 On Tue, Apr 26, 2022 at 3:30 AM Victor Stinner wrote: > On Tue, Apr 26, 2022 at 5:47 AM Brett Cannon wrote: > > After talking about this in the SC today, we agreed to deprecate mailcap > under the auspices of PEP 594: > https://github.com/python/peps/commit/701999a91dc5f976c00d5bde1510226ebd9c7822 > . > > Good. I proposed https://github.com/python/cpython/pull/91951 to > implement the deprecation in Python 3.11. > > Fixing or documenting the shell injection vulnerability CVE-2015-20107 > is still being discussed at: > https://github.com/python/cpython/issues/68966 > > Victor > ___ > Python-Dev mailing list -- python-dev@python.org > To unsubscribe send an email to python-dev-le...@python.org > https://mail.python.org/mailman3/lists/python-dev.python.org/ > Message archived at > https://mail.python.org/archives/list/python-dev@python.org/message/Y4IAWAWYTNKSIAVTXJGV2ZMQTFV5WYTT/ > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/INWQMSKTJCJGSVIZUDKV5V6FIJVYO3VA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Add -P command line option to not add sys.path[0]
Hi,
There are 4 main ways to run Python:
(1) python -m module [...]
(2) python script.py [...]
(3) python -c code [...]
(4) python [...]
(1) and (2) insert the directory of the module/script at sys.path[0].
(3) and (4) insert an empty string at sys.path[0].
This behavior is convenient and is maybe part of Python usability
success: importing a module in the current directory is as easy as
"import other_module" (load other_module.py). But it's also a threat
to security: an attacker can override a stdlib module by creating a
Python script with the same name than a stdlib module, like os.py or
shutil.py.
People learning Python commonly create a file with the same name than
a stdlib module (ex: random.py) and then are clueless in face of an
ImportError exception.
Changing the default behavior was discussed multiple times. No
consensus was reached, maybe because most users like the current
default behavior and are not affected by corner cases (see below).
I propose adding a -P option to Python command line interface to "not
add sys.path[0]":
https://github.com/python/cpython/pull/31542
See the documentation in the PR for the exact behavior of this option.
I prefer to add an environment variable, only pass the option
explicitly on the command line.
Since Python 3.4, there is already the -I ("isolated mode") option:
https://docs.python.org/dev/using/cmdline.html#cmdoption-I
The -I option has other effects like disabling user site directories,
it option doesn't fit use cases of the -P option.
One annoying issue of the Python default behavior is that running a
script in /usr/bin/ as root can create or override .pyc files in the
/usr directory, even in the /usr/bin/ directory. Example of this
surprising and annoying issue:
https://github.com/benjaminp/six/issues/359#issuecomment-996159668
The -P option can be used in #!/usr/bin/python shebang to avoid this issue.
--
An alternative would be to change the default behavior to not add
sys.path[0], and add an option to opt-in for Python 3.10 behavior.
Here are my notes about it:
https://github.com/vstinner/misc/blob/main/cpython/pep_path0.rst
What do you think?
Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/IU5Q2AXAURFVDPRWNU3BDFVKV2QX5NOR/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proposal to deprecate mailcap
On Tue, Apr 26, 2022 at 5:47 AM Brett Cannon wrote: > After talking about this in the SC today, we agreed to deprecate mailcap > under the auspices of PEP 594: > https://github.com/python/peps/commit/701999a91dc5f976c00d5bde1510226ebd9c7822 > . Good. I proposed https://github.com/python/cpython/pull/91951 to implement the deprecation in Python 3.11. Fixing or documenting the shell injection vulnerability CVE-2015-20107 is still being discussed at: https://github.com/python/cpython/issues/68966 Victor ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Y4IAWAWYTNKSIAVTXJGV2ZMQTFV5WYTT/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
> On 26 Apr 2022, at 07:32, Larry Hastings wrote: > > [… snip …] > Next we have the "continue" class statement. I'm going to spell it like this: > > continue class C(BaseClass, ..., metaclass=MyMetaclass): > # class body goes here > ... > > I'll mention other possible spellings later. The first change I'll point out > here: we've moved the base classes and the metaclass from the "forward" > statement to the "continue" statement. Technically we could put them either > place if we really cared to. But moving them here seems better, for reasons > you'll see in a minute. > > Other than that, this "continue class" statement is similar to what I (we) > proposed before. For example, here C is an expression, not a name. > > Now comes the one thing that we might call a "trick". The trick: when we > allocate the ForwardClass instance C, we make it as big as a class object can > ever get. (Mark Shannon assures me this is simply "heap type", and he knows > far more about CPython internals than I ever will.) Then, when we get to the > "continue class" statement, we convince metaclass.__new__ call to reuse this > memory, and preserve the reference count, but to change the type of the > object to "type" (or what-have-you). C has now been changed from a > "ForwardClass" object into a real type. (Which almost certainly means C is > now mutable.) > A problem with this trick is that you don’t know how large a class object can get because a subclass of type might add new slots. This is currently not possible to do in Python code (non-empty ``__slots__`` in a type subclass is rejected at runtime), but you can do this in C code. Ronald — Twitter / micro.blog: @ronaldoussoren Blog: https://blog.ronaldoussoren.net/ ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JMO4V4S6OHOFFSKQ3XGU2AEEQVYUAY6J/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: Proto-PEP part 4: The wonderful third option
Larry Hastings wrote: > [...] > > Now comes the one thing that we might call a "trick". The trick: when > we allocate the ForwardClass instance C, we make it as big as a class > object can ever get. (Mark Shannon assures me this is simply "heap > type", and he knows far more about CPython internals than I ever will.) It's possible that I'm misunderstanding the allocation mechanism (and it sounds like you've discussed it with people that know a lot more about the internals than me), but if C is an instance of the metaclass then surely you have to know the metaclass to know this. And it looks like the metaclass can definitely be a C type (and thus have a C struct defining an instance). A presumably the metaclass could be a variable size C type (i.e. like tuple), so possibly not even known until the instance is made. David ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Y4DXOFP6CFX3GKPLFP5MYTOEESRSBW7O/ Code of Conduct: http://python.org/psf/codeofconduct/
