[Python-ideas] Re: Proposed class for collections: dynamicdict
On Mon, Apr 13, 2020 at 06:43:53PM -0700, Caleb Donovick wrote:
> > Why can’t you just subclass dict and override that?
>
> Because TypeError: multiple bases have instance lay-out conflict is one of
> my least favorite errors.
For the benefit of the people on this list who aren't as familiar with
your code as you are, could you explain how that comment is relevant?
I don't like "multiple bases have instance lay-out conflict" errors
either. But I don't get them from subclassing dict.
py> class MyDict(dict):
... def __missing__(self, key):
... return (key, None)
...
py> MyDict()[999]
(999, None)
Works fine.
Normally, if I get that error, it's a sign that I'm probably using too
much multiple inheritence and not enough composition, or even that I
should step away from from the OO paradigm and reconsider whether or not
I actually need a hybrid list+float object :-)
So can you please explain why how this exception is relevant, and how
the proposal here will fix it?
defaultdict is subject to the same "lay-out conflict" issue, so from my
naive point of view, this proposal won't help you avoid the error.
py> from collections import defaultdict
py> class Weird(int, defaultdict):
... pass
...
Traceback (most recent call last):
File "", line 1, in
TypeError: multiple bases have instance lay-out conflict
> Perhaps `__missing__` could be a first class part of the getitem of
> protocol, instead of a `dict` specific feature. So that
> ```
> r = x[key]
> ```
> means:
> ```
> try:
> r = x.__getitem__(key)
> except KeyError as e: # should we also catch IndexError?
> try:
> missing = x.__missing__
> except AttributeError:
> raise e from None
> r = missing(key)
> ```
I don't see how this is much different from what dicts already do. You
can't add a `__missing__` attribute to literal dicts (or lists, tuples,
etc) since they don't take attributes.
{}.__missing__ = lambda key: key
fails, so you have to subclass anyway.
> Obviously this would come at some performance cost for non dict mappings so
> I don't know if this would fly.
Giving every single dict a `__missing__` member, for the sake of the
rare instance that needs one, would come at a performance and memory
cost too.
It would be nice if we could work out what the problem we are trying to
solve is before we jump straight into solutions mode.
To my mind, the naive problem "if `d[key]` fails, call a method with
`key` as argument" already has a solution. If the `__missing__` dunder
isn't a solution, I don't know what the problem is.
--
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/DF4TMSDTNOQ53VTESLT3I4DZFPNEXBVE/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Proposed class for collections: dynamicdict
>> On Apr 15, 2020, at 14:08, Caleb Donovick wrote: >> >> Besides performance, I don’t think it fits with Guido’s conception of the >> protocols as being more minimal than the builtin types—e.g., set has not >> just a & operator, but also an intersection method that takes 0 or more >> arbitrary iterables; the set protocol has no such method, so >> collections.abc.Set neither specifies nor provides an intersection method). >> It’s a bit muddy of a conception at the edges, but I think this goes over >> the line, and maybe have been explicitly thought about and rejected for the >> same reason as Set.intersection. > > Making __missing__ a first class part of how __getitem__ seems more analogous > to __getattr__ and __getattribute__ than the intersection method. That’s a good point, but… > Is there any other dunder that is only implicitly called on builtin types? Actually, IIRC, __getattr__ is itself another example. Python never calls __getattr__, only __getattribute__. However, object.__getattribute__ (and type.__getattribute__) calls __getattr__ on failure, the exact same way dict.__getitem__ calls __missing__ on failure. (Obviously there are differences in practice—e.g., every object is-a object but not every mapping is-a dict, and it’s pretty rare to implement __getattribute__ without super… but if you do implement __getattribute__ without super, or do the equivalent in a C extension class, __getattr__ doesn’t get called, just like __missing__; I’m pretty sure the docs explain this somewhere.) But there are fallbacks that really are a “first class part of the protocol” as you intended: iteration falls back to __getitem__, construction calls __init__ if __new__ returns an instance, addition tries __radd__ before and/or after __add__, etc., and that’s all definitely part of the protocol, baked into the interpreter itself, not just part of how some class implements the protocol. So your proposal is perfectly coherent and reasonable (at least if you change it from “like __getattr__” to “like __radd__” or something), and this is really just a footnote. But I think I’m still against it, both for the “protocols as minimal as possible” reason, and because there’s no obvious way to make it a first class part of the protocol for mappings without making it a first class part of the protocol for sequences (because at the level of the interpreter, and the syntax, they’re both the same subscription protocol). As a side note to the footnote, check out the pickle protocol. IIRC, the fallback from __reduce_ex__ to __reduce__ is a first-class part of the protocol (like iteration, construction, and arithmetic), but the fallback from __getnewargs__ or __getstate__ is just part of some builtin types’ implementations (like attribution and subscription). ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/AOJA2LRC4Z3S3CTZGHXXKAVKPMJUXSLM/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
On 4/15/2020 1:52 PM, Ricky Teachey wrote:
Therefore, it seems weird for __init__ not to call super.
I was not part of it, but I know this was heavily discussed during the
development of dataclasses prior to 3.7, and it was decided not to do
this, at least at THAT time (not saying that can't change, but there
were reasons). If Eric V Smith is about I'm sure he could provide
insight, or links to the relevant conversations.
Sorry, I've been tied up on other things.
In general, it's not possible to know how to call super.__init__() if
you don't a priori know the arguments it takes. That's why dataclasses
doesn't guess. The only general-purpose way to do it is to use *args and
**kwargs. Since a goal of dataclasses is to use good typing information
that works with type checkers, that seems to defeat part of the purpose.
One thing you could do in this scenario is to use a classmethod as an
"alternate constructor". This basically gives you any post- and pre-
init functionality you want, using any regular parameters or *args and
**kwargs as fits your case. This example shows how to do it with *args
and **kwargs, but you could just as easily do it with known param names:
from dataclasses import dataclass
class BaseClass:
def __init__(self, x, y):
print('initializing base class', x, y)
self.x = x
self.y = y
@dataclass
class MyClass(BaseClass):
a: int
b: int
c: int
@classmethod
def new(cls, a, b, c, *args, **kwargs):
self = MyClass(a, b, c)
super(cls, self).__init__(*args, **kwargs)
return self
c = MyClass.new(1, 2, 3, 10, 20)
print(c)
print(c.x, c.y)
If you got really excited about it, you could probably write a decorator
to do some of this boilerplate for you (not that there's a lot of it in
this example). But I don't think this would get added to dataclasses itself.
To Brett's point: has anyone looked to see what attrs does here? When
last I looked, they didn't call any super __init__, but maybe they've
added something to address this.
Eric
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/I72ZMHYLTQO46YZUCJG2GLRQMLIMJR4B/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Proposed class for collections: dynamicdict
> Besides performance, I don’t think it fits with Guido’s conception of the protocols as being more minimal than the builtin types—e.g., set has not just a & operator, but also an intersection method that takes 0 or more arbitrary iterables; the set protocol has no such method, so collections.abc.Set neither specifies nor provides an intersection method). It’s a bit muddy of a conception at the edges, but I think this goes over the line, and maybe have been explicitly thought about and rejected for the same reason as Set.intersection. Making __missing__ a first class part of how __getitem__ seems more analogous to __getattr__ and __getattribute__ than the intersection method. Is there any other dunder that is only implicitly called on builtin types? I mostly agree with everything else you said. On Tue, Apr 14, 2020 at 11:34 AM Steele Farnsworth wrote: > I've implemented the class as a stand-alone module here: > https://github.com/swfarnsworth/dynamicdict > > It could in theory be made significantly more concise if `defdict_type` > were the base for this class instead of `PyDict_Type`. > > > > On Tue, Apr 14, 2020 at 1:32 PM Andrew Barnert via Python-ideas < > python-ideas@python.org> wrote: > >> On Apr 13, 2020, at 18:44, Caleb Donovick >> wrote: >> >> >> I have built this data structure countless times. So I am in favor. >> >> >> Maybe you can give a concrete example of what you need it for, then? I >> think that would really help the proposal. Especially if your example needs >> a per-instance rather than per-class factory function. >> >> > Why can’t you just subclass dict and override that? >> >> Because TypeError: multiple bases have instance lay-out conflict is one >> of my least favorite errors. >> >> >> But defaultdict, being a subclass or dict, has the same problem in the >> same situations, and (although I haven’t checked) I assume the same is true >> for the OP’s dynamicdict. >> >> Perhaps `__missing__` could be a first class part of the getitem of >> protocol, instead of a `dict` specific feature. So that >> >> ``` >> r = x[key] >> ``` >> means: >> ``` >> try: >> r = x.__getitem__(key) >> except KeyError as e: # should we also catch IndexError? >> try: >> missing = x.__missing__ >> except AttributeError: >> raise e from None >> r = missing(key) >> ``` >> >> Obviously this would come at some performance cost for non dict mappings >> so I don't know if this would fly. >> >> >> Besides performance, I don’t think it fits with Guido’s conception of the >> protocols as being more minimal than the builtin types—e.g., set has not >> just a & operator, but also an intersection method that takes 0 or more >> arbitrary iterables; the set protocol has no such method, so >> collections.abc.Set neither specifies nor provides an intersection method). >> It’s a bit muddy of a conception at the edges, but I think this goes over >> the line, and maybe have been explicitly thought about and rejected for the >> same reason as Set.intersection. >> >> On the other hand, none of that is an argument or any kind against your >> method decorator: >> >> So instead maybe there could have standard decorator to get the same >> behavior? >> ``` >> def usemissing(getitem): >> @wraps(getitem) >> def wrapped(self, key): >> try: >> return getitem(self, key) >> except KeyError as e: >> try: >> missing = self.__missing__ >> except AttributeError: >> raise e from None >> return missing(key) >> return wrapped >> ``` >> >> >> This seems like a great idea, although maybe it would be easier to use as >> a class decorator rather than a method decorator. Either this: >> >> def usemissing(cls): >> missing = cls.__missing__ >> getitem = cls.__getitem__ >> def __getitem__(self, key): >> try: >> return getitem(self, key) >> except KeyError: >> return missing(self, key) >> cls.__getitem__ = __getitem__ >> return cls >> >> Or this: >> >>def usemissing(cls): >> getitem = cls.__getitem__ >> def __getitem__(self, key): >> try: >> return getitem(self, key) >> except KeyError: >> return type(self).__missing__(self, key) >> cls.__getitem__ = __getitem__ >> return cls >> >> This also preserves the usual class-based rather than instance-based >> lookup for most special methods (including __missing__ on dict subclasses). >> >> The first one has the advantage of failing at class decoration time >> rather than at first missing lookup time if you forget to include a >> __missing__, but it has the cost that (unlike a dict subclass) you can’t >> monkeypatch __missing__ after construction time. So I think I’d almost >> always prefer the first, but the second might be a better fit for the >> stdlib anyway? >> >> I think either the method decorator or the class decorator makes sense >> for
[Python-ideas] Re: Should dataclass init call super?
Scanning the messages, I met this from Ricky Teachey:
I did write a decorator of my own that replaces the dataclass init with one
> that calls super().__init__(*args, **kwargs) first before proceeding with
> the one written by dataclasses... I can't find it at the moment. But that
> has its own problems; one being the IDE doesn't know the init has been
> rewritten this way and so will complain about parameters sent to the
> dataclass that it doesn't know about.
yep. My approach is the same and will fail the same way - but that is only
so much IDEs can go
if they keep tring to statically guess parameters to a class' __init__ .
Anyway, I think it is a way to go if one
needs the collaborative dataclasses now.
On Wed, 15 Apr 2020 at 17:45, Joao S. O. Bueno
wrote:
> There is a way to have dataclasses as they are now behave collaboratively
> with
> a further decorator.
>
> For Python 3.8 livefcycle such decorator could live in an external package
> -
> if its good, it could go into the stdlib, or maybe, another
> "dataclasses.collaborative_dataclass" would already create a collaborative
> dataclass without breaking backwards compatibility.
>
>
> here is some sample code, I've tested locally:
>
> ```
> def colaborative(cls):
> if not hasattr(cls, "__dataclass_fields__") or not "__init__" in
> cls.__dict__:
> return cls
> cls._dataclass_init = cls.__init__
> super_ = cls.__mro__[1]
> @wraps(cls.__init__)
> def __init__(self, *args, **kw):
> # use inspect.signature stuff to proper retrieve 'args' into
> dataclass kw if needed, else:
> dataclass_kw = {}
> for key, value in list(kw.items()):
> if key in cls.__annotations__:
> dataclass_kw[key] = kw.pop(key)
> self._dataclass_init(**dataclass_kw)
> super_.__init__(self, *args, **kw)
> cls.__init__ = __init__
> return cls
>
> ```
>
> https://gist.github.com/jsbueno/5a207e6a2c6c433a7549c78ba2edab7d
>
> On Wed, 15 Apr 2020 at 16:40, Andrew Barnert via Python-ideas <
> python-ideas@python.org> wrote:
>
>> On Apr 15, 2020, at 10:16, Brett Cannon wrote:
>>
>>
>> On Wed, Apr 15, 2020 at 8:45 AM Christopher Barker
>> wrote:
>>
>>> > you'd just add *args and **kwargs to the init signature and call
>>> super().__init__(*args, **kwargs).
>>>
Which is what the OP is after.
>>>
>>> Hmm, makes me wonder if there should be an option to define a
>>> __pre_init__ method.
>>>
>>
>> Also note that the 'attr' package on PyPI is still available and provides
>> features that dataclasses do not. Generalizing something in the stdlib is
>> not always the best/necessary solution, especially if there's a
>> battle-tested alternative available on PyPI.
>>
>>
>> Wasn’t dataclass designed with customization/extension hooks for apps or
>> libraries to use, like the field metadata? Are any libs on PyPI taking
>> advantage of that? If not, maybe this would be a good test case for that
>> functionality. If it turns out to be easy and obvious, then as soon as
>> someone’s got something stable and popular, it could be proposed for a
>> merge into the stdlib—but if it turns out that there are multiple good ways
>> to handle it they could stay as competitors on PyPI forever, while if it
>> turns out that the extension hooks aren’t sufficient, someone could propose
>> exactly what needs to be changed to make the extension writable.
>>
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/python-ideas@python.org/message/27CGQQY63GEBDNNU6MYYOY6YS63VZFQU/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/RP5LEXLJ4WTXIK2OBSGGVNRTUQ6UJP3U/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
There is a way to have dataclasses as they are now behave collaboratively
with
a further decorator.
For Python 3.8 livefcycle such decorator could live in an external package
-
if its good, it could go into the stdlib, or maybe, another
"dataclasses.collaborative_dataclass" would already create a collaborative
dataclass without breaking backwards compatibility.
here is some sample code, I've tested locally:
```
def colaborative(cls):
if not hasattr(cls, "__dataclass_fields__") or not "__init__" in
cls.__dict__:
return cls
cls._dataclass_init = cls.__init__
super_ = cls.__mro__[1]
@wraps(cls.__init__)
def __init__(self, *args, **kw):
# use inspect.signature stuff to proper retrieve 'args' into
dataclass kw if needed, else:
dataclass_kw = {}
for key, value in list(kw.items()):
if key in cls.__annotations__:
dataclass_kw[key] = kw.pop(key)
self._dataclass_init(**dataclass_kw)
super_.__init__(self, *args, **kw)
cls.__init__ = __init__
return cls
```
https://gist.github.com/jsbueno/5a207e6a2c6c433a7549c78ba2edab7d
On Wed, 15 Apr 2020 at 16:40, Andrew Barnert via Python-ideas <
python-ideas@python.org> wrote:
> On Apr 15, 2020, at 10:16, Brett Cannon wrote:
>
>
> On Wed, Apr 15, 2020 at 8:45 AM Christopher Barker
> wrote:
>
>> > you'd just add *args and **kwargs to the init signature and call
>> super().__init__(*args, **kwargs).
>>
>>>
>>> Which is what the OP is after.
>>>
>>
>> Hmm, makes me wonder if there should be an option to define a
>> __pre_init__ method.
>>
>
> Also note that the 'attr' package on PyPI is still available and provides
> features that dataclasses do not. Generalizing something in the stdlib is
> not always the best/necessary solution, especially if there's a
> battle-tested alternative available on PyPI.
>
>
> Wasn’t dataclass designed with customization/extension hooks for apps or
> libraries to use, like the field metadata? Are any libs on PyPI taking
> advantage of that? If not, maybe this would be a good test case for that
> functionality. If it turns out to be easy and obvious, then as soon as
> someone’s got something stable and popular, it could be proposed for a
> merge into the stdlib—but if it turns out that there are multiple good ways
> to handle it they could stay as competitors on PyPI forever, while if it
> turns out that the extension hooks aren’t sufficient, someone could propose
> exactly what needs to be changed to make the extension writable.
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/27CGQQY63GEBDNNU6MYYOY6YS63VZFQU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/OCXNXU7LP6Z4FSU4HHU3UM43H3SEEATV/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
On Apr 15, 2020, at 10:16, Brett Cannon wrote: > > >> On Wed, Apr 15, 2020 at 8:45 AM Christopher Barker >> wrote: >> > you'd just add *args and **kwargs to the init signature and call >> > super().__init__(*args, **kwargs). >>> >>> Which is what the OP is after. >> >> Hmm, makes me wonder if there should be an option to define a __pre_init__ >> method. > > Also note that the 'attr' package on PyPI is still available and provides > features that dataclasses do not. Generalizing something in the stdlib is not > always the best/necessary solution, especially if there's a battle-tested > alternative available on PyPI. Wasn’t dataclass designed with customization/extension hooks for apps or libraries to use, like the field metadata? Are any libs on PyPI taking advantage of that? If not, maybe this would be a good test case for that functionality. If it turns out to be easy and obvious, then as soon as someone’s got something stable and popular, it could be proposed for a merge into the stdlib—but if it turns out that there are multiple good ways to handle it they could stay as competitors on PyPI forever, while if it turns out that the extension hooks aren’t sufficient, someone could propose exactly what needs to be changed to make the extension writable. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/27CGQQY63GEBDNNU6MYYOY6YS63VZFQU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: TLS session resumption
On Apr 15, 2020, at 12:11, Chris Angelico wrote: > > On Thu, Apr 16, 2020 at 4:55 AM Ander Juaristi wrote: >> TLS session resumption is currently supported, but only within the same >> process. To the best of my knowledge, there is no way to save the TLS >> session to a file to resume the TLS session later on. Please tell me how >> this is done if I'm wrong. > > Not a Python SSL expert, but have you tried pickling the session object? > > If that doesn't work, then I would say that adding pickle support > (using the semantics you describe) would be the cleanest way to do > this. I think this avoids the problem of “how do we effectively tell people that this could be dangerous if the pickles are stored somewhere accessible (or, worse, write-accessible)”, because it becomes just a specific case of the general danger the pickle module already warns about? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/OWCX3EUAFEGBBY7Z4LH4MZ5EIVMWSCT5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: TLS session resumption
On Thu, Apr 16, 2020 at 4:55 AM Ander Juaristi wrote: > TLS session resumption is currently supported, but only within the same > process. To the best of my knowledge, there is no way to save the TLS > session to a file to resume the TLS session later on. Please tell me how > this is done if I'm wrong. Not a Python SSL expert, but have you tried pickling the session object? If that doesn't work, then I would say that adding pickle support (using the semantics you describe) would be the cleanest way to do this. ChrisA ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DQ3RIE4USTWRCSGE25VE3AFQNK7VHT3U/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
> I think ultimately the argument you want to make really is the > “enlightened laziness” one: there are lots of optional Pandas-y parameters > in your superclass(es), and most of them you will definitely never care > about, but a few of them you actually might occasionally care about. What > then? Well, if your Y class is part of mission-critical interface code that > lives depend on, you probably do need to work out which those are and get > them nicely documented and statically checkable, but in a lot of cases it > isn’t nearly worth that much effort—just make Y pass everything through, > and the couple places you end up needing to pass down one of those Pandas-y > arguments you just do so, and it works, and that’s fine. > Concur 100% thanks for saying it in a way that makes sense. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/LTAC5P6GH4TCBRY7YP2RYKUSVP3RIA5F/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
> On Apr 15, 2020, at 10:48, Ricky Teachey wrote: > >>> It becomes more painful the more parameters the parent has- parameters >>> which the dataclass may not even care about. It not only makes the class >>> definition long, it adds so these additional parameters to the init >>> signature, which is icky for introspection and discoverability. Lots of >>> "What the heck is this parameter doing here?" head scratching for future me >>> (because I forget everything). >> >> I think that’s backward. The signature is there for the user of the >> dataclass, not the implementer. And the user had better care about that x >> argument, because it’s a mandatory parameter of the X class, so if they >> don’t pass one, they’re going to get an exception from inside some class >> they never heard of. So having x show up in the signature would be helpful >> for introspection and discovery, not harmful. It makes your users ask “What >> the heck is the x parameter doing here?” but that’s a question that they’d >> better have an answer to or they can’t construct a Y instance. (And notice >> that the X doesn’t take or pass along *args, so if the Y claims to take >> *args as well as **kwargs, that’s even more misleading, because passing any >> extra positional args to the constructor will also raise.) And that’s as >> true for tools as for human readers—an IDE auto-completing the parameters of >> Y(…) should be prompting you for an x; a static analyzer should be catching >> that you forgot to pass as x; etc. > > This is a good critique of what I said! Just want to clarify, though, that I > was thinking the entire time of OPTIONAL arguments-- like, for example, of > the kind heavily used in pandas. There tons of optional arguments in that > library I do not care about, ever. But you are correct about non-optional > arguments. Well, the OP’s example doesn’t have an optional argument, only a non-optional one. But for optional parameters that you really do not care about, ever, you’re never going to pass them to Y, so why do you care that Y won’t pass them along to X? And for optional parameters that are meaningful and useful to Y’s users, surely having them visible in the help, etc. would make things more discoverable, not less? I think ultimately the argument you want to make really is the “enlightened laziness” one: there are lots of optional Pandas-y parameters in your superclass(es), and most of them you will definitely never care about, but a few of them you actually might occasionally care about. What then? Well, if your Y class is part of mission-critical interface code that lives depend on, you probably do need to work out which those are and get them nicely documented and statically checkable, but in a lot of cases it isn’t nearly worth that much effort—just make Y pass everything through, and the couple places you end up needing to pass down one of those Pandas-y arguments you just do so, and it works, and that’s fine. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/RPCINUUZB375RC5VIGDZ3LTKWTEVOW4D/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] TLS session resumption
Hi list,
I haven't seen this being discussed, but TL;DR I just want to propose to
extend the ssl module so that TLS session state can be serialized to disk.
I see a similar issue was being discussed here [0], but cannot tell why
it (apparently) didn't get into. The link to the patch is broken.
TLS session resumption is currently supported, but only within the same
process. To the best of my knowledge, there is no way to save the TLS
session to a file to resume the TLS session later on. Please tell me how
this is done if I'm wrong.
#
# open a TLS connection
#
with socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) as sock:
with context.wrap_socket(sock, server_hostname=hostname) \
as ssock:
ssock.connect((hostname, 443))
ssl_session = ssock.session # get TLS session
#
# later on, open a new connection resuming session
#
with socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) as sock:
with context.wrap_socket(sock,
server_hostname=hostname,
session=ssl_session) as ssock:
ssock.connect((hostname, 443))
print("-- TLS session %s resumed --"
% ("WAS" if ssock.session_reused else "WAS NOT")
However TLS sessions live for
hours, if not days, and hence it may make sense to persist them
to disk for later retrieval. I think this is a real world use case
though I have not any numbers to back this up.
The values currently exposed by SSLSession (id, time, timeout) are not
enough to resume a TLS session. There are a number of parameters that
need to be stored to resume a TLS session at a later time, some of which
are not currently retrievable from python's ssl module. For instance,
the master secret.
I'm not a fan of exposing the master secret directly with a getter as
that could be and will definitely be misused by someone in spite of the
bright red warnings, leading to disaster.
What OpenSSL currently does, as do other implementations, is to allow
the SSL_SESSION object to be serialized, with i2d_SSL_SESSION(). This
function will return a DER-encoded ASN.1 object containing all the
needed state to resume the TLS session. I do believe exposing this
object from the ssl module (as an opaque bytes object) would be a useful
compromise with very little chance of misuse.
One possible incarnation of this idea would be add two new methods to
SSLSession, to_der(), and from_der():
class SSLSession:
def to_der():
""" Returns OpenSSL's DER-encoded SSL_SESSION object """
@staticmethod
def from_der(der_bytes):
"""
Creates a new SSLSession object from the given
DER-encoded SSL_SESSION object
"""
If this has already been discussed (likely), what are the reasons it was
rejected?
Thanks!
[0] https://bugs.python.org/issue19500
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/55RZJODLVNVP72FNE42GUZEO57RDE5W4/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
> > Therefore, it seems weird for __init__ not to call super. > I was not part of it, but I know this was heavily discussed during the development of dataclasses prior to 3.7, and it was decided not to do this, at least at THAT time (not saying that can't change, but there were reasons). If Eric V Smith is about I'm sure he could provide insight, or links to the relevant conversations. > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/V5OJSUSJ7JZMA2Z4H6MT2Q5HCDJXYCU3/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
> > It becomes more painful the more parameters the parent has- parameters > which the dataclass may not even care about. It not only makes the class > definition long, it adds so these additional parameters to the init > signature, which is icky for introspection and discoverability. Lots of > "What the heck is this parameter doing here?" head scratching for future me > (because I forget everything). > > > I think that’s backward. The signature is there for the user of the > dataclass, not the implementer. And the user had better care about that x > argument, because it’s a mandatory parameter of the X class, so if they > don’t pass one, they’re going to get an exception from inside some class > they never heard of. So having x show up in the signature would be helpful > for introspection and discovery, not harmful. It makes your users ask “What > the heck is the x parameter doing here?” but that’s a question that they’d > better have an answer to or they can’t construct a Y instance. (And notice > that the X doesn’t take or pass along *args, so if the Y claims to take > *args as well as **kwargs, that’s even more misleading, because passing any > extra positional args to the constructor will also raise.) And that’s as > true for tools as for human readers—an IDE auto-completing the parameters > of Y(…) should be prompting you for an x; a static analyzer should be > catching that you forgot to pass as x; etc. > This is a good critique of what I said! Just want to clarify, though, that I was thinking the entire time of OPTIONAL arguments-- like, for example, of the kind heavily used in pandas. There tons of optional arguments in that library I do not care about, ever. But you are correct about non-optional arguments. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/KIDHIKHW7WTRYMM3BCDFN2FXD3JCJKNA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
I'm just curious: what is the downside of calling super with kwargs? Usually when I define a class, the first I write is def __init__(self, **kwargs): super().__init__(**kwargs) just in case I want to use the class in cooperative inheritance. I always thought it couldn't hurt? I might be alone in this interpretation, but I imagine that there are three fundamental kinds of inheritance patterns for methods, which I defined in my ipromise package (https://pypi.org/project/ipromise/): implementing an abstract method, overriding, and augmenting. If I had to choose, I would say that __init__ should be an "augmenting" pattern. Therefore, it seems weird for __init__ not to call super. Even if you want Y t to override some behavior in X, what happens if Z inherits from Y and W? Now, Y.__init__'s decision not to call super would mean that W.__init__ would not be called. That seems like a bug. Instead, I would rather put the behavior that Y wants to override in a separate method, say X.f, which is called in X.__init__. Now, if Y.f overrides X.f, everything is okay. Even if Z inherits from Y and W, the override still works, and W.__init__ still gets called, angels sing, etc. Long story short, am I wrong to interpret __init__ as a "must augment" method and always call super().__init__(**kwargs)? On Wed, Apr 15, 2020 at 1:32 PM Andrew Barnert wrote: > On Apr 15, 2020, at 04:26, Ricky Teachey wrote: > > > >> For simple situations you can call super in the __post_init__ method and >>> things will work fine: >>> >> >> But not for the OP's case: he wanted to pass extra parameters in -- and >> the dataclass' __init__ won't accept extra arguments. >> >> >> Can’t you just create InitVar attributes for the extra args you want to >> pass through in that case? >> > > InitVar fields for all the desired parent class init parameters can often > solve the problem. > > But it can be painful to have to manually provide every parameter > explicitly when normally (when not using a dataclass) you'd just add *args > and **kwargs to the init signature and call super().__init__(*args, > **kwargs). > > > To handle that case, couldn’t we just add InitVarStar and InitVarStarStar > fields? If present, any extra positional and/or keyword args get captured > for the __postinit__ to pass along just like normal InitVars. > > I think that could definitely be useful, but not as useful as you seem to > think it would be. > > It becomes more painful the more parameters the parent has- parameters > which the dataclass may not even care about. It not only makes the class > definition long, it adds so these additional parameters to the init > signature, which is icky for introspection and discoverability. Lots of > "What the heck is this parameter doing here?" head scratching for future me > (because I forget everything). > > > I think that’s backward. The signature is there for the user of the > dataclass, not the implementer. And the user had better care about that x > argument, because it’s a mandatory parameter of the X class, so if they > don’t pass one, they’re going to get an exception from inside some class > they never heard of. So having x show up in the signature would be helpful > for introspection and discovery, not harmful. It makes your users ask “What > the heck is the x parameter doing here?” but that’s a question that they’d > better have an answer to or they can’t construct a Y instance. (And notice > that the X doesn’t take or pass along *args, so if the Y claims to take > *args as well as **kwargs, that’s even more misleading, because passing any > extra positional args to the constructor will also raise.) And that’s as > true for tools as for human readers—an IDE auto-completing the parameters > of Y(…) should be prompting you for an x; a static analyzer should be > catching that you forgot to pass as x; etc. > > There are cases where you need *args and/or **kwargs in a constructor. But > I don’t think they make sense for a dataclsss. For example, you’re not > going to write a functools.partial replacement or a generic RPC proxy > object as a dataclass. > > But there are cases where you _want_ them, just out of… call it > enlightened laziness. And those cases do seem to apply to dataclass at > least as much as normal classes. And that’s why I think it could be worth > having these new fields. The OP’s toy example looks like part if a design > where the X is one of a bag of mixins from some library that you compose up > however you want in your app. > You got it. > It would be more discoverable and readable if the final composed Y class > knew which parameters its mixins demanded—but it may not be worth the > effort to put that together (either manually, or programmatically at class > def time). If your Y class is being exposed as part of a library (or RPC or > bridge or whatever), or it forms part of the connection between two key > components in an air traffic control system, then you probably do want to >
[Python-ideas] Re: Should dataclass init call super?
> On Apr 15, 2020, at 04:26, Ricky Teachey wrote: > For simple situations you can call super in the __post_init__ method and things will work fine: >>> >>> But not for the OP's case: he wanted to pass extra parameters in -- and the >>> dataclass' __init__ won't accept extra arguments. >> >> Can’t you just create InitVar attributes for the extra args you want to pass >> through in that case? > > > InitVar fields for all the desired parent class init parameters can often > solve the problem. > > But it can be painful to have to manually provide every parameter explicitly > when normally (when not using a dataclass) you'd just add *args and **kwargs > to the init signature and call super().__init__(*args, **kwargs). To handle that case, couldn’t we just add InitVarStar and InitVarStarStar fields? If present, any extra positional and/or keyword args get captured for the __postinit__ to pass along just like normal InitVars. I think that could definitely be useful, but not as useful as you seem to think it would be. > It becomes more painful the more parameters the parent has- parameters which > the dataclass may not even care about. It not only makes the class definition > long, it adds so these additional parameters to the init signature, which is > icky for introspection and discoverability. Lots of "What the heck is this > parameter doing here?" head scratching for future me (because I forget > everything). I think that’s backward. The signature is there for the user of the dataclass, not the implementer. And the user had better care about that x argument, because it’s a mandatory parameter of the X class, so if they don’t pass one, they’re going to get an exception from inside some class they never heard of. So having x show up in the signature would be helpful for introspection and discovery, not harmful. It makes your users ask “What the heck is the x parameter doing here?” but that’s a question that they’d better have an answer to or they can’t construct a Y instance. (And notice that the X doesn’t take or pass along *args, so if the Y claims to take *args as well as **kwargs, that’s even more misleading, because passing any extra positional args to the constructor will also raise.) And that’s as true for tools as for human readers—an IDE auto-completing the parameters of Y(…) should be prompting you for an x; a static analyzer should be catching that you forgot to pass as x; etc. There are cases where you need *args and/or **kwargs in a constructor. But I don’t think they make sense for a dataclsss. For example, you’re not going to write a functools.partial replacement or a generic RPC proxy object as a dataclass. But there are cases where you _want_ them, just out of… call it enlightened laziness. And those cases do seem to apply to dataclass at least as much as normal classes. And that’s why I think it could be worth having these new fields. The OP’s toy example looks like part if a design where the X is one of a bag of mixins from some library that you compose up however you want in your app. It would be more discoverable and readable if the final composed Y class knew which parameters its mixins demanded—but it may not be worth the effort to put that together (either manually, or programmatically at class def time). If your Y class is being exposed as part of a library (or RPC or bridge or whatever), or it forms part of the connection between two key components in an air traffic control system, then you probably do want to put in that effort. If it’s an internal class that only gets constructed in one place, in a tool for trolling each other with bad music in the office stereo that only you and three colleagues will ever run, why do the extra work to get earlier checking that you don’t need? The fact that Python leaves that kind of choice up to you to decide (because you’re the only one who knows), and so do most pythonic libraries like the one you got that X mixin out of… that’s a big part of why you wrote that script in Python in the first place. And if dataclasses get in the way of that, it’s a problem, and probably worth fixing. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/2M6D4U65J3MN73AQDMWWQY567DFLBHUF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
On Wed, Apr 15, 2020 at 8:45 AM Christopher Barker
wrote:
> > you'd just add *args and **kwargs to the init signature and call
> super().__init__(*args, **kwargs).
>
>>
>> Which is what the OP is after.
>>
>
> Hmm, makes me wonder if there should be an option to define a __pre_init__
> method.
>
Also note that the 'attr' package on PyPI is still available and provides
features that dataclasses do not. Generalizing something in the stdlib is
not always the best/necessary solution, especially if there's a
battle-tested alternative available on PyPI.
-Brett
>
> Then you could customize the signature, but still use data classes nifty
> features for the primary __init__
>
> And no, I haven’t thought this out, it would be tricky, and maybe
> impossible.
>
> Which brings me back to the suggestion in a PR:
>
> Optional have the __init__ accept *args, *kwargs, and then store them in
> self. Then users could do whatever they like with them in __post_init
>
> -Chris
>
> It becomes more painful the more parameters the parent has- parameters
>> which the dataclass may not even care about. It not only makes the class
>> definition long, it adds so these additional parameters to the init
>> signature, which is icky for introspection and discoverability. Lots of
>> "What the heck is this parameter doing here?" head scratching for future me
>> (because I forget everything).
>>
>> There's currently not a very compelling solution, AFAIK, to be able to
>> use dataclasses in these kinds of situations ("these kinds" = any situation
>> other than the most simple) other than the solution Christopher Barker
>> suggested: using a mixin approach that treats the dataclass parameters
>> specially. So I just haven't.
>>
>> I did write a decorator of my own that replaces the dataclass init with
>> one that calls super().__init__(*args, **kwargs) first before proceeding
>> with the one written by dataclasses... I can't find it at the moment. But
>> that has its own problems; one being the IDE doesn't know the init has been
>> rewritten this way and so will complain about parameters sent to the
>> dataclass that it doesn't know about.
>>
>>> --
> Christopher Barker, PhD
>
> Python Language Consulting
> - Teaching
> - Scientific Software Development
> - Desktop GUI and Web Development
> - wxPython, numpy, scipy, Cython
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/TH4I7L5FMDEAHCLN7EE3DZFZFPPXG5GH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/E6RM5FFELZO5RTCGOQRQ5I6QDVWCTMKV/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
> you'd just add *args and **kwargs to the init signature and call
super().__init__(*args, **kwargs).
>
> Which is what the OP is after.
>
Hmm, makes me wonder if there should be an option to define a __pre_init__
method.
Then you could customize the signature, but still use data classes nifty
features for the primary __init__
And no, I haven’t thought this out, it would be tricky, and maybe
impossible.
Which brings me back to the suggestion in a PR:
Optional have the __init__ accept *args, *kwargs, and then store them in
self. Then users could do whatever they like with them in __post_init
-Chris
It becomes more painful the more parameters the parent has- parameters
> which the dataclass may not even care about. It not only makes the class
> definition long, it adds so these additional parameters to the init
> signature, which is icky for introspection and discoverability. Lots of
> "What the heck is this parameter doing here?" head scratching for future me
> (because I forget everything).
>
> There's currently not a very compelling solution, AFAIK, to be able to use
> dataclasses in these kinds of situations ("these kinds" = any situation
> other than the most simple) other than the solution Christopher Barker
> suggested: using a mixin approach that treats the dataclass parameters
> specially. So I just haven't.
>
> I did write a decorator of my own that replaces the dataclass init with
> one that calls super().__init__(*args, **kwargs) first before proceeding
> with the one written by dataclasses... I can't find it at the moment. But
> that has its own problems; one being the IDE doesn't know the init has been
> rewritten this way and so will complain about parameters sent to the
> dataclass that it doesn't know about.
>
>> --
Christopher Barker, PhD
Python Language Consulting
- Teaching
- Scientific Software Development
- Desktop GUI and Web Development
- wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/TH4I7L5FMDEAHCLN7EE3DZFZFPPXG5GH/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
>
> For simple situations you can call super in the __post_init__ method and
>> things will work fine:
>>
>
> But not for the OP's case: he wanted to pass extra parameters in -- and
> the dataclass' __init__ won't accept extra arguments.
>
>
> Can’t you just create InitVar attributes for the extra args you want to
> pass through in that case?
>
InitVar fields for all the desired parent class init parameters can often
solve the problem.
But it can be painful to have to manually provide every parameter
explicitly when normally (when not using a dataclass) you'd just add *args
and **kwargs to the init signature and call super().__init__(*args,
**kwargs).
Which is what the OP is after.
It becomes more painful the more parameters the parent has- parameters
which the dataclass may not even care about. It not only makes the class
definition long, it adds so these additional parameters to the init
signature, which is icky for introspection and discoverability. Lots of
"What the heck is this parameter doing here?" head scratching for future me
(because I forget everything).
There's currently not a very compelling solution, AFAIK, to be able to use
dataclasses in these kinds of situations ("these kinds" = any situation
other than the most simple) other than the solution Christopher Barker
suggested: using a mixin approach that treats the dataclass parameters
specially. So I just haven't.
I did write a decorator of my own that replaces the dataclass init with one
that calls super().__init__(*args, **kwargs) first before proceeding with
the one written by dataclasses... I can't find it at the moment. But that
has its own problems; one being the IDE doesn't know the init has been
rewritten this way and so will complain about parameters sent to the
dataclass that it doesn't know about.
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/4LVOHYQNTZSFXAOB5USEMQO6USNLBXD2/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Should dataclass init call super?
On Apr 14, 2020, at 20:53, Christopher Barker wrote: > >> On Tue, Apr 14, 2020 at 7:46 PM Ricky Teachey wrote: >> For simple situations you can call super in the __post_init__ method and >> things will work fine: > > But not for the OP's case: he wanted to pass extra parameters in -- and the > dataclass' __init__ won't accept extra arguments. Can’t you just create InitVar attributes for the extra args you want to pass through in that case? ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/T24XVSZ6JYACCWRVY5TL76SLFZEECXL6/ Code of Conduct: http://python.org/psf/codeofconduct/
