Good proposal! I have a few questions.
On Mon, Mar 15, 2021 at 2:22 PM Eric V. Smith <e...@trueblade.com
<mailto:e...@trueblade.com>> wrote:
[I'm sort of loose with the terms field, parameter, and
argument here.
Forgive me: I think it's still understandable. Also I'm not
specifying
types here, I'm using Any everywhere. Use your imagination and
substitute real types if it helps you.]
Here's version 2 of my proposal:
There have been many requests to add keyword-only fields to
dataclasses.
These fields would result in __init__ parameters that are
keyword-only.
In a previous proposal, I suggested also including positional
arguments
for dataclasses. That proposal is at
https://mail.python.org/archives/list/python-ideas@python.org/message/I3RKK4VINZUBCGF2TBJN6HTDV3PVUEUQ/
<https://mail.python.org/archives/list/python-ideas@python.org/message/I3RKK4VINZUBCGF2TBJN6HTDV3PVUEUQ/>
. After some discussion, I think it's clear that positional
arguments
aren't going to work well with dataclasses. The deal breaker
for me is
that the generated repr would either not work with eval(), or
it would
contain fields without names (since they're positional).
There are
additional concerns mentioned in that thread. Accordingly,
I'm going to
drop positional arguments from this proposal.
Basically, I want to add a flag to each field, stating
whether the field
results in a normal parameter or a keyword-only parameter to
__init__.
Then when I'm generating __init__, I'll examine those flags
and put the
normal arguments first, followed by the keyword-only ones.
The trick becomes: how do you specify what type of parameter
each field
represents?
What attrs does
---------------
First, here's what attrs does. There's a parameter to their
attr.ib()
function (the moral equivalent of dataclasses.field()) named
kw_only,
which if set, marks the field as being keyword-only. From
https://www.attrs.org/en/stable/examples.html#keyword-only-attributes
<https://www.attrs.org/en/stable/examples.html#keyword-only-attributes>
:
>>> @attr.s
... class A:
... a = attr.ib(kw_only=True)
>>> A()
Traceback (most recent call last):
...
TypeError: A() missing 1 required keyword-only argument: 'a'
>>> A(a=1)
A(a=1)
There's also a parameter to attr.s (the equivalent of
dataclasses.dataclass), also named kw_only, which if true
marks every
field as being keyword-only:
>>> @attr.s(kw_only=True)
... class A:
... a = attr.ib()
... b = attr.ib()
>>> A(1, 2)
Traceback (most recent call last):
...
TypeError: __init__() takes 1 positional argument but 3 were
given
>>> A(a=1, b=2)
A(a=1, b=2)
dataclasses proposal
--------------------
I propose to adopt both of these methods
(dataclass(kw_ony=True) and
field(kw_only=True) in dataclasses. The above example would
become:
>>> @dataclasses.dataclass
... class A:
... a: Any = field(kw_only=True)
>>> @dataclasses.dataclass(kw_only=True)
... class A:
... a: Any
... b: Any
But, I'd also like to make this a little easier to use,
especially in
the case where you're defining a dataclass that has some
normal fields
and some keyword-only fields. Using the attrs approach, you'd
need to
declare the keyword-only fields using the
"=field(kw_only=True)" syntax,
which I think is needlessly verbose, especially when you have
many
keyword-only fields.
The problem is that if you have 1 normal parameter and 10
keyword-only
ones, you'd be forced to say:
@dataclasses.dataclass
class LotsOfFields:
a: Any
b: Any = field(kw_only=True, default=0)
c: Any = field(kw_only=True, default='foo')
d: Any = field(kw_only=True)
e: Any = field(kw_only=True, default=0.0)
f: Any = field(kw_only=True)
g: Any = field(kw_only=True, default=())
h: Any = field(kw_only=True, default='bar')
i: Any = field(kw_only=True, default=3+4j)
j: Any = field(kw_only=True, default=10)
k: Any = field(kw_only=True)
That's way too verbose for me.
Ideally, I'd like something like this example:
@dataclasses.dataclass
class A:
a: Any
# pragma: KW_ONLY
b: Any
And then b would become a keyword-only field, while a is a
normal field.
But we need some way of telling dataclasses.dataclass what's
going on,
since obviously pragmas are out.
I propose the following. I'll add a singleton to the
dataclasses module:
KW_ONLY. When scanning the __attribute__'s that define the
fields, a
field with this type would be ignored, except for assigning
the kw_only
flag to fields declared after these singletons are used. So
you'd get:
@dataclasses.dataclass
class B:
a: Any
_: dataclasses.KW_ONLY
b: Any
This would generate:
def __init__(self, a, *, b):
This example is equivalent to:
@dataclasses.dataclass
class B:
a: Any
b: Any = field(kw_only=True)
The name of the KW_ONLY field doesn't matter, since it's
discarded. I
think _ is a fine name, and '_: dataclasses.KW_ONLY' would be
the
pythonic way of saying "the following fields are keyword-only".
My example above would become:
@dataclasses.dataclass
class LotsOfFields:
a: Any
_: dataclasses.KW_ONLY
b: Any = 0
c: Any = 'foo'
d: Any
e: Any = 0.0
f: Any
g: Any = ()
h: Any = 'bar'
i: Any = 3+4j
j: Any = 10
k: Any
Which I think is a lot clearer.
The generated __init__ would look like:
def __init__(self, a, *, b=0, c='foo', d, e=0.0, f, g=(),
h='bar',
i=3+4j, j=10, k):
The idea is that all normal argument fields would appear
first in the
class definition, then all keyword argument fields. This is
the same
requirement as in a function definition. There would be no
switching
back and forth between the two types of fields: once you use
KW_ONLY,
all subsequent fields are keyword-only. A field of type
KW_ONLY can
appear only once in a particular dataclass (but see the
discussion below
about inheritance).
Re-ordering args in __init__
----------------------------
If, using field(kw_only=True), you specify keyword-only
fields before
non-keyword-only fields, all of the keyword-only fields will
be moved to
the end of the __init__ argument list. Within the list of
non-keyword-only arguments, all arguments will keep the same
relative
order as in the class definition. Ditto for within
keyword-only arguments.
So:
@dataclasses.dataclass
class C:
a: Any
b: Any = field(kw_only=True)
c: Any
d: Any = field(kw_only=True)
Then the generated __init__ will look like:
def __init__(self, a, c, *, b, d):
__init__ is the only place where this rearranging will take
place.
Everywhere else, and importantly in __repr__ and any dunder
comparison
methods, the order will be the same as it is now: in field
declaration
order.
Can you be specific and show what the repr() would be? E.g. if I
create C(1, 2, b=3, d=4) the repr() be C(a=1, b=3, c=2, d=4), right?
This is the same behavior that attrs uses.
Nevertheless I made several typos trying to make the examples in
my sentence above correct. Perhaps we could instead disallow
mixing kw-only and regular args? Do you know why attrs does it
this way?
Inheritance
-----------
There are a few additional quirks involving inheritance, but the
behavior would follow naturally from how dataclasses already
handles
fields via inheritance and the __init__ argument re-ordering
discussed
above. Basically, all fields in a derived class are computed
like they
are today. Then any __init__ argument re-ordering will take
place, as
discussed above.
Consider:
@dataclasses.dataclass(kw_only=True)
class D:
a: Any
@dataclasses.dataclass
class E(D):
b: Any
@dataclasses.dataclass(kw_only=True)
class F(E):
c: Any
This will result in the __init__ signature of:
def __init__(self, b, *, a, c):
However, the repr() will still produce the fields in order a,
b, c.
Comparisons will also use the same order.
This can be simulated by flattening the inheritance tree and
adding explicit field(kw_only=True) to all fields of classes
using kw_only=True in the class decorator as well as all fields
affected by _: KW_ONLY, right? So the above would behave like this:
@dataclasses.dataclass
class F:
a: Any = field(kw_only=True)
b: Any
c: Any = field(kw_only=True)
which IIUC indeed gives the same __init__ signature and repr().
Conclusion
----------
Remember, the only point of all of these hoops is to add a
flag to each
field saying what type of __init__ argument it becomes:
normal or
keyword-only. Any of the 3 methods discussed above (kw_only
flag to
@dataclass(), kw_only flag to field(), or the KW_ONLY marker)
all have
the same result: setting the kw_only flag on one or more fields.
The value of that flag, on a per-field basis, is used to
re-order
__init__ arguments, and is used in generating the __init__
signature.
It's not used anywhere else.
I expect the two most common use cases to be the kw_only flag to
@dataclass() and the KW_ONLY marker. I would expect the usage
of the
kw_only flag on field() to be rare, but since it's the
underlying
mechanism and it's needed for more complex field layouts, it
is included
in this proposal.
So, what do you think? Is this a horrible idea? Should it be
a PEP, or
just a 'simple' feature addition to dataclasses? I'm worried
that if I
have to do a full blown PEP I won't get to this for 3.10.
I don't think it is very controversial, do you? Then again maybe
you should ask a SC member if they would object.
mypy and other type checkers would need to be taught about
all of this.
Yeah, that's true. But the type checkers have bigger fish to fry
(e.g. pattern matching).
--
--Guido van Rossum (python.org/~guido <http://python.org/~guido>)
/Pronouns: he/him //(why is my pronoun here?)/
<http://feministing.com/2015/02/03/how-using-they-as-a-singular-pronoun-can-change-the-world/>
_______________________________________________
Python-ideas mailing list --python-ideas@python.org
<mailto:python-ideas@python.org>
To unsubscribe send an email topython-ideas-le...@python.org
<mailto:python-ideas-le...@python.org>
https://mail.python.org/mailman3/lists/python-ideas.python.org/
<https://mail.python.org/mailman3/lists/python-ideas.python.org/>
Message archived
athttps://mail.python.org/archives/list/python-ideas@python.org/message/AZN66ZACAH6BGX2OGDQI7GV3X6SETRUP/
<https://mail.python.org/archives/list/python-ideas@python.org/message/AZN66ZACAH6BGX2OGDQI7GV3X6SETRUP/>
Code of Conduct:http://python.org/psf/codeofconduct/
<http://python.org/psf/codeofconduct/>
