[issue35540] dataclasses.asdict breaks with defaultdict fields

[Tiger]

Change by Tiger :


--
nosy: +kwsp
nosy_count: 7.0 -> 8.0
pull_requests: +30148
pull_request: https://github.com/python/cpython/pull/32056

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Raymond Xu]

Raymond Xu  added the comment:

I am seeing this bug with 3.9.7

--
nosy: +greenfish6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Alexandru Coca]

Alexandru Coca  added the comment:

I was wondering if this issue is still being tracked for resolution? I found 
the same bug in Python 3.8.

--
nosy: +alexcoca

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Paul Ganssle]

Paul Ganssle  added the comment:

I checked and it appears that `attrs` handles this by creating *all* dicts 
using the default dict_factory (similar to my original suggestion of just using 
`dict` instead of the specific type), if I'm reading this right: 
https://github.com/python-attrs/attrs/blob/master/src/attr/_funcs.py#L102

Using `attr.asdict` seems to bear this out, as `defaultdict` attributes are 
converted to `dict` when the dict factory is not specified.

I think changing the default behavior like that would be a 
backwards-incompatible change at this point (and one that it's really hard to 
warn about, unfortunately), but we could still use the "fall back to 
dict_factory" behavior by trying to construct a `type(obj)(...)` and in the 
case of an exception return `dict_factory(...)`.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Paul Ganssle]

Paul Ganssle  added the comment:

Considering that `namedtuple` is special-cased, I think it's reasonable to 
special-case `defaultdict` as well, though it may be worth considering more 
general solutions that will also work for things other than the standard 
library. One would be to solve this the same way that other "subclasses may 
have a different constructor" problems are solved (e.g. `float`, `int`, 
formerly `datetime`) and ignore the subclass (or selectively ignore it if it's 
a problem), for example changing _asdict_inner to something like this:

if isinstance(obj, dict):
new_keys = tuple((_asdict_inner(k, dict_factory),
  _asdict_inner(v, dict_factory))
  for k, v in obj.items())

try:
return type(obj)(new_keys)
except Exception:
return dict(new_keys)

Another more general alternative would be to add a type registry for `asdict`, 
either as an additional parameter or with a new transformer class of some sort. 
I created a quick proof of concept for this in GH-16356 to see one way it could 
look.

In any case I think it's quite unfortunate that we can't easily just support 
anything that has a __deepcopy__ defined. There may be some crazy solution that 
involves passing a class with a custom __getitem__ to the `memo` argument of 
copy.deepcopy, but if it's even possible (haven't thought about it enough) I'm 
not sure it's *advisable*.

--
nosy: +p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Paul Ganssle]

Change by Paul Ganssle :


--
pull_requests: +15935
pull_request: https://github.com/python/cpython/pull/16356

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

Hi @wrmsr, this happens because the constructor for `collections.defaultdict` 
differs from the one of `dict`.

I think the argument that collections.defaultdict is in the stdlib and should 
be supported is right, the changes in PR #11361 should do what you want.

--
nosy: +remi.lapeyre

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch, patch
pull_requests: +10682, 10683
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch
pull_requests: +10682
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch, patch, patch
pull_requests: +10682, 10683, 10684
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
nosy: +levkivskyi

__
Python tracker 

__
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Eric V. Smith]

Change by Eric V. Smith :


--
assignee:  -> eric.smith
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35540] dataclasses.asdict breaks with defaultdict fields

[Will T]

New submission from Will T :

_asdict_inner attempts to manually recursively deepcopy dicts by calling 
type(obj) with a generator of transformed keyvalue tuples @ 
https://github.com/python/cpython/blob/b2f642ccd2f65d2f3bf77bbaa103dd2bc2733734/Lib/dataclasses.py#L1080
 . defaultdicts are dicts so this runs but unlike other dicts their first arg 
has to be a callable or None:

import collections
import dataclasses as dc

@dc.dataclass()
class C:
d: dict

c = C(collections.defaultdict(lambda: 3, {}))
d = dc.asdict(c)

assert isinstance(d['d'], collections.defaultdict)
assert d['d']['a'] == 3

=>

Traceback (most recent call last):
  File "boom.py", line 9, in 
d = dc.asdict(c)
  File 
"/Users/spinlock/.pyenv/versions/3.7.1/lib/python3.7/dataclasses.py", line 
1019, in asdict
return _asdict_inner(obj, dict_factory)
  File 
"/Users/spinlock/.pyenv/versions/3.7.1/lib/python3.7/dataclasses.py", line 
1026, in _asdict_inner
value = _asdict_inner(getattr(obj, f.name), dict_factory)
  File 
"/Users/spinlock/.pyenv/versions/3.7.1/lib/python3.7/dataclasses.py", line 
1058, in _asdict_inner
for k, v in obj.items())
TypeError: first argument must be callable or None

I understand that it isn't this bit of code's job to support every dict (and 
list etc.) subclass under the sun but given defaultdict is stdlib it's imo 
worth supporting explicitly.

--
components: Library (Lib)
messages: 332166
nosy: wrmsr
priority: normal
severity: normal
status: open
title: dataclasses.asdict breaks with defaultdict fields
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com