[issue20774] collections.deque should ship with a stdlib json serializer

[Ken Jin]

Ken Jin  added the comment:

Sorry to butt into this conversation, but I wanted to add that I have interest 
in this feature - deques are the fourth most common container types I use in 
Python. Is there any way I can help to get this PR across the finish line?

So far I've forked the PR, rebased it, then applied some changes (docs, news, 
and performance) to try to lessen the impact of checking for deque:

(Python/master branch)
>>> timeit.timeit(stmt="json.dumps(['test'])", setup="import json", 
>>> number=1_000_000)
2.25838627

>>> timeit.timeit(stmt="json.dumps(1)", setup="import json", 
>>> number=1_000_000)
1.984512199975

(Python/pr_830 branch)
>>> timeit.timeit(stmt="json.dumps(['test'])", setup="import json", 
>>> number=1_000_000)
2.32430339991

>>> timeit.timeit(stmt="json.dumps(1)", setup="import json", 
>>> number=1_000_000)
1.968071199971

The PR branch is here https://github.com/Fidget-Spinner/cpython/tree/pr_830.

I'm not a Git wizard, so I don't know what's the best next step. Do I 
a. Make a PR against Lisa's PR (or)
b. Make a brand new PR against cpython master ?

If the core devs here feel that after 6 years, this change might be unneeded 
after all, I don't mind closing the branch either. Thanks for reading.

--
nosy: +kj

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Lisa Roach]

Lisa Roach  added the comment:

Serhiy might be right, it looks significantly worse with benchmarking:

lisroach$ python3 -m timeit "import json; json.dumps(['test'])"
10 loops, best of 3: 2.73 usec per loop

lisroach$ ./python.exe -m timeit "import json; json.dumps(['test'])"
1 loops, best of 5: 21.2 usec per loop

lisroach$ python3 -m timeit "import json; json.dumps(1)"
10 loops, best of 3: 2.49 usec per loop

lisroach$ ./python.exe -m timeit "import json; json.dumps(1)"
2 loops, best of 5: 16.3 usec per loop

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Seems there are reference leaks. And I afraid that importing a module for every 
serialized object can significantly hit the performance. Can you run some 
benchmarks?

> An __json__ attribute would have to convert to a list first.  Adding support 
> directly to the json module would allow the deque to be read directly.

With PR 830 the deque is converted to a list by json encoder.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Raymond Hettinger]

Raymond Hettinger added the comment:

Thanks Lisa.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Lisa Roach]

Lisa Roach added the comment:

I made PR 830 for this issue, it seems to be a nice feature to have in my 
opinion. 

Let me know if I should add some unit tests :)

--
nosy: +lisroach
pull_requests: +734

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Raymond Hettinger]

Raymond Hettinger added the comment:

FWIW, one of the design goal for deques was to make them easily substitutable 
for lists when needed.  This feature request is a nice-to-have that moves us a 
little closer.

That said, I think a __json__ attribute is too big of a hammer for this simple 
proposal.

Also, please add Bob Ippolito to all JSON issues.  He has excellent design 
sensibilities and considerable contact with users of the json module.

--
nosy: +bob.ippolito

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Raymond Hettinger]

Raymond Hettinger added the comment:

There is a difference.  An __json__ attribute would have to convert to a list 
first.  Adding support directly to the json module would allow the deque to be 
read directly.

I think you all are leaning towards premature generalization and making this 
harder than it needs to be.  Chris and Tarek's proposal is a reasonable and 
straight-forward, but it is not being pushed towards PEP territory and I think 
Guido would need to opine on whether to enshrine yet another dunder method that 
would infest the library and privilege the json serialization format over all 
formats.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[R. David Murray]

R. David Murray added the comment:

I disagree, I think a __json__ protocol is sensible.  But this is why it needs 
to be discussed on python-dev or python-ideas first :)  In  the meantime adding 
deque support like we added enum support is reasonable, but IMO we shouldn't go 
to crazy adding support for non-base types before talking about a __json__ 
protocol.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Raymond Hettinger]

Raymond Hettinger added the comment:

For now, just hardcoding deque support is fine. 

Support for a __json__ attribute or JSON array registry is a topic for another 
day.  Even then, I don't think that within the standard library support for 
JSONification should have its responsibility shifted outside the of json module 
itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See issue27362 for more general approach.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Raymond Hettinger]

Raymond Hettinger added the comment:

See also, the same feature request from Tarek Ziadé, 
http://bugs.python.org/issue29663 , "collections.deque could be serialized in 
JSON as a simple array. The only thing we can lose in the process is the maxlen 
value, but I think it's a decent behaviour to ignore it when encoding and to 
set it to None when decoding."

+1 from me as well.  This isn't really different that how we handle tuples and 
I can see that it would be useful to be able to dump a deque into JSON.  I 
concur that it is reasonable to ignore maxlen because that is primarily a 
convenience feature (auto-popping on overflow) rather than something that is 
intrinsic to the semantics of data itself.

For now, just adding deque support is reasonable.  We can't just do all 
sequences because string/bytearray like objects would need to be excluded.

--
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Antoine Pitrou]

Antoine Pitrou added the comment:

The problem is that it would be deserialized as a list; this breaks the general 
expectation that serialization formats should round-trip.

(yes, tuple already does this; but I think it is less of a problem for tuples, 
since the list API is a superset of the tuple API except for hashing)

So, perhaps we could ship an optional serializer (under which form?) accepting 
any sequence type (and perhaps any mapping type?), but it shouldn't be the 
default.

--
nosy: +ezio.melotti, pitrou, rhettinger
type:  - enhancement
versions: +Python 3.5 -Python 2.7, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20774
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Gareth Rees]

Gareth Rees added the comment:

The JSON implementation uses these tests to determine how to serialize a Python 
object:

isinstance(o, (list, tuple))
isinstance(o, dict)

So any subclasses of list and tuple are serialized as a list, and any subclass 
of dict is serialized as an object. For example:

 json.dumps(collections.defaultdict())
'{}'
 json.dumps(collections.OrderedDict())
'{}'
 json.dumps(collections.namedtuple('mytuple', ())())
'[]'

When deserialized, you'll get back a plain dictionary or list, so there's no 
round-trip property here.

The tests could perhaps be changed to:

isinstance(o, collections.abc.Sequence)
isinstance(o, collections.abc.Mapping)

I'm not a JSON expert, so I have no informed opinion on whether this is a good 
idea or not, but in any case, this change wouldn't help with deques, as a deque 
is not a Sequence. That's because deques don't have an index method (see 
issue10059 and issue12543).

--
nosy: +Gareth.Rees

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20774
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[Chris Adams]

New submission from Chris Adams:

Currently the stdlib json module requires a custom serializer to avoid throwing 
a TypeError on collections.deque instances:

Python 3.3.4 (default, Feb 12 2014, 09:35:54) 
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type help, copyright, credits or license for more information.
 from collections import deque
 import json
 d = deque(range(0, 10))
 json.dumps(d)
Traceback (most recent call last):
  File stdin, line 1, in module
  File 
/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/__init__.py,
 line 233, in dumps
return _default_encoder.encode(obj)
  File 
/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py,
 line 191, in encode
chunks = self.iterencode(o, _one_shot=True)
  File 
/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py,
 line 249, in iterencode
return _iterencode(o, 0)
  File 
/usr/local/Cellar/python3/3.3.4/Frameworks/Python.framework/Versions/3.3/lib/python3.3/json/encoder.py,
 line 173, in default
raise TypeError(repr(o) +  is not JSON serializable)
TypeError: deque([0, 1, 2, 3, 4, 5, 6, 7, 8, 9]) is not JSON serializable

--
messages: 212215
nosy: acdha
priority: normal
severity: normal
status: open
title: collections.deque should ship with a stdlib json serializer
versions: Python 2.7, Python 3.3

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20774
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20774] collections.deque should ship with a stdlib json serializer

[R. David Murray]

R. David Murray added the comment:

json is only designed to serialize standard data types out of the box.  
Anything else is an extension.  I presume you are asking for this because a 
deque looks more-or-less like a list.  I'm not sure that's reason enough, but 
we'll see what others think.

--
nosy: +r.david.murray

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue20774
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com