[issue16535] json encoder unable to handle decimal

[Luca Lesinigo]

Change by Luca Lesinigo :


--
nosy: +luca.lesinigo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> I wonder if adding something similar to what `simplejson` uses 
> (i.e. explicitly specifying in `json.dump(s)` how to
> serialize `decimal.Decimal`) could be acceptable.

+1 for this approach.  For financial applications, we need the recommended 
solution to be simple.

--
nosy: +bob.ippolito
versions: +Python 3.10 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[mike bayer]

Change by mike bayer :


--
nosy: +zzzeek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Richard Musil]

Richard Musil  added the comment:

It looks like I am resurrecting an old item, but I have been just hit by this 
and was directed to this issue 
(https://mail.python.org/archives/list/python-id...@python.org/thread/WT6Z6YJDEZXKQ6OQLGAPB3OZ4OHCTPDU/)

I wonder if adding something similar to what `simplejson` uses (i.e. explicitly 
specifying in `json.dump(s)` how to serialize `decimal.Decimal`) could be 
acceptable.

Or, the other idea would be to expose a method in JSONEncoder, which would 
accept "raw" textual output, i.e. string (or even `bytes`) and would encode it 
without adding additional characters to it. (as explained in my posts in the 
other threads).

As it seems right now, there is no way to serialize `decimal.Decimal` the same 
way it is deserialized, i.e. while preserving the (arbitrary) precision.

--
nosy: +risa2000

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The trick from msg176158 no longer works since issue26719.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Mads Jensen]

Mads Jensen added the comment:

Hi @cvrebert and team - do you know if this was ever implemented. It seems that 
it is still an issue for financial applications, and that the solution proposed 
would be relevant and helpful.

--
nosy: +mjensen

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Stefan Krah]

Changes by Stefan Krah :


--
nosy:  -skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Chris Rebert]

Chris Rebert added the comment:

> 1) JSON just support floats

If you read the JSON standards documents, you'll see that this isn't accurate.

Regardless, a general solution for non-built-in numeric types does seem 
preferable.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Christian Heimes]

Christian Heimes added the comment:

I'm EuroPython 2014 in Berlin. Ralph has approached me and asked me about 
progress on the progress of this patch. I'm reluctant to implement a special 
case for decimals for two reasons:

1) JSON just support floats and decimals are IMHO incompatible with floats. The 
conversion of decial to JSON floats is a loosely operation.

2) Rather than having a special case I would rather go with a general 
implementation that uses an ABC to JSON dump some float-like objects.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Ralph Heinkel]

Changes by Ralph Heinkel :


--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Stefan Krah]

Stefan Krah added the comment:

I think we should really apply #19232. At least that would take care
of the import issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Antoine Pitrou]

Antoine Pitrou added the comment:

The patch isn't really ok, IMO. It forcibly imports the decimal module and then 
looks up the type there. The decimal module is a rather large one and it 
shouldn't get imported if it doesn't get used.

I think it would be better to rely on the __float__ special method, which would 
also automatically accept other numberish types such as Fraction.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Chris Rebert]

Changes by Chris Rebert :


--
nosy: +cvrebert

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Éric Araujo]

Éric Araujo added the comment:

Patch looks good and contains tests for the C and Python code.

Documentation is missing (a note to tell that json.dump converts 
decimal.Decimal instances to JSON numbers, a versionchanged directive, maybe a 
link to the doc that explains parse_float=decimal.Decimal).

--
stage: needs patch -> patch review
versions: +Python 3.5 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Éric Araujo]

Changes by Éric Araujo :


--
hgrepos:  -202

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Ralph Heinkel]

Ralph Heinkel added the comment:

This patch was implemented on Europython 2013 sprint. It's my first addition to 
Python core ever so please bear with me if it's not perfect. 

Decimal support is implemented both in the C and Python JSON code.

There is one peculiarity to mention about the Decimal addition in function 
_json.c:encoder_listencode_obj() of my patch:
The addition of

else if (PyObject_IsInstance(obj, (PyObject*)PyDecimalType)) {
PyObject *encoded = encoder_encode_decimal(s, obj);
if (encoded == NULL)
return -1;
return _steal_accumulate(acc, encoded);
}

was has to be located AFTER lists and dicts are handled in the JSON encoder, 
otherwise the unittest "test_highly_nested_objects_encoding()" from 
test_recursion.py fails with a nasty, unrecoverable Python exception.
My guess is that this is due additional stack allocation when the stack space 
is almost used up by the deeply nested recursion code.

--
hgrepos: +202
keywords: +patch
nosy: +ralhei
Added file: http://bugs.python.org/file30836/json_decimal.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Arfrever Frehtes Taifersar Arahesis]

Changes by Arfrever Frehtes Taifersar Arahesis :


--
nosy: +Arfrever

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Antoine Pitrou]

Antoine Pitrou added the comment:

Decimal numbers should simply be serializable by default. It doesn't make sense 
to add a specialized option.

--
components: +Library (Lib)
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Jesús Cea Avión]

Changes by Jesús Cea Avión :


--
nosy: +jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Éric Araujo]

Éric Araujo added the comment:

Thanks for the workaround Serhiy, I stole that and ran with it :)

For 3.4 I still think something built-in would be best.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Mark Dickinson]

Mark Dickinson added the comment:

Judging by the discussion that Éric points to, and by the various stackoverflow 
questions on the topic ([1], [2]), this is a common enough need that I think it 
would make sense to have some support for it in the std. lib.

There's a sense in which Decimal is the 'right' type for json, and we shouldn't 
make it harder for people to do the right thing with respect to (e.g.) 
financial data in databases.





[1] http://stackoverflow.com/questions/4019856/decimal-to-json
[2] 
http://stackoverflow.com/questions/1960516/python-json-serialize-a-decimal-object

--
nosy: +mark.dickinson
stage:  -> needs patch
versions: +Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The json module already has too many options. No need for yet one such 
specialized.

>>> class number_str(float):
... def __init__(self, o):
... self.o = o
... def __repr__(self):
... return str(self.o)
... 
>>> def decimal_serializer(o):
... if isinstance(o, decimal.Decimal):
... return number_str(o)
... raise TypeError(repr(o) + " is not JSON serializable")
... 
>>> print(json.dumps([decimal.Decimal('0.20001')], 
>>> default=decimal_serializer))
[0.20001]

You can extend this to support complex numbers, fractions, date and time, and 
many other custom types. Have specialized options for this would be cumbersome.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Stefan Krah]

Changes by Stefan Krah :


--
nosy: +skrah

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Éric Araujo]

Éric Araujo added the comment:

See lengthy discussion that lead to inclusion in simplejson here: 
http://code.google.com/p/simplejson/issues/detail?id=34

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16535] json encoder unable to handle decimal

[Éric Araujo]

New submission from Éric Araujo:

In 2.7 and other versions, the json module has incomplete support for decimals:

>>> json.loads('0.2', parse_float=Decimal)
Decimal('0.2')
>>> json.dumps(json.loads('0.2', parse_float=Decimal))
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python2.7/json/__init__.py", line 231, in dumps
return _default_encoder.encode(obj)
  File "/usr/lib/python2.7/json/encoder.py", line 201, in encode
chunks = self.iterencode(o, _one_shot=True)
  File "/usr/lib/python2.7/json/encoder.py", line 264, in iterencode
return _iterencode(o, 0)
  File "/usr/lib/python2.7/json/encoder.py", line 178, in default
raise TypeError(repr(o) + " is not JSON serializable")
TypeError: Decimal('0.2') is not JSON serializable

simplejson encodes decimals out of the box, but json can’t round-trip.

--
messages: 176135
nosy: eric.araujo, ezio.melotti, pitrou, rhettinger
priority: normal
severity: normal
status: open
title: json encoder unable to handle decimal
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com