Antony Mayi created ARROW-2153:

             Summary: decimal conversion not working for exponential notation
                 Key: ARROW-2153
             Project: Apache Arrow
          Issue Type: Bug
          Components: Python
    Affects Versions: 0.8.0
            Reporter: Antony Mayi

import pyarrow as pa
import pandas as pd
import decimal

pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'), 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas 
line 350, in dataframe_to_arrays
line 349, in <listcomp>
    for c, t in zip(columns_to_convert,
line 345, in convert_column
    return pa.array(col, from_pandas=True, type=ty)
  File "pyarrow/array.pxi", line 170, in pyarrow.lib.array 
  File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array 
  File "pyarrow/error.pxi", line 77, in pyarrow.lib.check_status 
pyarrow.lib.ArrowInvalid: Expected base ten digit or decimal point but found 
'E' instead.
In manual cases clearly we can write {{decimal.Decimal('20')}} instead of 
{{decimal.Decimal('2E+1')}} but during arithmetical operations inside an 
application the exponential notation can be produced out of control (it is 
actually the _normalized_ form of the decimal number) plus for some values the 
exponential notation is the only form expressing the significance so this 
should be accepted.

The [documentation|] suggests 
using following transformation but that's only possible when the significance 
information doesn't need to be kept:
def remove_exponent(d):
    return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize()

This message was sent by Atlassian JIRA

Reply via email to