[jira] [Updated] (ARROW-2153) [C++/Python] Decimal conversion not working for exponential notation
[ https://issues.apache.org/jira/browse/ARROW-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wes McKinney updated ARROW-2153: Fix Version/s: 0.9.0 > [C++/Python] Decimal conversion not working for exponential notation > > > Key: ARROW-2153 > URL: https://issues.apache.org/jira/browse/ARROW-2153 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Affects Versions: 0.8.0 >Reporter: Antony Mayi >Assignee: Phillip Cloud >Priority: Major > Labels: pull-request-available > Fix For: 0.9.0 > > > {code:java} > import pyarrow as pa > import pandas as pd > import decimal > pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'), > decimal.Decimal('2E+1')]})) > {code} > > {code:java} > Traceback (most recent call last): > File "", line 1, in > File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas > (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927) > File > "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py", > line 350, in dataframe_to_arrays > convert_types)] > File > "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py", > line 349, in > for c, t in zip(columns_to_convert, > File > "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py", > line 345, in convert_column > return pa.array(col, from_pandas=True, type=ty) > File "pyarrow/array.pxi", line 170, in pyarrow.lib.array > (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224) > File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array > (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465) > File "pyarrow/error.pxi", line 77, in pyarrow.lib.check_status > (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:8270) > pyarrow.lib.ArrowInvalid: Expected base ten digit or decimal point but found > 'E' instead. > {code} > In manual cases clearly we can write {{decimal.Decimal('20')}} instead of > {{decimal.Decimal('2E+1')}} but during arithmetical operations inside an > application the exponential notation can be produced out of control (it is > actually the _normalized_ form of the decimal number) plus for some values > the exponential notation is the only form expressing the significance so this > should be accepted. > The [documentation|https://docs.python.org/3/library/decimal.html] suggests > using following transformation but that's only possible when the significance > information doesn't need to be kept: > {code:java} > def remove_exponent(d): > return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize() > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (ARROW-2153) [C++/Python] Decimal conversion not working for exponential notation
[ https://issues.apache.org/jira/browse/ARROW-2153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Phillip Cloud updated ARROW-2153: - Summary: [C++/Python] Decimal conversion not working for exponential notation (was: [C++] Decimal conversion not working for exponential notation) > [C++/Python] Decimal conversion not working for exponential notation > > > Key: ARROW-2153 > URL: https://issues.apache.org/jira/browse/ARROW-2153 > Project: Apache Arrow > Issue Type: Bug > Components: Python >Affects Versions: 0.8.0 >Reporter: Antony Mayi >Assignee: Phillip Cloud >Priority: Major > Labels: pull-request-available > > {code:java} > import pyarrow as pa > import pandas as pd > import decimal > pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'), > decimal.Decimal('2E+1')]})) > {code} > > {code:java} > Traceback (most recent call last): > File "", line 1, in > File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas > (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927) > File > "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py", > line 350, in dataframe_to_arrays > convert_types)] > File > "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py", > line 349, in > for c, t in zip(columns_to_convert, > File > "/home/skadlec/.local/lib/python3.6/site-packages/pyarrow/pandas_compat.py", > line 345, in convert_column > return pa.array(col, from_pandas=True, type=ty) > File "pyarrow/array.pxi", line 170, in pyarrow.lib.array > (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224) > File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array > (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465) > File "pyarrow/error.pxi", line 77, in pyarrow.lib.check_status > (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:8270) > pyarrow.lib.ArrowInvalid: Expected base ten digit or decimal point but found > 'E' instead. > {code} > In manual cases clearly we can write {{decimal.Decimal('20')}} instead of > {{decimal.Decimal('2E+1')}} but during arithmetical operations inside an > application the exponential notation can be produced out of control (it is > actually the _normalized_ form of the decimal number) plus for some values > the exponential notation is the only form expressing the significance so this > should be accepted. > The [documentation|https://docs.python.org/3/library/decimal.html] suggests > using following transformation but that's only possible when the significance > information doesn't need to be kept: > {code:java} > def remove_exponent(d): > return d.quantize(Decimal(1)) if d == d.to_integral() else d.normalize() > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)