[
https://issues.apache.org/jira/browse/ARROW-2145?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16363574#comment-16363574
]
Antony Mayi commented on ARROW-2145:
------------------------------------
I am trying to use it for a real system where an event with zero value
({{decimal.Decimal('0')}}) is distinct from no event
({{decimal.Decimal('nan')}}) while both cases need to be stored. Being able to
store decimal {{nan}} within the same column spares me from introducing another
column just to flag the no-event cases.
This is the same case like with floats - {{nan}}/{{inf}} is valid float value
(and supported by pyarrow/parquet) - but here I need to use decimal because of
the precision...
> decimal conversion not working for NaN values
> ---------------------------------------------
>
> Key: ARROW-2145
> URL: https://issues.apache.org/jira/browse/ARROW-2145
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
> Affects Versions: 0.8.0
> Reporter: Antony Mayi
> Assignee: Phillip Cloud
> Priority: Major
>
> {code:python}
> import pyarrow as pa
> import pandas as pd
> import decimal
> pa.Table.from_pandas(pd.DataFrame({'a': [decimal.Decimal('1.1'),
> decimal.Decimal('NaN')]}))
> {code}
> throws following exception:
> {code}
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "pyarrow/table.pxi", line 875, in pyarrow.lib.Table.from_pandas
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927)
> File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 350, in
> dataframe_to_arrays
> convert_types)]
> File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 349, in
> <listcomp>
> for c, t in zip(columns_to_convert,
> File "/lib/python3.6/site-packages/pyarrow/pandas_compat.py", line 345, in
> convert_column
> return pa.array(col, from_pandas=True, type=ty)
> File "pyarrow/array.pxi", line 170, in pyarrow.lib.array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224)
> File "pyarrow/array.pxi", line 70, in pyarrow.lib._ndarray_to_array
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465)
> File "pyarrow/error.pxi", line 98, in pyarrow.lib.check_status
> (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:9068)
> pyarrow.lib.ArrowException: Unknown error: an integer is required (got type
> str)
> {code}
> Same problem with other special decimal values like {{infinity}}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
