[ https://issues.apache.org/jira/browse/ARROW-2205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Albert Shieh updated ARROW-2205: -------------------------------- Affects Version/s: 0.8.0 > [Python] Option for integer object nulls > ---------------------------------------- > > Key: ARROW-2205 > URL: https://issues.apache.org/jira/browse/ARROW-2205 > Project: Apache Arrow > Issue Type: New Feature > Components: C++, Python > Affects Versions: 0.8.0 > Reporter: Albert Shieh > Priority: Major > Labels: pull-request-available > > I have a use case where the loss of precision in casting integers to floats > matters, and pandas supports storing integers with nulls without loss of > precision in object columns. However, a roundtrip through arrow will cast the > object columns to float columns, even though the object columns are stored in > arrow as integers with nulls. > This is a minimal example demonstrating the behavior of a roundtrip: > {code} > import numpy as np > import pandas as pd > import pyarrow as pa > df = pd.DataFrame({"a": np.array([None, 1], dtype=object)}) > df_pa = pa.Table.from_pandas(df).to_pandas() > print(df) > print(df_pa) > {code} > The output is: > {code} > a > 0 None > 1 1 > a > 0 NaN > 1 1.0 > {code} > This seems to be the desired behavior, given test_int_object_nulls in > test_convert_pandas. > I think it would be useful to add an option in the to_pandas methods to allow > integers with nulls to be returned as object columns. The option can default > to false in order to preserve the current behavior. -- This message was sent by Atlassian JIRA (v7.6.3#76005)