[
https://issues.apache.org/jira/browse/ARROW-5139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833656#comment-16833656
]
Joris Van den Bossche commented on ARROW-5139:
----------------------------------------------
A "quick and dirty" fix would be to special case tables with zero rows and zero
columns in the check in {{_reconstruct_index}}
([https://github.com/apache/arrow/blob/982f341bc81e1e22d4b25f8cf00ef882a34766b6/python/pyarrow/pandas_compat.py#L695-L697]),
when recreating the pandas DataFrame from an empty arrow Table.
But I think the proper fix to this would be to allow the Table to have a
non-zero {{num_rows}}. However, this would need separate fixes depending on
where the Table is created (so eg a separate fix in the
{{Table.from_pandas(columns=[])}} and the parquet read code, for both examples
above), and complicate the code in several places, so I am not sure we want to
go down that route. (cc [~wesmckinn] ?)
> [Python/C++] Empty column selection no longer restores index
> ------------------------------------------------------------
>
> Key: ARROW-5139
> URL: https://issues.apache.org/jira/browse/ARROW-5139
> Project: Apache Arrow
> Issue Type: Bug
> Components: C++, Python
> Affects Versions: 0.12.1
> Reporter: Florian Jetter
> Priority: Minor
> Labels: parquet
>
> The index of a dataframe is no longer reconstructed when using empty column
> selection. This is a regression to 0.12.1 and probably only happens for
> pd.RangeIndex
> {code:python}
> import pandas as pd
> import pyarrow as pa
> import pyarrow.parquet as pq
> from kartothek.serialization import ParquetSerializer
> from storefact import get_store_from_url
> print(pa.__version__)
> df = pd.DataFrame(
> {"a": [1, 2]}
> )
> print(df.index)
> table = pa.Table.from_pandas(df)
> buf = pa.BufferOutputStream()
> pq.write_table(table, buf)
> reader = pa.BufferReader(buf.getvalue().to_pybytes())
> table_restored = pq.read_pandas(reader, columns=[])
> df_restored = table_restored.to_pandas()
> print(len(df_restored))
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
