[
https://issues.apache.org/jira/browse/ARROW-2059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352024#comment-16352024
]
Jingyuan Wang commented on ARROW-2059:
--------------------------------------
Stack trace of the failure when processing 100M-row data (feather-0.4.0):
{code:none}
Traceback (most recent call last):
File "test.py", line 44, in <module>
elapsed_ts.append(test(df, i))
File "test.py", line 18, in test
feather.write_dataframe(df, fthfilename)
File
"/home/jingyuan/anaconda3/envs/arrow-2059-feather-0.4.0/lib/python3.6/site-packages/pyarrow/feather.py",
line 100, in write_feather
writer.write(df)
File
"/home/jingyuan/anaconda3/envs/arrow-2059-feather-0.4.0/lib/python3.6/site-packages/pyarrow/feather.py",
line 80, in write
batch = RecordBatch.from_pandas(df, preserve_index=False)
File "table.pxi", line 705, in pyarrow.lib.RecordBatch.from_pandas
File "table.pxi", line 739, in pyarrow.lib.RecordBatch.from_arrays
TypeError: Cannot convert pyarrow.lib.ChunkedArray to pyarrow.lib.Array
{code}
> [Python] Possible performance regression in Feather read/write path
> -------------------------------------------------------------------
>
> Key: ARROW-2059
> URL: https://issues.apache.org/jira/browse/ARROW-2059
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Wes McKinney
> Assignee: Jingyuan Wang
> Priority: Major
> Fix For: 0.9.0
>
>
> See discussion in https://github.com/wesm/feather/issues/329. Needs to be
> investigated
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
