Jingyuan Wang commented on ARROW-2059:

Stack trace of the failure when processing 100M-row data (feather-0.4.0):

Traceback (most recent call last):
  File "test.py", line 44, in <module>
    elapsed_ts.append(test(df, i))
  File "test.py", line 18, in test
    feather.write_dataframe(df, fthfilename)
 line 100, in write_feather
 line 80, in write
    batch = RecordBatch.from_pandas(df, preserve_index=False)
  File "table.pxi", line 705, in pyarrow.lib.RecordBatch.from_pandas
  File "table.pxi", line 739, in pyarrow.lib.RecordBatch.from_arrays
TypeError: Cannot convert pyarrow.lib.ChunkedArray to pyarrow.lib.Array


> [Python] Possible performance regression in Feather read/write path
> -------------------------------------------------------------------
>                 Key: ARROW-2059
>                 URL: https://issues.apache.org/jira/browse/ARROW-2059
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>            Reporter: Wes McKinney
>            Assignee: Jingyuan Wang
>            Priority: Major
>             Fix For: 0.9.0
> See discussion in https://github.com/wesm/feather/issues/329. Needs to be 
> investigated

This message was sent by Atlassian JIRA

Reply via email to