[jira] [Commented] (ARROW-2227) [Python] Table.from_pandas does not create chunked_arrays.

[Rok Mihevc (Jira)]

    [ 
https://issues.apache.org/jira/browse/ARROW-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17659255#comment-17659255
 ] 

Rok Mihevc commented on ARROW-2227:
-----------------------------------

This issue has been migrated to [issue 
#18190|https://github.com/apache/arrow/issues/18190] on GitHub. Please see the 
[migration documentation|https://github.com/apache/arrow/issues/14542] for 
further details.

> [Python] Table.from_pandas does not create chunked_arrays.
> ----------------------------------------------------------
>
>                 Key: ARROW-2227
>                 URL: https://issues.apache.org/jira/browse/ARROW-2227
>             Project: Apache Arrow
>          Issue Type: Bug
>          Components: Python
>    Affects Versions: 0.8.0
>            Reporter: Left Screen
>            Assignee: Wes McKinney
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 0.9.0
>
>
> When creating a large enough array, pyarrow raises an exception:
> {code:java}
> import numpy as np
> import pandas as pd
> import pyarrow as pa
> x = list('1' * 2**31)
> y = pd.DataFrame({'x': x})
> t = pa.Table.from_pandas(y)
> # ArrowInvalid: BinaryArrow cannot contain more than 2147483646 bytes, have 
> 2147483647{code}
> The array should be chunked for the user. As is, data frames with >2 GiB in 
> binary data will struggle to get into arrow.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)