[jira] [Updated] (ARROW-2227) [Python] Table.from_pandas does not create chunked_arrays.
[
https://issues.apache.org/jira/browse/ARROW-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated ARROW-2227:
--
Labels: pull-request-available (was: )
> [Python] Table.from_pandas does not create chunked_arrays.
> --
>
> Key: ARROW-2227
> URL: https://issues.apache.org/jira/browse/ARROW-2227
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Chris Ellison
>Assignee: Wes McKinney
>Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> When creating a large enough array, pyarrow raises an exception:
> {code:java}
> import numpy as np
> import pandas as pd
> import pyarrow as pa
> x = list('1' * 2**31)
> y = pd.DataFrame({'x': x})
> t = pa.Table.from_pandas(y)
> # ArrowInvalid: BinaryArrow cannot contain more than 2147483646 bytes, have
> 2147483647{code}
> The array should be chunked for the user. As is, data frames with >2 GiB in
> binary data will struggle to get into arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-2227) [Python] Table.from_pandas does not create chunked_arrays.
[
https://issues.apache.org/jira/browse/ARROW-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-2227:
Priority: Blocker (was: Major)
> [Python] Table.from_pandas does not create chunked_arrays.
> --
>
> Key: ARROW-2227
> URL: https://issues.apache.org/jira/browse/ARROW-2227
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Chris Ellison
>Assignee: Wes McKinney
>Priority: Blocker
> Fix For: 0.9.0
>
>
> When creating a large enough array, pyarrow raises an exception:
> {code:java}
> import numpy as np
> import pandas as pd
> import pyarrow as pa
> x = list('1' * 2**31)
> y = pd.DataFrame({'x': x})
> t = pa.Table.from_pandas(y)
> # ArrowInvalid: BinaryArrow cannot contain more than 2147483646 bytes, have
> 2147483647{code}
> The array should be chunked for the user. As is, data frames with >2 GiB in
> binary data will struggle to get into arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-2227) [Python] Table.from_pandas does not create chunked_arrays.
[
https://issues.apache.org/jira/browse/ARROW-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-2227:
Fix Version/s: (was: 0.10.0)
0.9.0
> [Python] Table.from_pandas does not create chunked_arrays.
> --
>
> Key: ARROW-2227
> URL: https://issues.apache.org/jira/browse/ARROW-2227
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Chris Ellison
>Assignee: Wes McKinney
>Priority: Major
> Fix For: 0.9.0
>
>
> When creating a large enough array, pyarrow raises an exception:
> {code:java}
> import numpy as np
> import pandas as pd
> import pyarrow as pa
> x = list('1' * 2**31)
> y = pd.DataFrame({'x': x})
> t = pa.Table.from_pandas(y)
> # ArrowInvalid: BinaryArrow cannot contain more than 2147483646 bytes, have
> 2147483647{code}
> The array should be chunked for the user. As is, data frames with >2 GiB in
> binary data will struggle to get into arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-2227) [Python] Table.from_pandas does not create chunked_arrays.
[
https://issues.apache.org/jira/browse/ARROW-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-2227:
Fix Version/s: (was: 0.9.0)
0.10.0
> [Python] Table.from_pandas does not create chunked_arrays.
> --
>
> Key: ARROW-2227
> URL: https://issues.apache.org/jira/browse/ARROW-2227
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Chris Ellison
>Assignee: Wes McKinney
>Priority: Major
> Fix For: 0.10.0
>
>
> When creating a large enough array, pyarrow raises an exception:
> {code:java}
> import numpy as np
> import pandas as pd
> import pyarrow as pa
> x = list('1' * 2**31)
> y = pd.DataFrame({'x': x})
> t = pa.Table.from_pandas(y)
> # ArrowInvalid: BinaryArrow cannot contain more than 2147483646 bytes, have
> 2147483647{code}
> The array should be chunked for the user. As is, data frames with >2 GiB in
> binary data will struggle to get into arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-2227) [Python] Table.from_pandas does not create chunked_arrays.
[
https://issues.apache.org/jira/browse/ARROW-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wes McKinney updated ARROW-2227:
Fix Version/s: 0.9.0
> [Python] Table.from_pandas does not create chunked_arrays.
> --
>
> Key: ARROW-2227
> URL: https://issues.apache.org/jira/browse/ARROW-2227
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Chris Ellison
>Priority: Major
> Fix For: 0.9.0
>
>
> When creating a large enough array, pyarrow raises an exception:
> {code:java}
> import numpy as np
> import pandas as pd
> import pyarrow as pa
> x = list('1' * 2**31)
> y = pd.DataFrame({'x': x})
> t = pa.Table.from_pandas(y)
> # ArrowInvalid: BinaryArrow cannot contain more than 2147483646 bytes, have
> 2147483647{code}
> The array should be chunked for the user. As is, data frames with >2 GiB in
> binary data will struggle to get into arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (ARROW-2227) [Python] Table.from_pandas does not create chunked_arrays.
[
https://issues.apache.org/jira/browse/ARROW-2227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris Ellison updated ARROW-2227:
-
Summary: [Python] Table.from_pandas does not create chunked_arrays. (was:
Table.from_pandas does not create chunked_arrays.)
> [Python] Table.from_pandas does not create chunked_arrays.
> --
>
> Key: ARROW-2227
> URL: https://issues.apache.org/jira/browse/ARROW-2227
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
>Affects Versions: 0.8.0
>Reporter: Chris Ellison
>Priority: Major
>
> When creating a large enough array, pyarrow raises an exception:
> {code:java}
> import numpy as np
> import pandas as pd
> import pyarrow as pa
> x = list('1' * 2**31)
> y = pd.DataFrame({'x': x})
> t = pa.Table.from_pandas(y)
> # ArrowInvalid: BinaryArrow cannot contain more than 2147483646 bytes, have
> 2147483647{code}
> The array should be chunked for the user. As is, data frames with >2 GiB in
> binary data will struggle to get into arrow.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
