[
https://issues.apache.org/jira/browse/ARROW-10172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17208105#comment-17208105
]
Artem KOZHEVNIKOV commented on ARROW-10172:
-------------------------------------------
btw, casting to large_string is not supported neither (it's maybe linked):
{code:python}
str_array.cast(pa.large_string())
ArrowNotImplementedError Traceback (most recent call last)
<ipython-input-2-7399d0bc10ae> in <module>
----> 1 str_array.cast(pa.large_string())
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/table.pxi in
pyarrow.lib.ChunkedArray.cast()
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/compute.py in
cast(arr, target_type, safe)
85 else:
86 options = _pc.CastOptions.unsafe(target_type)
---> 87 return call_function("cast", [arr], options)
88
89
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/_compute.pyx in
pyarrow._compute.call_function()
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/_compute.pyx in
pyarrow._compute.Function.call()
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/error.pxi in
pyarrow.lib.pyarrow_internal_check_status()
/opt/conda/envs/model/lib/python3.7/site-packages/pyarrow/error.pxi in
pyarrow.lib.check_status()
ArrowNotImplementedError: Unsupported cast from string to large_utf8 using
function cast_large_string
{code}
> cancat_arrays requires upcast for large array
> ---------------------------------------------
>
> Key: ARROW-10172
> URL: https://issues.apache.org/jira/browse/ARROW-10172
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Affects Versions: 1.0.1
> Reporter: Artem KOZHEVNIKOV
> Priority: Major
>
> I'm sorry if this was already reported, but there's an overflow issue in
> concatenation of large arrays
> {code:python}
> In [1]: import pyarrow as pa
> In [2]: str_array = pa.array(['a' * 128] * 10**8)
> In [3]: large_array = pa.concat_arrays([str_array] * 50)
> Segmentation fault (core dumped)
> {code}
> I suppose that this should be handled by upcast to large_string.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
