[
https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16788150#comment-16788150
]
Maarten Breddels commented on ARROW-4810:
-----------------------------------------
> Having arrays with > 2GB elements or binary arrays with > 2GB of data would
> be considered an anti-pattern in the context of database systems, regardless
> of whether the offsets are 32- or 64-bit. So in light of these it doesn't
> make sense to have the default type be 64-bit capable if this capability is
> seldom used
I agree it's not the best idea, but people will find a reason to do it, and
since there will not be a straightforward workaround, it may spin off another
'standard' :)
But, since allow/supporting it will solve both issues (>2GB elements, and less
code complexity) I thought I would mention that as well.
As far as the implementation, are you thinking about a new class (apart from
ArrayList), or does it seem feasible to include a type for the value_offsets?
> [Format][C++] Add "LargeList" type with 64-bit offsets
> ------------------------------------------------------
>
> Key: ARROW-4810
> URL: https://issues.apache.org/jira/browse/ARROW-4810
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++, Format
> Reporter: Wes McKinney
> Priority: Major
> Fix For: 0.14.0
>
>
> Mentioned in https://github.com/apache/arrow/issues/3845
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
