[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-14 Thread Micah Kornfield (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16792367#comment-16792367 ] Micah Kornfield commented on ARROW-4810: I see Message.fbs got updated 2-years ago to use 64-bit

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-13 Thread Micah Kornfield (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16791521#comment-16791521 ] Micah Kornfield commented on ARROW-4810: How does this reconcile with other 32 bit length

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-10 Thread Philipp Moritz (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789169#comment-16789169 ] Philipp Moritz commented on ARROW-4810: --- I agree that we should have a reference implementation in

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-10 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16789043#comment-16789043 ] Wes McKinney commented on ARROW-4810: - Yes, I agree we should create an implementation for Java as

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-10 Thread Jacques Nadeau (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788994#comment-16788994 ] Jacques Nadeau commented on ARROW-4810: --- I'm -1 on this (and any format change) unless there are

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-08 Thread Francois Saint-Jacques (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788448#comment-16788448 ] Francois Saint-Jacques commented on ARROW-4810: --- I'm not a afan of multiplying types, have

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-08 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788240#comment-16788240 ] Wes McKinney commented on ARROW-4810: - Yes -- there's a fair bit of boilerplate to consider. If we

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-08 Thread Maarten Breddels (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788160#comment-16788160 ] Maarten Breddels commented on ARROW-4810: - I see BinaryArray/StringArray classes have similar

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-08 Thread Wes McKinney (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788155#comment-16788155 ] Wes McKinney commented on ARROW-4810: - Yes, we'd want a separate container type, e.g.

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-08 Thread Maarten Breddels (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788150#comment-16788150 ] Maarten Breddels commented on ARROW-4810: - > Having arrays with > 2GB elements or binary arrays

[jira] [Commented] (ARROW-4810) [Format][C++] Add "LargeList" type with 64-bit offsets

2019-03-08 Thread Benjamin Kietzman (JIRA)
[ https://issues.apache.org/jira/browse/ARROW-4810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16788025#comment-16788025 ] Benjamin Kietzman commented on ARROW-4810: -- This would probably also be useful for Binary and