zeroshade commented on code in PR #14126:
URL: https://github.com/apache/arrow/pull/14126#discussion_r1087062331
##########
go/arrow/array/concat.go:
##########
@@ -517,6 +523,50 @@ func concat(data []arrow.ArrayData, mem memory.Allocator)
(arrow.ArrayData, erro
if err != nil {
return nil, err
}
+ case *arrow.RunEndEncodedType:
+ physicalLength, overflow := int(0), false
+ // we can't use gatherChildren because the Offset and Len of
+ // data doesn't correspond to the physical length or offset
+ runs := make([]arrow.ArrayData, len(data))
+ values := make([]arrow.ArrayData, len(data))
+ for i, d := range data {
+ plen := encoded.GetPhysicalLength(d)
+ off := encoded.FindPhysicalOffset(d)
+
+ runs[i] = NewSliceData(d.Children()[0], int64(off),
int64(off+plen))
+ defer runs[i].Release()
+ values[i] = NewSliceData(d.Children()[1], int64(off),
int64(off+plen))
+ defer values[i].Release()
+
+ physicalLength, overflow = addOvf(physicalLength,
int(plen))
+ if overflow {
+ return nil, fmt.Errorf("%w: run length encoded
array length must fit into a 32-bit signed integer",
Review Comment:
so, this is more of a short circuit. We have to add up all of the physical
lengths so we know the size to allocate for the full concatenated array. If we
end up overflowing when adding up the physical length we don't have to bother
calculating the new run ends, we know we're going to overflow no matter what
the offset type is. So this doesn't need to depend on the offset type.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
