Jonathan A Sternberg created ARROW-5212:
-------------------------------------------
Summary: Array BinaryBuilder in Go library has no access to resize
the values buffer
Key: ARROW-5212
URL: https://issues.apache.org/jira/browse/ARROW-5212
Project: Apache Arrow
Issue Type: Improvement
Reporter: Jonathan A Sternberg
When you are dealing with a binary builder, there are three buffers: the null
bitmap, the offset indexes, and the values buffer which contains the actual
data.
When {{Reserve}} or {{Resize}} are used, the null bitmap and the offsets are
modified to allow for additional appends to function. This seems correct to me.
There's no way to know how much the values buffer should be resized until the
values are being appended with just the number of values alone.
But, when you are then appending a bunch of string values, there's no
additional API to preallocate the size of that last buffer. That means that
batch appending a large amount of strings will constantly allocate even if you
know the size ahead of time.
There should be some additional API to modify this last buffer such as maybe
{{ReserveBytes}} and {{ResizeBytes}} that would correspond with the {{Reserve}}
and {{Resize}} methods, but would related to the values buffer.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
