[
https://issues.apache.org/jira/browse/ARROW-2822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16538195#comment-16538195
]
Dimitri Vorona edited comment on ARROW-2822 at 7/10/18 9:07 AM:
----------------------------------------------------------------
Hi Wes,
yeah, this should work. We'd still have to make sure that Finish is called,
since else the overprovisioned portion of the buffer will still be
uninitialized, but a memset of avg 32 byte to the last cache line shouldn't be
too bad, especially since the time is dominated by a realloc anyway.
It just came up to me: we (almost) didn't have a padding problem with the
buffer builders after all, since they always resize the data_ to a power of 2,
so its a multiple of 64 for every case except rare ones like Type == UInt8 and
capacity == kMinBuilderCapacity (which is 32 right now). Which explains why we
weren't swamped in valgrind warnings.
I would still say, that it's more of a happy coincidence, and we should just
zero everything between size_ and capacity_. The AppendNull problem would still
be there, too.
Cheers
was (Author: alendit):
Hi Wes,
yeah, this should work. We'd still have to make sure that Finish is called,
since else the overprovisioned portion of the buffer will still be
uninitialized, but a memset of avg 32 byte to the last cache line shouldn't be
too bad, especially since the time is dominated by a realloc anyway.
It just came up to me: we (almost) didn't have a padding problem with the
buffer after all, since they always resize the data_ to a power of 2, so its a
multiple of 64 for every case except rare ones like Type == UInt8 and capacity
== kMinBuilderCapacity (which is 32 right now). Which explains why we weren't
swamped in valgrind warnings.
I would still say, that it's more of a happy coincidence, and we should just
zero everything between size_ and capacity_. The AppendNull problem would still
be there, too.
Cheers
> [C++] Zero padding bytes in PoolBuffer::Resize
> ----------------------------------------------
>
> Key: ARROW-2822
> URL: https://issues.apache.org/jira/browse/ARROW-2822
> Project: Apache Arrow
> Issue Type: Improvement
> Components: C++
> Reporter: Wes McKinney
> Priority: Major
>
> This is a possible proposed fix which would supersede some of the changes
> made in ARROW-2790 as elsewhere.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
