tustvold opened a new pull request, #1722: URL: https://github.com/apache/arrow-rs/pull/1722
# Which issue does this PR close? Closes #1708. # Rationale for this change `unpack32` as used by `BitReader::get_batch` always decodes to an array of 32, 32-bit integers. If the type being read is a different size it will read to a temporary buffer and then either truncate or zero-extend the data. An oversight meant that instead of zero-extending, it would just set the first 4 bytes, and leave any remaining bytes alone. In practice this didn't matter as we were only using this for decoding 16-bit levels, however, with #1284 the DeltaBitPackDecoder was changed to also use this method. As this does get used with 64-bit types, buffer reuse could easily lead to garbage data on decode. # What changes are included in this PR? Fixes `BitReader::get_batch` to correctly zero-extend, and reduces the use of unsafe in the process. # Are there any user-facing changes? No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
