alamb commented on code in PR #8527:
URL: https://github.com/apache/arrow-rs/pull/8527#discussion_r2395850528
##########
parquet/src/column/writer/mod.rs:
##########
@@ -1073,6 +1073,7 @@ impl<'a, E: ColumnValueEncoder> GenericColumnWriter<'a,
E> {
if let Some(ref mut cmpr) = self.compressor {
let mut compressed_buf =
Vec::with_capacity(uncompressed_size);
cmpr.compress(&buffer[..], &mut compressed_buf)?;
+ compressed_buf.shrink_to_fit();
Review Comment:
I wonder if we should apply the same optimization to V2 path below 🤔
Also, @mapleFU recently updated the compression check for V2 pages to use
the uncompressed values if the compression didn't actually reduce the space.
Maybe we should apply that to V1 pages too
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
