ClifHouck commented on code in PR #40957:
URL: https://github.com/apache/arrow/pull/40957#discussion_r1570815640
##########
cpp/src/parquet/column_writer_test.cc:
##########
@@ -1524,21 +1538,24 @@ TEST_F(ColumnWriterTestSizeEstimated, NonBuffered) {
auto required_writer =
this->BuildWriter(Compression::UNCOMPRESSED, /* buffered*/ false);
// Write half page, page will not be flushed after loop
- for (int32_t i = 0; i < 50; i++) {
+ for (int32_t i = 0; i < 127; i++) {
required_writer->WriteBatch(1, nullptr, nullptr, &i);
}
// Page not flushed, check size
EXPECT_EQ(0, required_writer->total_bytes_written());
EXPECT_EQ(0, required_writer->total_compressed_bytes()); // unbuffered
EXPECT_EQ(0, required_writer->total_compressed_bytes_written());
// Write half page, page be flushed after loop
- for (int32_t i = 0; i < 50; i++) {
+ for (int32_t i = 0; i < 127; i++) {
required_writer->WriteBatch(1, nullptr, nullptr, &i);
}
// Page flushed, check size
+ /* FIXME: For whatever reason, despite being flushed, this does not actually
Review Comment:
So I'm trying to run this down, but if someone familiar with with
`DELTA_BINARY_PACKED` encoding can chime in that would be helpful. Could this
possibly be a bug? Doesn't flushing imply that we're forcing the writer to
*actually* write out any un-flushed data?
It looks like flushing does actually happen, as in
`DeltaBitPackEncoder::Put` calls `FlushBlock()` in my testing, so maybe this is
just a book-keeping bug?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
