github-actions[bot] commented on code in PR #65674:
URL: https://github.com/apache/doris/pull/65674#discussion_r3588836827
##########
be/src/format_v2/parquet/reader/scalar_column_reader.cpp:
##########
@@ -142,33 +145,32 @@ Status ScalarColumnReader::read(int64_t rows,
MutableColumnPtr& column, int64_t*
return Status::InternalError("Parquet record reader is not initialized
for column {}",
_name);
}
- auto reader = leaf_reader();
- ParquetLeafBatch leaf_batch;
- RETURN_IF_ERROR(reader.read_batch(rows, &leaf_batch, rows_read));
+ auto& reader = leaf_reader();
+ RETURN_IF_ERROR(reader.read_batch(rows, &_leaf_batch, rows_read));
Review Comment:
Holding the batch in `_leaf_batch` now keeps the Arrow binary chunks
returned by `GetBuilderChunks()`/`GetResult()` alive after `append_values()`
has copied them into the Doris column. Those chunks own the string/binary
payloads and are only released when the next batch is collected or the
row-group reader is torn down, so a large STRING/BINARY batch can sit in memory
alongside the materialized output. The same lifetime extension shows up for
selected dictionary reads, nested reads, and the levels-only discard path.
Please release the chunk references once materialization/discard is complete
while still preserving vector capacity for reuse.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
