github-actions[bot] commented on code in PR #68593:
URL: https://github.com/apache/doris/pull/68593#discussion_r4129170064
##########
contrib/apache-orc:
##########
@@ -1 +1 @@
-Subproject commit be0f1b73a7aeb78824a03e0dcb692c50a176d513
+Subproject commit 34a46c9e2de3214eea527f50c45c94264c14cdcd
Review Comment:
[P2] Allocate the ORC input scratch only when a chunk spans input buffers.
On ARM this submodule update switches ZLIB streams to
`ZlibDecompressionStreamByLibDeflate`, whose `BlockDecompressionStream` base
immediately allocates and zeroes `inputDataBuffer(pool, blockSize)` for every
stream, in addition to the existing output buffer. A nullable direct-string
column opens PRESENT, LENGTH, and DATA streams; 100 selected columns at a valid
256 KiB block size therefore add about 75 MiB per active reader before data is
read. Doris defaults to an 8 MiB natural input read, so this scratch is often
unused. `NextDecompress` needs it only for split chunks; lazy allocation would
preserve the fast path without this wide-scan memory cost.
##########
be/src/util/block_compression.cpp:
##########
@@ -1663,13 +1653,7 @@ Status
get_block_compression_codec(tparquet::CompressionCodec::type parquet_code
*codec = ZstdBlockCompression::instance();
break;
case tparquet::CompressionCodec::GZIP:
Review Comment:
[P1] Reject empty GZIP input when the page expects output. On AArch64 this
now selects `GzipBlockCompressionByLibdeflate`, whose empty-input branch
returns OK and sets `output->size = 0` even when the caller provided a nonzero
destination. A dictionary page with `num_values=1`, `uncompressed_page_size=4`,
and `compressed_page_size=0` reaches the legacy Parquet reader; it checks only
Status and then reads the unwritten four-byte allocation as an INT32 dictionary
value. The previous ARM zlib path rejected that page, and the new test
exercises empty input only with an empty destination. Please reject this size
mismatch and test it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
