zwy991114 commented on code in PR #66169:
URL: https://github.com/apache/doris/pull/66169#discussion_r3803836403
##########
be/src/util/block_compression.cpp:
##########
@@ -1123,47 +1231,51 @@ class ZstdBlockCompression : public
BlockCompressionCodec {
compressed_buf.size = max_len;
}
- // set compression level to default 3
- auto ret = ZSTD_CCtx_setParameter(context->ctx,
ZSTD_c_compressionLevel,
- ZSTD_CLEVEL_DEFAULT);
- if (ZSTD_isError(ret)) {
- return Status::InvalidArgument("ZSTD_CCtx_setParameter
compression level error: {}",
-
ZSTD_getErrorString(ZSTD_getErrorCode(ret)));
- }
- // set checksum flag to 1
- ret = ZSTD_CCtx_setParameter(context->ctx, ZSTD_c_checksumFlag, 1);
- if (ZSTD_isError(ret)) {
- return Status::InvalidArgument("ZSTD_CCtx_setParameter
checksumFlag error: {}",
-
ZSTD_getErrorString(ZSTD_getErrorCode(ret)));
- }
-
ZSTD_outBuffer out_buf = {compressed_buf.data,
compressed_buf.size, 0};
+ {
+ SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER(
+
ExecEnv::GetInstance()->block_compression_mem_tracker());
+ auto ret = ZSTD_CCtx_setParameter(context->ctx,
ZSTD_c_compressionLevel,
Review Comment:
考虑到 ZSTD_CCtx_setPledgedSrcSize() 只能降低单个 ZSTD context 的内存，无法限制多 level 的
ZSTD/LZ4HC context 数量累积，因此移除原有budget，但保留全局 idle context 数量限制
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
