csun5285 commented on code in PR #66169:
URL: https://github.com/apache/doris/pull/66169#discussion_r3794576291
##########
fe/fe-core/src/main/java/org/apache/doris/alter/MaterializedViewHandler.java:
##########
@@ -694,6 +694,20 @@ private List<Column>
checkAndPrepareMaterializedView(CreateMaterializedViewComma
return newMVColumns;
}
+ private Column toMVColumn(MVColumnItem mvColumnItem, OlapTable olapTable,
+ Map<String, String> sessionVariables) throws DdlException {
+ Column mvColumn = mvColumnItem.toMVColumn(sessionVariables);
+ if (mvColumnItem.getDefineExpr() instanceof SlotRef) {
+ SlotRef slotRef = (SlotRef) mvColumnItem.getDefineExpr();
+ Column baseColumn =
Preconditions.checkNotNull(olapTable.getColumn(slotRef.getColumnName()),
Review Comment:
抛异常这里
##########
be/src/util/block_compression.cpp:
##########
@@ -1123,47 +1231,51 @@ class ZstdBlockCompression : public
BlockCompressionCodec {
compressed_buf.size = max_len;
}
- // set compression level to default 3
- auto ret = ZSTD_CCtx_setParameter(context->ctx,
ZSTD_c_compressionLevel,
- ZSTD_CLEVEL_DEFAULT);
- if (ZSTD_isError(ret)) {
- return Status::InvalidArgument("ZSTD_CCtx_setParameter
compression level error: {}",
-
ZSTD_getErrorString(ZSTD_getErrorCode(ret)));
- }
- // set checksum flag to 1
- ret = ZSTD_CCtx_setParameter(context->ctx, ZSTD_c_checksumFlag, 1);
- if (ZSTD_isError(ret)) {
- return Status::InvalidArgument("ZSTD_CCtx_setParameter
checksumFlag error: {}",
-
ZSTD_getErrorString(ZSTD_getErrorCode(ret)));
- }
-
ZSTD_outBuffer out_buf = {compressed_buf.data,
compressed_buf.size, 0};
+ {
+ SCOPED_SWITCH_THREAD_MEM_TRACKER_LIMITER(
+
ExecEnv::GetInstance()->block_compression_mem_tracker());
+ auto ret = ZSTD_CCtx_setParameter(context->ctx,
ZSTD_c_compressionLevel,
Review Comment:
1. 这里没设 pledged src size。以前 level 固定是 3，单个 CCtx 才几 MB，22 之后就跳到数百 MB
了，怀疑这才是需要加 budget 那套的真正原因。
2. ZSTD 的工作内存由 (level, srcSize) 共同决定。
ZSTD_CCtx_setPledgedSrcSize(context->ctx, uncompressed_size); 按 64KB
钳完大约降到几百 KB，压缩率无损失。如果成立，
LeveledCompressionContextPoolBudget 可以整个去掉。
3. 这样可能导致输出的字节和之前不一样，建议测试一下。包括升降级之类的还是否ok
##########
fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java:
##########
@@ -1022,6 +1045,10 @@ private boolean processModifyColumn(ModifyColumnOp
modifyColumnOp, OlapTable ola
&& modColumn.getDataType() ==
PrimitiveType.VARIANT) {
lightSchemaChange =
olapTable.getEnableLightSchemaChange();
}
+ if (col.hasCompressionOverride() ||
modColumn.hasCompressionOverride()) {
Review Comment:
这里的col 表示当前表中已存在的列，这样会导致这一列只要指定了 COMPRESSION，之后对这一列任何 modify column
都会被拒，我们只期望modify column compression 被拒
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
