csun5285 commented on code in PR #66169:
URL: https://github.com/apache/doris/pull/66169#discussion_r3710921115
##########
be/src/storage/tablet/tablet_schema.h:
##########
@@ -109,6 +109,9 @@ class TabletColumn : public MetadataAdder<TabletColumn> {
void set_type(FieldType type) { _type = type; }
bool is_key() const { return _is_key; }
bool is_nullable() const { return _is_nullable; }
+ bool has_compression() const { return _has_compression; }
Review Comment:
这个用 CompressionTypePB 是否是unkown 判断即可
##########
gensrc/proto/olap_file.proto:
##########
@@ -422,6 +422,9 @@ message ColumnPB {
// Number of buckets used to store doc map in variant doc mode.
optional int32 variant_doc_hash_shard_count = 33 [default = 64];
optional bool variant_enable_nested_group = 34 [default = false];
+ // per-column generic compression override; absent => inherit table-level
compression
+ optional segment_v2.CompressionTypePB compression_type = 35;
Review Comment:
增加一个默认值是 unkown
##########
be/src/storage/tablet/tablet_schema.h:
##########
@@ -302,6 +305,10 @@ class TabletColumn : public MetadataAdder<TabletColumn> {
bool _has_default_value = false;
std::string _default_value;
+ bool _has_compression = false;
Review Comment:
不需要这个变量
##########
be/src/util/block_compression.cpp:
##########
@@ -1615,6 +1636,84 @@ Status
get_block_compression_codec(segment_v2::CompressionTypePB type,
return Status::OK();
}
+// Process-wide registry of level-aware codecs, keyed by (type, level). All
+// column writers that request the same codec+level share one instance, so its
+// internal context pool is reused according to actual write concurrency rather
+// than allocated once per column. Instances live for the process lifetime
(like
+// the type-only singletons above), so their native contexts are never torn
down
+// per segment.
+namespace {
+class LeveledCompressionCodecPool {
+public:
+ static LeveledCompressionCodecPool& instance() {
+ static LeveledCompressionCodecPool s_instance;
+ return s_instance;
+ }
+
+ Status get(segment_v2::CompressionTypePB type, int level,
BlockCompressionCodec** codec) {
Review Comment:
这里如果是旧表的话会传入 level = 0，之前旧表的默认的 ZSTD 或者 LZ4HC 的 level 是 0 吗
##########
fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java:
##########
@@ -1022,6 +1022,12 @@ private boolean processModifyColumn(ModifyColumnOp
modifyColumnOp, OlapTable ola
&& modColumn.getDataType() ==
PrimitiveType.VARIANT) {
lightSchemaChange =
olapTable.getEnableLightSchemaChange();
}
+ // compression-only change: the per-column codec is
per-segment metadata,
Review Comment:
先不支持 schema change
##########
be/src/storage/rowset/segment_creator.cpp:
##########
@@ -144,6 +144,7 @@ Status
SegmentFlusher::_create_segment_writer(std::unique_ptr<segment_v2::Segmen
writer_options.mow_ctx = _context.mow_context;
if (no_compression) {
writer_options.compression_type = NO_COMPRESSION;
+ writer_options.suppress_compression = true;
Review Comment:
这个变量不需要，当设置 ColumnMetaPB 的时候优先使用 TabletColumn 的 compression_type，如果
TabletColumn 的 compression_type 是 UNKOWN ，使用 writer_options.compression_type。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
