eldenmoon commented on code in PR #69221:
URL: https://github.com/apache/doris/pull/69221#discussion_r4336272716
##########
be/src/storage/segment/column_writer.h:
##########
@@ -651,12 +659,29 @@ class VariantSubcolumnWriter : public ColumnWriter {
std::span<const uint8_t> outer_nulls);
Status _ensure_input_format(const VariantColumnData& column);
Status _initialize_v2_builder();
+ Status _promote_chunks(DataTypePtr* inferred_type);
+ Status _seal_chunk();
+ // Bounded value chunks; each chunk is finalized and flushed independently
so the writer never
+ // holds a segment-sized ColumnString.
+ struct ValueChunk {
+ std::unique_ptr<VariantPathBuilder> builder;
+ size_t row_begin = 0;
+ size_t rows = 0;
+ };
+ static constexpr size_t kChunkBytes = 256ULL << 20;
+ static constexpr size_t kChunkValues = 4ULL << 20;
bool is_finalized() const;
bool _is_finalized = false;
ordinal_t _next_rowid = 0;
size_t none_null_size = 0;
VariantWriterInputFormat _input_format = VariantWriterInputFormat::UNSET;
std::unique_ptr<VariantPathBuilder> _v2_builder;
Review Comment:
顺序写，所以不用记录 rows 这个不完全成立：NULL 占据的逻辑行数仍要记录，只是可以由 builder 统一负责。跨 chunk 类型提升和
JSONB 回退也需要这个。
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
