csun5285 commented on code in PR #69221:
URL: https://github.com/apache/doris/pull/69221#discussion_r4336077353
##########
be/src/storage/segment/column_writer.h:
##########
@@ -651,12 +659,29 @@ class VariantSubcolumnWriter : public ColumnWriter {
std::span<const uint8_t> outer_nulls);
Status _ensure_input_format(const VariantColumnData& column);
Status _initialize_v2_builder();
+ Status _promote_chunks(DataTypePtr* inferred_type);
+ Status _seal_chunk();
+ // Bounded value chunks; each chunk is finalized and flushed independently
so the writer never
+ // holds a segment-sized ColumnString.
+ struct ValueChunk {
+ std::unique_ptr<VariantPathBuilder> builder;
+ size_t row_begin = 0;
+ size_t rows = 0;
+ };
+ static constexpr size_t kChunkBytes = 256ULL << 20;
+ static constexpr size_t kChunkValues = 4ULL << 20;
bool is_finalized() const;
bool _is_finalized = false;
ordinal_t _next_rowid = 0;
size_t none_null_size = 0;
VariantWriterInputFormat _input_format = VariantWriterInputFormat::UNSET;
std::unique_ptr<VariantPathBuilder> _v2_builder;
Review Comment:
直接把这个 换成 `std::vector<std::unique_ptr<VariantPathBuilder> _v2_builder>`
是不是就简单了，compaction 是顺序写的 row_begin rows 这些 也不用记
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
