github-actions[bot] commented on code in PR #67468:
URL: https://github.com/apache/doris/pull/67468#discussion_r3921182215
##########
be/src/storage/segment/vertical_segment_writer.cpp:
##########
@@ -569,67 +686,120 @@ void VerticalSegmentWriter::_abandon_index_staging() {
}
}
-// A failure below can land AFTER the ANN and BKD indexes have already been
built
-// into their staging files. The caller then returns before
close_inverted_index(),
-// so the seal that would have consumed them never runs, and the rowset writer
-// keeps this segment's IndexFileWriter -- with its staging files and their
open
-// descriptors -- until the whole load or compaction unwinds. Drop them here.
-//
-// ONLY on failure. On the success path close_inverted_index() is what consumes
-// the staging, so dropping it here would silently seal a container with no ANN
-// or BKD index in it.
-Status VerticalSegmentWriter::finalize_columns_index(uint64_t* index_size) {
- Status status = _finalize_columns_index_impl(index_size);
- if (!status.ok()) {
- _abandon_index_staging();
+Status VerticalSegmentWriter::_finalize_columns_data() {
+ // check disk capacity before writing the buffered pages out
+ if (_data_dir != nullptr &&
_data_dir->reach_capacity_limit((int64_t)estimate_segment_size())) {
Review Comment:
[P1] Exclude finalized key-index bytes from value-group capacity checks
In a multi-group vertical compaction, the key group is finalized before each
value group, but clear() leaves
_primary_key_index_builder/_short_key_index_builder populated. The new
_finalize_columns_data() calls estimate_segment_size() for the value group, and
_key_index_size() adds those already-written key-index bytes to the incoming
size. Near the flood threshold this can return DISK_REACH_CAPACITY_LIMIT even
when the value-group buffers fit. Please exclude finalized key-index state for
has_key == false (or reset/snapshot it) before this capacity check.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
