github-actions[bot] commented on code in PR #70350:
URL: https://github.com/apache/doris/pull/70350#discussion_r4560434941
##########
be/src/storage/index/inverted/snii/compaction/snii_index_compaction.cpp:
##########
@@ -265,10 +270,14 @@ Status SniiPlainT2MergePlan::prepare(std::vector<const
reader::LogicalIndexReade
std::vector<std::unique_ptr<SniiPostingReadContext>> read_contexts;
read_contexts.reserve(source_indexes.size());
- for (const reader::LogicalIndexReader* source : source_indexes) {
+ for (size_t source_id = 0; source_id < source_indexes.size(); ++source_id)
{
+ const auto* source = source_indexes[source_id];
auto context = std::make_unique<SniiPostingReadContext>(
source, per_source_read_ahead_budget, memory_reporter.get());
RETURN_IF_ERROR(context->init());
+ if (!rowid_conversion.source_mapping(source_id).is_dense()) {
+ RETURN_IF_ERROR(context->reserve_rowid_cursor());
Review Comment:
[P2] Charge compressed row-ID cursors to the SNII merge cap
`reserve_rowid_cursor()` allocates 32 KiB per compressed source without a
`MemoryReporter` reservation; the shared row-ID decode workspace is also
excluded. These are new plan-owned decoder buffers, whereas the input mapping
is borrowed. For 1,024 eligible docs-only sources with no NULLs or norms, the
native read-ahead caches reserve 64 MiB and these cursors retain another 32
MiB. With `inverted_index_ram_buffer_size=80`, `prepare()` therefore succeeds
with at least 96 MiB of live plan buffers while its hard-limit reporter counts
only 64 MiB. Using 65,536 physical rows per source also exceeds the default 500
MiB compression threshold. Task-level allocator tracking does not enforce this
separate merge cap or trigger its intended over-budget rejection. Reserve the
cursor/shared-workspace bytes through the merge reporter before allocation and
retain the reservations for their lifetimes; add a compressed-mapping cap test.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
