eldenmoon opened a new pull request, #70723: URL: https://github.com/apache/doris/pull/70723
### Background Doris stores Variant subcolumn metadata outside the segment footer in the V3 format. Compaction reads this metadata to collect path types and statistics. It can reuse the segment reader across attempts. ### What problem does this PR solve? Issue Number: None Related PR: None PR Type: Bugfix A transient page allocation failure can prevent later compactions from loading the same Variant metadata even after memory pressure subsides. The reader now retries the metadata load on the next call after a memory limit error. ### Details - **Frequency:** Affected segments keep returning a stale failure while subsequent compaction attempts reuse their cached reader. - **Root cause:** PageIO converts the allocation exception into `MEM_LIMIT_EXCEEDED`. The call-once guard caches this status for the lifetime of the reader. - **Why existing tests missed it:** They exercise metadata loading and caching with healthy page reads. - **Fix:** Let the Variant bulk metadata loader opt out of caching `MEM_LIMIT_EXCEEDED`. Concurrent attempts still use the existing lock. A retry skips paths that have already been merged. - **Other paths and branches:** Variant queries and compaction share the bulk loader. The same failure cache exists on branch-4.1 and branch-4.2. ### Testing - Before the fix, an ASAN test injected a single allocation exception during a real PageIO read. The second call on the same reader returned the stale error. - After the fix, the second call read the metadata and recovered the expected subcolumn and statistics. The third call reused the result without issuing more reads. - ASAN BE unit tests on dev-x passed 159 tests. Four tests were skipped because this build does not enable Nested Group writes. - The ASAN BE/FE build, all 14 related V3 regression suites, BE build hygiene, clang-format, and changed-line clang-tidy passed on dev-x. ### Release note Compaction can recover from a transient memory limit error while loading Variant external metadata. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Ready ### AI Disclosure - Tool: OpenAI Codex - Generated: Implementation, tests, and PR description - Human review: Pending -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
