yiguolei opened a new pull request, #70738: URL: https://github.com/apache/doris/pull/70738
### Background Segments cache column readers across requests. Their indexes and Variant external metadata are initialized lazily, and `DorisCallOnce` retains the initialization result for later callers. ### What problem does this PR solve? Issue Number: None Related PR: #70723 (replaces external-metadata memory-error retry with reader reconstruction) PR Type: Bugfix Problem Summary: A transient IO or memory failure during lazy initialization can leave subsequent requests using the same failed reader, or leave ANN queries using brute-force fallback. Track each reader's health and rebuild unhealthy instances on subsequent cache access. Recursive checks cover owned readers without modifying their parents' stored health; conditional eviction protects replacements published concurrently. Recovery and concurrency tests account for 342 of the 731 changed lines. ### Details - **Change map:** 389 production lines and 342 test lines; no generated files or documentation. Production changes cover column health recording (66), index health and ANN integration (104), Variant/binary recursion (73), and cache eviction/publication (146). Tests cover retained errors, exceptions, independent health state, reconstruction and replacement races. - **Frequency:** Not observed locally; found by source inspection. Trigger: cached reader initialization fails, then a later request acquires that reader again. - **Root cause:** `DorisCallOnce` retains failures, but `ColumnReaderCache` previously returned cached instances without checking health. Direct reader calls and ANN fallback can bypass SegmentIterator error marking. - **Why existing tests missed it:** `ColumnReaderCacheTest` covered successful hits, LRU and concurrency, without failed-load reacquisition. `ExternalColMetaUtilTest` expected in-place memory-error retry; this PR updates that expectation. ANN tests lacked aggregate owner-health checks. Inspected cloud-cache and Variant SQL suites did not assert recovery after this injected failure; no new SQL test is added. - **Fix:** Each ColumnReader and IndexReader owns an `AtomicStatus`. Loading helpers record failures and rethrow exceptions unchanged. Aggregate health recursively checks owned readers. Cache checks run outside its mutex; eviction compares instance identity under the mutex so an old failure cannot delete a replacement. - **Other paths and branches:** Covers ordinal, zone-map, Bloom-filter, Variant external-metadata and ANN initialization. Segment invalidation remains. Query-local failures and independent inverted searcher-cache recovery are outside this fix; the latter remains follow-up work with no tracking issue. Cached branch-3.1/4.0/4.1/4.2 refs lacked these health checks, but were not refreshed and backport compatibility is unverified. The introducing PR is unknown. - **Behavior change:** External-metadata memory errors retire the reader instead of resetting its once guard. Existing references remain valid but are not repaired. A child's failure can evict its ancestor's cache entry without changing parent or sibling local health. ### Testing - Passed locally: `bash build-support/check-build-hygiene.sh`, `git diff --check`, and clang-format 16 changed-line comparison. - Added/updated BE tests cover load-error eviction, exception preservation, recursive health, publication races, Variant reconstruction and ANN owner health. They were not run: `run-be-ut.sh` stopped at Java environment checking; JDK 17, supported LLVM and third-party build dependencies are unavailable locally. - Compilation, before/after runtime reproduction, sanitizer tests, clang-tidy and cluster regression remain unverified. No CI result is claimed. ### Notes for Reviewers Cache hits traverse owned readers and acquire reader-local index locks; no performance benchmark was run. ### Release note Subsequent cache access can recover from transient failures initializing column readers and the indexes covered by this fix. ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No - Pre-submission status: Partially verified: compilation and BE unit tests have not run. ### AI Disclosure - Tool: Codex (GPT-6) - Generated: code, tests and this description - Human review: Not stated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
