bobhan1 opened a new pull request, #70563: URL: https://github.com/apache/doris/pull/70563
### Background Query read-ahead from #67292 predicts data pages before scalar decoding, fetches compressed file ranges asynchronously, and feeds consumed ranges into the existing File Cache writeback and hole-fill services. SegmentIterator currently coordinates page plans and registers callbacks into the byte-reading layer. ColumnIterator already defines the query-facing cursor, pruning, access-path, and lazy-materialization contract. ### What problem does this PR solve? Issue Number: None Related PR: #67292 (introduces query read-ahead and File Cache range services), #70558 (alternative composition implementation; not a dependency) PR Type: Refactor Problem Summary: Move query page prediction behind a complete `ReadAheadColumnIterator` decorator so callers provide candidate rows, dependency roles, and scan direction without managing scalar page plans. The decorator forwards the full ColumnIterator interface and owns the per-column read-ahead controller; its inner iterator remains the sole owner of read state and decoding. A synchronous physical-page observer covers page transitions inside a batch and dictionary inspection during initialization. The remaining diff replaces the centralized range registry and migrates scan, point-query, and row-store integration and tests to independent column plans. ### Details - **Change map:** 2,046 changed production lines across 25 files and 2,439 changed test lines across 9 files; no generated files or documentation changes. The production changes extract shared immutable resources, a per-column controller and an IO-only exact-slice buffer; add the complete decorator and page observer; simplify scan/rowid callers; and remove SegmentReadAhead. Tests replace the removed central-owner model and cover both the underlying range services and the decorator contract. - **Moves vs. logic changes:** Window prediction, shared query/BE admission, cancellation, per-submission writeback epochs, consumed-range notification, Page Cache behavior, checksum/decoder fallback, and the File Cache background services are retained. The controller moves into the decorator, and FileColumnIterator only invokes a generic foreground page observer. Existing state accessors become virtual where necessary so the decorator forwards state rather than copying it; metadata capability and the protected observer setter have default no-op/null implementations for other iterator types. - **Behavior change:** Each eligible column independently plans and submits its ranges, preserving coalescing within a column and the shared query budget. Cross-column GET coalescing is removed, so request counts, admission order, and plan counters can change. Active decorators suppress the legacy prefetcher; without active resources, the full legacy interface is forwarded. - **Compatibility:** The existing Cloud/query/two-switch gate and configuration defaults remain. Only resolved top-level physical scalars are decorated, after access paths and type checks; nested children, extracted Variant paths, synthetic/default columns, and empty iterators retain their existing path. The original reader and on-disk/RPC formats are unchanged. - **Performance impact:** Six existing state accessors become virtual for all iterators; enabled scalar reads also add decorator forwarding and a synchronous observer call at physical page boundaries. No cluster or remote-object-store performance comparison was run, so the effect of independent ranges and extra virtual dispatch is not quantified. - **Rollback:** Revert this single commit; the File Cache hole-fill queue, scheduler algorithm, and asynchronous writer implementation are unchanged. ### Testing - Local production ASAN/PCH build: `./build.sh --be -j100`, `MAKE_TEST=OFF`; glibc compatibility gate passed. - Local ASAN/PCH BE UT: `./run-be-ut.sh --run -j100 --filter=...`; all 240 tests in 18 suites passed. - Repository clang-format 16 formatter and full format check passed. - Header/build hygiene and `git diff --check` passed. - clang-tidy was not run. CI, cluster regressions, and remote-object-store performance tests have not been run. Exact UT filter: ```text ReadAheadColumnIteratorTest.*:ColumnReadAheadTest.*:ColumnReaderTest.*:SegmentIteratorReadAheadTest.*:RowIdReadAheadTest.*:ReadAheadRangeBufferTest.*:FileRangeReadSchedulerTest.*:FileRangePlannerTest.*:AsyncCachedRemoteFileReaderTest.*:*PointQuery*:*CountEmit*:SegmentIteratorTopNRuntimeFilterTest.*:QueryContextReadAheadTest.*:RangeCacheWritebackTest.*:RangeWritebackDispatcherTest.* ``` New tests exercise shared cursor and error propagation, inner-owned read state and filtered placeholders, nested control flow versus parent metadata, pruning results, legacy prefetch forwarding, real dictionary-init corruption/retry, string OFFSET access paths, and factory eligibility/exclusions. Existing targeted suites cover page windows, forward/reverse scans, cache hits and budget rejection, exact rowid reads, point queries, query cancellation, range buffers, scheduling, and File Cache writeback. ### Notes for Reviewers Start with `read_ahead_column_iterator.h`, the virtual state accessors and protected observer hook in `column_reader.h`, and the factory gate in `segment.cpp`. The observer is attached before inner initialization because dictionary inspection can read a data page there. Retries remain at the physical loader, where retrying cannot duplicate an already appended batch. The observer is borrowed only synchronously and detached before decorator destruction; workers never retain an iterator or observer. This is an independent alternative to #70558, based on the same analysis baseline. It does not include that PR as a parent commit. ### Release note None ### Check List (For Author) - Behavior changed: Yes (see Details) - Does this need documentation: No; existing configuration and defaults are unchanged. - Pre-submission status: Partially verified: the production ASAN build, targeted ASAN UT, format, and header/build hygiene pass locally; CI, cluster/performance validation, and human review remain pending. ### AI Disclosure - Tool: Codex (GPT-6) - Generated: Code, tests, and this description. - Human review: Pending; this Draft PR is for implementation and design review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
