bobhan1 opened a new pull request, #67612:
URL: https://github.com/apache/doris/pull/67612
### What problem does this PR solve?
Issue Number: Part of #67611
Related PR: #67292 (full implementation reference; unchanged)
This is **PR 1 of 4** in the PageIO read-ahead, I/O coalescing, and
cache-block hole-filling series. It provides the shared range-planning and
asynchronous-read infrastructure needed by the later PRs. The original
implementation is split by component so each part can be reviewed and tested
independently.
Storage-compute separated reads need to combine small, sometimes sparse
data-page intervals into useful remote requests, execute independent reads
concurrently, and retain the resulting buffers under query/BE memory limits.
Cache-aware planning and asynchronous execution have different
responsibilities: the planner decides the physical byte ranges; the scheduler
executes those exact ranges without deciding page windows or cache-writeback
policy.
**Scope:** range coalescing, optional boundary-block completion,
asynchronous range reads, exact no-write cache reads, and runtime ownership.
Regular segment scans, point queries, TopN second-stage reads, and background
hole filling are connected in later PRs. Setting the new enable switch alone
does not start PageIO read-ahead in this PR.
#### Component boundaries
```mermaid
flowchart LR
Input["Ordered, disjoint input ranges"] --> Planner["FileRangePlanner"]
Coalescer["FileRangeCoalescer"] --> Planner
Planner --> Plan["Physical ranges and input-to-buffer locations"]
Plan --> Scheduler["FileRangeReadScheduler"]
Query["QueryContext: shared cancellation and query byte budget"] -->
Scheduler
Exec["ExecEnv: scheduler ownership and BE byte budget"] --> Scheduler
Scheduler --> Pool["Existing segment_prefetch_thread_pool: one task per
range"]
Pool --> Reader["FileReader: exact read into an owned buffer"]
Reader --> Handle["FileRangeRead: status, statistics, and buffer slices"]
```
| Component | Responsibility | Deliberately outside its scope |
| --- | --- | --- |
| `FileRangeCoalescer` | One pass over sorted, disjoint intervals; apply
gap, merged-size, and read-amplification limits | Sorting, normalization, file
reads |
| `FileRangePlanner` | Coalesce inputs, consider completing boundary cache
blocks, and map each input to its final buffer | Buffers, submission, writeback
|
| `FileRangeReadScheduler` | Query/BE byte admission, one task per range on
the existing pool, completion, cancellation, and shutdown | Page windows,
coalescing, cache-block completion |
| `CachedRemoteFileReader` | Serve exact `NO_WRITE` reads from fully
covering in-flight buffers or downloaded cache, otherwise issue an exact remote
read | Speculative range writeback |
| `ExecEnv` / `QueryContext` | BE scheduler lifetime and shared per-query
context | Scanner integration, added in PR 3 |
#### Planning and read boundaries
Coalescing is linear in the number of inputs. A merge must satisfy all three
limits below; a single indivisible input larger than `max_range_bytes` stays as
one read.
| Option | Meaning |
| --- | --- |
| `max_gap_bytes` | Maximum gap between the current merged range and the
next input |
| `max_range_bytes` | Maximum byte span produced by a merge or optional
block completion |
| `max_read_amplification_ratio` | Maximum merged byte span divided by the
sum of original input bytes in that merged range |
After coalescing, the planner considers the boundary cache blocks of each
range. Blocks meeting `block_fill_min_coverage`, measured using original
requested bytes, become candidates. Candidates are tried in increasing order of
additional bytes; a completion is accepted only if the resulting physical range
respects `max_range_bytes`. The final short block uses valid file bytes for its
coverage denominator. The planner then returns the containing range index and
buffer offset for every input.
The coalescing amplification limit governs the initial merge pass; optional
block completion is separately governed by block coverage and final range size.
These are reusable API options. Workload-specific read-ahead and hole-fill
configuration is added with the corresponding consumers in later PRs.
#### Submission, buffer ownership, and failure behavior
```mermaid
flowchart TD
Submit["try_submit: validate the entire batch"] --> Admit{"Query and BE
bytes available?"}
Admit -- No --> Reject["Reject batch; no read handles or retained
reservation"]
Admit -- Yes --> Allocate["Reserve bytes and allocate every range
buffer"]
Allocate -- Failure --> Reject
Allocate -- Success --> Queue["Submit one thread-pool task per range"]
Queue -- "All rejected" --> Reject
Queue -- "At least one accepted" --> Handles["Return a handle for every
range"]
Handles --> Accepted["Accepted task: exact read and terminal status"]
Handles --> Rejected["Unsubmitted range: FAILED handle"]
Accepted --> Retain["Resident bytes remain charged while handles retain
buffers"]
Rejected --> Retain
Retain --> Release["Last handle released: free buffer and release
query/BE bytes"]
```
The existing thread pool owns queuing and controls execution concurrency.
The scheduler has no private task queue, scheduler thread, or extra concurrency
quota; it tracks accepted handles for cancellation and draining. Worker I/O
contexts own the query ID and use task-local statistics rather than
caller-owned statistics pointers.
Query cancellation skips queued reads and marks running reads cancelled
after their source I/O returns. Shutdown stops admission, cancels accepted
reads, and waits for their tasks before the shared executor is destroyed. The
scheduler never shuts down the shared thread pool itself.
#### Exact cached reads
```mermaid
flowchart TD
Request["Exact NO_WRITE range"] --> Inflight{"In-flight buffers cover
the whole request?"}
Inflight -- Yes --> Copy["Copy requested bytes; no remote read or cache
write"]
Inflight -- No --> Cache{"Downloaded cache blocks cover the whole
request?"}
Cache -- Yes --> Local["Read requested bytes from cache"]
Cache -- No --> Remote["One exact remote read; no block-alignment
expansion or cache population"]
```
The in-flight lookup supports requests spanning multiple cache blocks and
checks complete coverage before copying. Partial coverage keeps the existing
all-cache-or-remote behavior. No-write reads reuse
`_read_remote_only_on_cache_miss()`; write-mode resolution is kept in the main
reader implementation because it selects among read paths.
#### Runtime configuration
| BE configuration | Default | Application |
| --- | --- | --- |
| `enable_query_read_ahead` | `false` | Mutable switch introduced for later
PageIO consumers; no scanner read-ahead is wired by this PR |
| `read_ahead_max_bytes_per_query` | 256 MiB | Startup-configured limit on
range buffers retained by one query |
| `read_ahead_max_bytes_per_be` | 1 GiB | Startup-configured limit on range
buffers retained across the BE |
Cloud-mode `ExecEnv` creates the scheduler even while the switch is
disabled, so later consumers can observe runtime enablement. `QueryContext`
creates its shared range context lazily and propagates cancellation to it.
#### Commit-by-commit review
The five commits are cherry-picked from #67292 in dependency order:
| Commit | Source commit | Review scope |
| --- | --- | --- |
| 1 | `8bf17ed7648` | Coalescer and tests |
| 2 | `9393cb97620` | Cache-aware planner and tests |
| 3 | `1ebdf0d0079` | Scheduler, buffer lifecycle, and tests |
| 4 | `4655d17ee2e` | Exact no-write cache reads and in-flight-buffer tests |
| 5 | `b381a736a15` | Runtime ownership, configuration, and query-context
tests |
The only split-specific code adaptation is to keep scheduler destruction
before thread-pool destruction in commit 5; the full branch already contains
that ordering in a later commit. Include ordering was also formatted. The
original PR branch is untouched.
### Release note
Add internal file-range coalescing and asynchronous-read infrastructure.
Exact no-write cache reads can reuse fully covering in-flight write buffers.
Scanner/PageIO read-ahead and background hole filling will be enabled by later
PRs in #67611.
### Check List (For Author)
- Test
- [ ] Unit Test — focused ASAN build/run in progress; results will be
updated before review readiness.
- [x] Formatting — repository clang-format 16 formatting and check scripts
passed.
- [x] BE build hygiene and `git diff --check` passed.
- [ ] Regression test — scanner integrations are outside this PR;
integrated cloud smoke and regression belong to the later consumer PRs.
- Behavior changed:
- [x] Yes — exact no-write cache reads can reuse fully covered in-flight
data; this PR does not activate scanner read-ahead.
- Does this need documentation?
- [x] No — internal infrastructure; the series and component boundaries
are documented in #67611 and this description.
#### Validation coverage
Focused ASAN/PCH build and test command (no clean rebuild; `-j100`):
```bash
./run-be-ut.sh --run
--filter='FileRangeCoalescerTest.*:FileRangePlannerTest.*:FileRangeReadSchedulerTest.*:QueryContextReadAheadTest.*:AsyncCachedRemoteFileReaderTest.*:BlockFileCacheTest.async_write*:BlockFileCacheTest.cache_write_mode_is_resolved_for_each_read_context:BlockFileCacheTest.direct_partial_hit_with_downloaded_remainder_should_not_read_remote_again'
-j100
```
| Test area | Coverage |
| --- | --- |
| Coalescer / planner | Gap, size and amplification boundaries; oversized
input; optional block completion; EOF; input-to-buffer mapping |
| Scheduler | Shared executor, parallel execution, whole-batch byte
admission, partial executor rejection, buffer lifetime, cancellation races,
shutdown, allocation/read failures |
| Cached reader | Exact cold misses, complete in-flight coverage across
blocks, partial in-flight coverage, existing asynchronous cache-read paths |
| Query context | Shared context and cancellation before/after context
creation |
No workload performance claims or external-object-store regression results
are included.
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
