bobhan1 opened a new issue, #67611:
URL: https://github.com/apache/doris/issues/67611
## Summary
Track the delivery of PageIO read-ahead, I/O coalescing, and cache-block
hole filling as four separately reviewable PRs.
The complete implementation is available in #67292 at
`d6c611b5918f82347d969dafa7d48354fe1e49e2`. That PR remains unchanged as an
end-to-end reference. The split PRs will reuse its component commits, with only
the changes needed to integrate them independently with master.
## Motivation
Storage-compute separated queries pay both object-store request latency and
read amplification:
- Data pages normally target 64 KiB before compression and can be around
20–30 KiB after compression, while the file-cache block is normally 1 MiB.
Fetching a full block for a sparse page read transfers substantially more data
than the query needs.
- Small scanner batches limit how many pages become visible together.
Loading one page at a time leaves object-store reads serial even when later
pages or other columns could be read concurrently.
- Exact-range reads and read-ahead need a cache population path: useful
complete blocks can be written immediately, while partial blocks need
background hole filling before entering the existing fixed-block asynchronous
cache writer.
The intended result is bounded concurrent page reads, controlled I/O
coalescing, and best-effort cache population. Performance improvements and safe
defaults must be validated with workloads; this issue does not claim measured
speedups.
## End-to-end architecture
```mermaid
flowchart TD
Regular["Regular segment scans: eager and lazy byte windows"] --> Plan
Rowids["TopN second-stage and point-query data pages"] --> Plan
Plan["FileRangePlanner: coalesce ranges and optionally complete boundary
blocks"] --> Read
Read["FileRangeReadScheduler: query/BE byte admission; one range per
thread-pool task"] --> Cache["CachedRemoteFileReader: exact reads without
foreground cache writes"]
Cache --> Consume["PageIO consumes slices of completed ranges"]
Consume --> Writeback["RangeCacheWriteback: submit consumed ranges"]
Writeback --> Full["Complete cache blocks"]
Writeback --> Partial["Partial cache blocks"]
Partial --> Fill["PartialBlockWritebackManager: merge queued fragments
and fill holes in background"]
Fill --> Full
Full --> Persist["Existing AsyncCacheWriteManager"]
```
## Delivery plan
Each PR includes its own component tests. PR 1 starts from current upstream
master; later PRs follow the dependency order below. The checkboxes track
**merged**, not merely opened, PRs.
```mermaid
flowchart LR
Infra["PR 1: range I/O infrastructure"] --> Hole["PR 2: cache writeback
and hole filling"]
Hole --> Scan["PR 3: regular segment page read-ahead"]
Scan --> Exact["PR 4: rowid and point-query page read-ahead"]
```
- [ ] **PR 1 — File-range I/O infrastructure** — preparing.
- `FileRangeCoalescer`: linear coalescing of ordered, disjoint intervals,
constrained by gap, merged size, and read amplification. An individual input
range may exceed the merge-size limit.
- `FileRangePlanner`: map requested intervals to coalesced reads, then
optionally extend reads to complete sufficiently covered cache-block boundaries.
- `FileRangeReadScheduler`: exact asynchronous range reads on the existing
`segment_prefetch_thread_pool`; query/BE resident-buffer budgets, task
completion, cancellation, and shutdown. No separate scheduling thread or extra
concurrency cap.
- `CachedRemoteFileReader`: reuse the remote-only-on-cache-miss path for
`NO_WRITE` reads; reuse in-flight cache-write buffers when they fully cover a
requested interval, including intervals spanning multiple blocks.
- `ExecEnv`/`QueryContext`: scheduler ownership, shared per-query read
context, runtime enable switch, and configurable byte budgets.
- Review focus: interval invariants and complexity; rejection and partial
submission; buffer lifetime/accounting; query cancellation and executor
lifetime. This PR does not connect scanner or PageIO callers yet.
- [ ] **PR 2 — Consumed-range cache writeback and background hole filling**
— planned; depends on PR 1.
- Complete-block submission and owned-buffer handoff to
`AsyncCacheWriteManager`, including low-priority spare-capacity admission.
- `RangeCacheWriteback`: split consumed ranges by cache-block boundaries
and route complete versus partial blocks.
- Hole planning: take the complement of covered intervals inside one cache
block, then apply the range-coalescing algorithm with hole-fill-specific
limits. Hole-fill requests remain within that block.
- `PartialBlockWritebackManager`: queued-fragment merging, bounded memory,
queued-task eviction, dynamically sized workers, and a dedicated remote-read
pool.
- Review focus: queued/active task ownership, locking and cancellation,
background GET concurrency, pressure handling, and complete-block handoff.
Cache population remains best effort.
- [ ] **PR 3 — Regular segment PageIO read-ahead** — planned; depends on PRs
1–2.
- `ColumnReadAhead` byte windows and `SegmentReadAhead` coordination,
wired into column readers, PageIO, and the regular `SegmentIterator` path.
- Eager and lazy columns participate together; dependent lazy columns use
smaller windows. Consumption/discard advances the window and drives refill.
- Consume completed range slices, retain synchronous fallback, and submit
eligible consumed ranges for writeback.
- Review focus: batch/window progression, eager/lazy dependencies, page
lifetime, fallback, and avoidance of writes for unused speculative ranges.
- [ ] **PR 4 — Rowid fetches, TopN second-stage reads, and point queries** —
planned; depends on PR 3.
- Use exact requested rowids to identify data pages and submit their
ranges concurrently.
- Reuse the shared planning, reading, and writeback machinery; keep the
added logic separate from existing query execution.
- Review focus: sparse/disjoint page selection, second-stage and
point-query correctness, fallback, and resource cleanup.
## Source commit mapping
The order below is the intended cherry-pick order within each PR. Follow-up
adaptations belong with the affected component rather than in a separate mixed
cleanup commit.
| PR | Commits from #67292 |
| --- | --- |
| 1 | `8bf17ed7648` coalescer → `9393cb97620` planner → `1ebdf0d0079`
scheduler → `4655d17ee2e` cached reads → `b381a736a15` runtime ownership |
| 2 | `d92714f7acd` complete-block API → `17f357a70d9` range dispatcher →
`ec7c4963239` hole planner → `b47e7134ad9` owned-buffer handoff → `c7659f1e81e`
low-priority admission → `b390f082270` partial-block manager → `c47d0ea24a6`
runtime wiring → `f88c23f6c3f` range writeback |
| 3 | `bef2809fd86` column windows → `3b5f2db1555` segment coordination →
`a42bf1b617d` column-reader wiring → `204ff574957` segment wiring →
`8d0ebc74cb5` writeback wiring |
| 4 | `bc273b96872` rowid/TopN fetches → `d6c611b5918` point queries |
## Validation and completion criteria
- Each PR builds independently against its declared base and passes focused
ASAN unit tests for its included components, formatting checks, and BE
build-hygiene checks.
- PR 1 covers coalescing/planning boundaries, oversized inputs, budgets,
executor rejection, cancellation, shutdown, exact cache reads, and fully
covered in-flight-buffer reads.
- PR 2 covers block splitting, hole complements/coalescing, fragment
merging, pressure/eviction, worker resizing, failures, and complete-block write
admission.
- PRs 3–4 cover regular scans and exact-rowid/point reads, including
disabled/enabled operation, fallback, and cleanup. Run local storage-compute
separated import/query smoke and suitable self-contained regression cases after
those paths are integrated.
- Keep measured results on the corresponding PRs. Close this issue only when
all four PRs are merged and the integrated paths have been validated.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
