lucasfang opened a new issue, #289: URL: https://github.com/apache/paimon-cpp/issues/289
Labels: enhancement ## Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-cpp/issues) and found nothing similar. ## Motivation Reading a primary-key table in merge-on-read mode serializes the first read of every data file behind the consumption of the previous one, so on remote storage the merge pays one network round trip per file instead of overlapping them. A split is partitioned into sections of overlapping files, and a section is merged by a `LoserTree` (the default sort engine) whose leaves are the section's sorted runs. Each run is a `ConcatKeyValueRecordReader` over its files, and each file is a `KeyValueDataFileRecordReader` over a stack of `FileBatchReader` wrappers that ends in `DelegatingPrefetchReader` and `PrefetchFileBatchReaderImpl`. `PrefetchFileBatchReaderImpl` issues its prefetch reads from a background thread that it starts lazily, inside the first `NextBatchWithBitmap()` call, so nothing of a file is in flight before the consumer actually asks that file for its first batch. That costs at two levels: - Within a run the files are read strictly one after another, and a filtered read of a keyed table yields about one batch per file. The consumer therefore reaches the end of one file and only then starts the next file's thread and first read, so the two never overlap. - Across the runs of a section, `LoserTree::InitializeIfNeeded` advances its leaves one at a time and each advance blocks on that leaf's own first read. The leaves are independent sorted runs, so with `k` runs in a section the merge waits for `k` first reads in sequence even though they could all be in flight at once. The latency being serialized is dominated by remote storage access, so on a section with several runs and several files per run it is a multiple of a single round trip spent doing nothing but waiting. ## Solution Add one optional `Warmup()` verb to the two reader abstractions and call it where the consumer already knows what it is going to read next, so the first read of that reader is issued while the current one is still being consumed. - Declare `virtual Status Warmup()` with a default `Status::OK()` body on both `FileBatchReader` and the internal `KeyValueRecordReader`. A reader with nothing to start, or one not yet ready to start it, is a no-op, so no existing implementation has to change and warming can never fail a read. - Implement it once, in `PrefetchFileBatchReaderImpl`, by extracting the lazy background-thread start into `EnsureBackgroundThread()` and sharing that between `NextBatchWithBitmap()` and `Warmup()`, so warming and reading cannot diverge. Warmup returns OK without starting anything when the read ranges are not refreshed yet or the reader is shut down; the read path keeps rejecting the former, so a genuinely unprepared read is not hidden. - Forward the call through the wrappers a data file's stack is built from (`FieldMappingReader`, `CompleteRowTrackingFieldsBatchReader`, `ApplyDeletionVectorBatchReader`, `ApplyBitmapIndexBatchReader`, `ShreddingFileReader`, `VectorFileBatchReader`, `DelegatingPrefetchReader`) and through `KeyValueDataFileRecordReader` on the KeyValue side. `FieldMappingReader` is the outermost wrapper of every file's stack, so swallowing the call there would leave the whole stack cold, and `DelegatingPrefetchReader` returns OK when prefetching is off because it bypasses the prefetch layer entirely in that case and a bypassed reader has no background thread to start. - Warm one file ahead in `ConcatKeyValueRecordReader::NextBatch()`, which is idempotent and so costs one virtual call plus a pointer test per already-warm reader. The lookahead is one file because each warm file holds a prefetch queue of its own, and a strictly sequential consumer cannot use more overlap than the file it will read next. - Warm every leaf in `LoserTree::InitializeIfNeeded()` before the advance loop, in the same order the loop consumes them, so the leaf the merge blocks on first is the one whose read was started first and the loop then collects reads that are already in flight. - Build a section's readers in one pass: flatten the section's runs into one file list, call `CreateRawFileReaders` once for it, and hand the readers back to their run by position, with both section read paths (merge and raw) going through that one build. This drops `CreateReaderForRun`, gives one `FieldMappingBuilder` per section instead of one per run, and keeps the two paths from producing different reader stacks for the same section. Since readers are matched to runs positionally, a build that dropped a reader would shift every following file into the wrong run, so the existing `assert` becomes an explicit `Status::Invalid`. ## Anything else? Warming changes when a file is opened, never what it reads: ordering, merging, filtering, deletion vectors and metrics are untouched, and warmup is unconditional with no configuration knob. Two known limits, both left as follow-ups. Warming does not cross a section boundary yet, because sections are wrapped in `SortMergeKeyValueRecordReader` inside the outer `ConcatKeyValueRecordReader` and that wrapper holds a `SortMergeReader`, which has no warmup verb to forward to. And the `min-heap` sort engine gets the intra-run warmup but not the cross-run warmup at merge initialization, since only `LoserTree::InitializeIfNeeded` warms its leaves. `FileBatchReader` is a public exported class, so the new virtual is additive and source compatible for every existing subclass, but it does change the vtable layout of that class. ## Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
