lucasfang opened a new issue, #365: URL: https://github.com/apache/paimon-cpp/issues/365
## Search before asking - [x] I searched in the [issues](https://github.com/apache/paimon-cpp/issues) and found nothing similar. ## Motivation `WarmupLevel::DECODED` is the current default, so every read that goes through the prefetching reader starts a background decode loop on the next file while the current one is still being consumed. That is the most aggressive level: it materializes decoded Arrow batches for a file the query has not reached yet, so the peak footprint of a read carries the decoded batches of the current file plus those of the next one, per reader in flight. A merge-on-read section with `k` sorted runs warms one file per leaf, which multiplies that extra decoded buffer by `k`. The cost is paid up front and is wasted whenever the scan stops early — a `LIMIT`, or a predicate selective enough to empty a split — because the warmed file is then never read. The latency it hides is dominated by the remote fetch, which the cheaper `RAW` level already covers, so the default trades a large amount of memory for the smaller, decode-side part of the win. ## Solution Make `WarmupLevel::RAW` the default and leave `DECODED` as an explicit opt-in for latency-bound readers that can afford the memory. - `ReadContextBuilder` defaults `warmup_level_` to `RAW`, in both the member initializer and `Reset()`, so a reused builder keeps the same default. - `DataFileReadOptions::warmup_level` defaults to `RAW` as well, which is what an internal caller that builds read options directly gets. - The enum and `SetWarmupLevel()` documentation move the "this is the default" note to `RAW`. No enum value, signature, or plumbing changes: `NONE`, `RAW` and `DECODED` all keep their behavior, and a caller that sets a level explicitly is unaffected. ## Anything else? One behavior consequence worth stating: `RAW` warms through the read-ahead cache and falls back to `NONE` when that cache is disabled, so a reader built with `SetReadAheadCacheEnabled(false)` now gets no warmup by default, where it previously got a background decode. Such a reader has to ask for `DECODED` explicitly to keep warming. This is a default-value change to public API behavior in `include/paimon/`; no signature, storage format, or protocol change. ## Are you willing to submit a PR? - [x] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
