XiaoHongbo-Hope opened a new pull request, #988: URL: https://github.com/apache/paimon-rust/pull/988
### Purpose Linked issue: close #987 Native scan planning currently hard-codes 64 concurrent reads for manifest lists and manifest files. This ignores `scan.manifest.parallelism`, so a lower configured value does not protect object storage from request bursts when many planners start together. ### Brief change log - Parse `scan.manifest.parallelism` in `CoreOptions`. - Default to the process CPU count when the option is absent, matching the Java option contract. - Apply the configured bound to incremental manifest-list reads and manifest-file reads. - Reject zero and invalid values before calling `StreamExt::buffered`. For example, a table configured with `scan.manifest.parallelism=8` now limits each native planner to eight concurrent manifest reads instead of the previous fixed 64. ### Tests - `cargo test -p paimon test_scan_manifest_parallelism --lib` - `cargo test -p paimon table::table_scan::tests --lib` (108 passed) - `cargo clippy -p paimon --lib --tests -- -D warnings` - `cargo fmt --all -- --check` ### API and Format No public API or storage format change. This makes an existing table option effective in the native planner. ### Documentation No new option is introduced; behavior is aligned with the existing Java/Python option. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
