JingsongLi opened a new pull request, #861: URL: https://github.com/apache/paimon-rust/pull/861
### Purpose Add Java Paimon-compatible manifest block sidecars so scans can avoid decoding Avro blocks that cannot match partition, row-ID, or bucket filters. ### Brief change log - Implement the version-1 `ManifestSidecar` codec, builder, validation, CRC checking, block selection, and ranged block reads. - Record physical Avro OCF block offsets and lengths. - Generate and publish `.avro.sidecar` files through manifest `_EXTRA_FILES` during commits. - Use sidecars in table scans with conservative fallback for missing, corrupt, unsupported, or mismatched files. - Preserve incremental ADD/DELETE validation before row-range pruning. - Add `manifest.sidecar.enabled`, inheriting `manifest-sort.enabled` when unset. ### Tests - `cargo test -p paimon --lib --quiet` — 2859 passed, 0 failed, 2 ignored. - `cargo test -p paimon manifest_sidecar --lib --quiet` — 10 passed. - `cargo clippy -p paimon --lib --tests -- -D warnings`. - `cargo fmt --all -- --check` and `git diff --check`. ### API and Format Adds public `ManifestSidecar`, `ManifestSidecarBuilder`, selection/block types, and the `.avro.sidecar` suffix. The binary format is compatible with Java Paimon ManifestSidecar version 1 and is explicitly referenced from manifest metadata. ### Documentation Documents the table option, format compatibility, publication lifecycle, and fallback behavior in `docs/src/sql.md`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
