XiaoHongbo-Hope opened a new pull request, #1166: URL: https://github.com/apache/paimon-rust/pull/1166
## Summary - Decode selected shredded Variant residual scalars directly, without rebuilding a `GenericVariant` or copying shared metadata for every field. - Keep eligible typed siblings column-wise when a projection also contains residual fields. - Reuse only successfully validated metadata through an exact-byte, thread-local cache (at most 4 entries / 1 MiB of metadata bytes; entries over 512 KiB bypass it). Each residual value is still validated. No file-format, public API, or storage I/O changes. ## Validation - `cargo +1.94.0 test --locked --offline -p paimon --lib variant`: 99 passed, 1 ignored. - `cargo +1.94.0 test --locked --offline -p paimon --lib`: 4,259 passed, 7 ignored. - `cargo +1.94.0 fmt --all -- --check` and `git diff --check`: passed. - Covers typed/residual/mixed values, SQL and Variant nulls, nested object/array paths, strict and try casts, malformed value/metadata, conflicting typed/residual values, different metadata bytes, and cache bounds. ## Synthetic decode-only comparison An ignored, repeatable test builds 5,342 metadata keys and 200 projected fields, including 27 integer residuals read as `FLOAT`. It checks identical values, then measures 512 in-memory rows for five alternating rounds on Rust 1.94 release mode (8-vCPU Intel Xeon Platinum 8269CY). Median: existing extraction + cast **54.2 ms**, direct residual cast **2.83 ms**. Both paths run in the same patched binary, so this isolates the extraction method, not total PR impact. This is not Parquet/OSS I/O or DataLoader throughput; no customer data or workload is used. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
