XiaoHongbo-Hope opened a new pull request, #997: URL: https://github.com/apache/paimon-rust/pull/997
### Purpose Wide numeric Variant workloads currently materialize many Python dictionaries and scalar objects before building numeric arrays. ### Changes - Extract ordered, literal top-level numeric fields into Arrow `FixedSizeList<Float32>`. - Reuse the Variant metadata and exact object layout within a batch while validating object offsets and selected child boundaries. - Preserve SQL null rows and represent missing or Variant-null fields as child nulls. - Expose `pypaimon_rust.data.variant_get_float32` and release the GIL during extraction. - Return `NotImplementedError` for non-numeric selected values so an integration can explicitly fall back. ### Scope and trade-offs - This reduces decoding and Python object materialization; it does not reduce Parquet or object-store I/O. - Supported values are Variant integer, float, double, and decimal. Conversion to `float32` may lose precision. - The result follows the requested field order. Field names are literal, not Variant path expressions. - Automatic PyPaimon selection and application SDK integration are intentionally outside this PR. ### Tests - `cargo test -p paimon --lib variant` (46 passed) - `pytest -q bindings/python/tests/test_variant.py` (1 passed) - `cargo clippy -p paimon --lib -p pypaimon_rust -- -D warnings` - `cargo fmt --all --check` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
