sdf-jkl opened a new issue, #3705: URL: https://github.com/apache/iceberg-rust/issues/3705
### Apache Iceberg Rust version Upstream `main` at `4be78085219af9f1c2093bfcea66ff3d847c7613`; locked Arrow/Parquet 60.1.0. ### Describe the bug A manifest containing a `lower_bounds` or `upper_bounds` entry for a Variant field fails to load with: ```text DataInvalid => field v is not a primitive type ``` [`parse_bytes_entry`](https://github.com/apache/iceberg-rust/blob/4be78085219af9f1c2093bfcea66ff3d847c7613/crates/iceberg/src/spec/manifest/_serde.rs#L306-L325) requires every bound's field type to be primitive. `Type::Variant` fails that check before the bound bytes are decoded. Both bounds maps pass through this function when decoding a data file, so this blocks manifest loading even when the scan would not select the Variant column. The [Iceberg specification](https://iceberg.apache.org/spec/#bounds-for-variant) permits bounds under the Variant column's field ID. These encode per-path bounds as Variant objects; they are not ordinary binary min/max values. This is the manifest Variant-bounds lead listed as unfiled in #2411. Related: Variant umbrella #3685. Writing or using Variant statistics for pruning is separate from accepting manifests that contain them. ### To Reproduce Add this test to the existing `tests` module in `crates/iceberg/src/spec/manifest/_serde.rs`: ```rust #[test] fn review_variant_bounds_do_not_block_manifest_loading() { use super::{BytesEntry, parse_bytes_entry}; let schema = Schema::builder().with_fields(vec![ NestedField::optional(1, "v", Type::Variant(crate::spec::VariantType)).into(), ]).build().unwrap(); let entry = BytesEntry { key: 1, value: serde_bytes::ByteBuf::from(vec![1, 0, 0, 2, 0, 0]), }; let result = parse_bytes_entry(vec![entry], &schema); assert!(result.is_ok(), "Variant bounds must be read or ignored: {result:?}"); } ``` Run `cargo test -p iceberg review_variant_bounds_do_not_block_manifest_loading --lib`. The test fails with the error above. It exercises the bounds decoder directly; the rejection happens before payload decoding. ### Expected behavior Valid Variant bounds must not prevent manifest loading. If their representation or pruning support is not implemented yet, conservatively ignoring these optional bounds would allow reads without making incorrect pruning decisions. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
