sdf-jkl opened a new issue, #3705:
URL: https://github.com/apache/iceberg-rust/issues/3705

   ### Apache Iceberg Rust version
   
   Upstream `main` at `4be78085219af9f1c2093bfcea66ff3d847c7613`; locked 
Arrow/Parquet 60.1.0.
   
   ### Describe the bug
   
   A manifest containing a `lower_bounds` or `upper_bounds` entry for a Variant 
field fails to load with:
   
   ```text
   DataInvalid => field v is not a primitive type
   ```
   
   
[`parse_bytes_entry`](https://github.com/apache/iceberg-rust/blob/4be78085219af9f1c2093bfcea66ff3d847c7613/crates/iceberg/src/spec/manifest/_serde.rs#L306-L325)
 requires every bound's field type to be primitive. `Type::Variant` fails that 
check before the bound bytes are decoded. Both bounds maps pass through this 
function when decoding a data file, so this blocks manifest loading even when 
the scan would not select the Variant column.
   
   The [Iceberg 
specification](https://iceberg.apache.org/spec/#bounds-for-variant) permits 
bounds under the Variant column's field ID. These encode per-path bounds as 
Variant objects; they are not ordinary binary min/max values.
   
   This is the manifest Variant-bounds lead listed as unfiled in #2411. 
Related: Variant umbrella #3685. Writing or using Variant statistics for 
pruning is separate from accepting manifests that contain them.
   
   ### To Reproduce
   
   Add this test to the existing `tests` module in 
`crates/iceberg/src/spec/manifest/_serde.rs`:
   
   ```rust
   #[test]
   fn review_variant_bounds_do_not_block_manifest_loading() {
       use super::{BytesEntry, parse_bytes_entry};
       let schema = Schema::builder().with_fields(vec![
           NestedField::optional(1, "v",
               Type::Variant(crate::spec::VariantType)).into(),
       ]).build().unwrap();
       let entry = BytesEntry {
           key: 1,
           value: serde_bytes::ByteBuf::from(vec![1, 0, 0, 2, 0, 0]),
       };
       let result = parse_bytes_entry(vec![entry], &schema);
       assert!(result.is_ok(), "Variant bounds must be read or ignored: 
{result:?}");
   }
   ```
   
   Run `cargo test -p iceberg 
review_variant_bounds_do_not_block_manifest_loading --lib`.
   
   The test fails with the error above. It exercises the bounds decoder 
directly; the rejection happens before payload decoding.
   
   ### Expected behavior
   
   Valid Variant bounds must not prevent manifest loading. If their 
representation or pruning support is not implemented yet, conservatively 
ignoring these optional bounds would allow reads without making incorrect 
pruning decisions.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to