anjalitripathibiz-byte opened a new issue, #3716:
URL: https://github.com/apache/iceberg-rust/issues/3716

   ### Apache Iceberg Rust version
   
   0.10.1 (the code is unchanged on main at 14e0919489)
   
   ### Describe the bug
   
   A row filter that contains an `IN` list whose literal `get_arrow_datum` 
cannot convert panics the reader thread, instead of returning an error. The 
comparison visitors (`less_than`, `eq`, ...) propagate that error with `?`. 
`r#in` and `not_in` call `.unwrap()` instead:
   
   ```rust
   // crates/iceberg/src/arrow/reader/predicate_visitor.rs, fn r#in (and fn 
not_in)
   let literals: Vec<_> = literals
       .iter()
       .map(|lit| get_arrow_datum(lit).unwrap())
       .collect();
   ```
   
   I hit it from SQL through iceberg-datafusion 0.10.1, with a user-supplied 
literal outside the column's 32-bit range:
   
   ```sql
   -- c1 is an Iceberg `int` column
   SELECT c2 FROM t2 WHERE c1 IN (3, -9223372036854775808);
   ```
   
   ```
   thread '...' panicked at 
.../iceberg-0.10.1/src/arrow/reader/predicate_visitor.rs:586:
   called `Result::unwrap()` on an `Err` value: FeatureUnsupported => 
Converting datum from type Int to arrow not supported yet
   ```
   
   The panic was observed on 0.10.1. On main I only checked that the two 
`.unwrap()` calls are still there (lines 590 and 620); I did not re-run the 
query against main.
   
   ### To Reproduce
   
   An `int` column and a scan whose row filter is 
`Reference::new("c1").is_in([Datum::int(3), <a datum that binds to type Int 
with a Long literal>])`. Through DataFusion, the SQL above against an 
`IcebergStaticTableProvider` gets there.
   
   ### Expected behavior
   
   Either:
   
   1. return the `FeatureUnsupported` error instead of panicking (PR #3448 
changes these two lines to `?` as part of the time-literal fix for #3446), or, 
better,
   2. not produce an out-of-range literal for an `int` column at all. A literal 
outside `i32` can never equal an `int` value, so for `IN` it can be dropped 
from the set, and for `NOT IN` it is always true. That is how 
`AboveMax`/`BelowMin` are already handled for comparisons.
   
   With (1) alone, the query above fails rather than returning the rows where 
`c1 = 3`.
   
   Found by a differential SQL fuzzer comparing DataFusion over Iceberg with 
DuckDB.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to