manuzhang opened a new issue, #18949: URL: https://github.com/apache/iceberg/issues/18949
### Feature Request / Improvement Add permanent Spark 4.2 coverage for a static `PartitionPredicate` whose evaluation throws while planning an Iceberg scan. [SPARK-59572](https://github.com/apache/spark/pull/58851) fixes failure being reported as a partition match after the predicate has been fully pushed. Existing successful UDF pushdown tests do not verify this failure contract. #18294 contains `partitionPredicateEvaluationFailureIsPropagated`, but it is absent from [main's TestFilterPushDown](https://github.com/apache/iceberg/blob/234d55774d3c7355f92c8e93f8e117b70726e556/spark/v4.2/spark/src/test/java/org/apache/iceberg/spark/sql/TestFilterPushDown.java). Preserve the test independently of the candidate-version draft. Acceptance criteria: - [ ] Create an Iceberg table partitioned by a string field with both a valid numeric value and a value such as `hr`. - [ ] Register a parsing UDF and filter on its result, forcing the second-pass partition predicate to evaluate during optimized-plan construction. - [ ] Assert a `SparkException` with the stable `FAILED_EXECUTE_UDF` condition and a `NumberFormatException` root cause. - [ ] Exercise LOCAL and DISTRIBUTED planning through the containing suite. [Starting test at the draft head](https://github.com/apache/iceberg/blob/35711c6a0be10942e55ea5b824c71bb7b87e9ff6/spark/v4.2/spark/src/test/java/org/apache/iceberg/spark/sql/TestFilterPushDown.java). Current main already uses Spark 4.2.1; a staging repository is unnecessary. Related feature tracker: #17959. ### Query engine Spark -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
