0lai0 commented on PR #4653: URL: https://github.com/apache/datafusion-comet/pull/4653#issuecomment-5898231525
Thanks @andygrove and @sunchao. Updated: - **Filter pushdown path:** there's no clean fix on the Comet side. DataFusion 54/55 stringifies the predicate error in `row_filter.rs` before it reaches the JNI bridge, so recovering it would mean parsing the `Debug` string or disabling pushdown for these predicates. It's fixed upstream in apache/datafusion#24638 (not released yet), so I've kept that part of the #4514 workaround, narrowed to errors containing `Error evaluating filter predicate`, with a note to remove it after the upgrade. Happy to open a tracking issue if you prefer. - **Duplicated walker / `Shared`:** replaced both walkers with one `error_chain` helper that follows `Error::source()`. It covers `Shared`, `Diagnostic`, and the Arrow/Parquet wrappers too. Added Rust unit tests for each. After merging main, `parquet_external_spark_error` is subsumed by it (its test now goes through `extract_spark_payload`), and `typed_jvm_exception` uses it too. - **Second error class:** added an `ARITHMETIC_OVERFLOW` test to `CometCodegenSuite`. - **Diffs:** regenerated from Spark checkouts rather than hand-edited, for all five versions including 4.2.0. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
