peterxcli commented on PR #5169: URL: https://github.com/apache/datafusion-comet/pull/5169#issuecomment-5155139951
> 1: The fused rescale path is still a raw Arrow error Changed `DecimalRescaleCheckOverflow` to raise `SparkError::NumericValueOutOfRange`, unwrap Arrow external errors, and propagate query context. Large scale deltas now preserve zero/null behavior and return typed ANSI errors. Added a note that current Spark 3.4–4.2 serialization does not reach this fusion. > 2: The reported value diverges from Spark for wide-decimal overflow Confirmed the message-value difference and documented it at the Rust call site and in the Scala test. The test intentionally compares error class, SQLSTATE, and query context rather than the differing value parameter. Opened https://github.com/apache/datafusion-comet/issues/5211 with the example query and corresponding Spark/Comet messages. Linked it from the Rust call site and Scala regression-test comment. > 3: Query context is looked up and then dropped where it originates Wide-decimal and decimal-division expressions are now wrapped directly when query context exists. The ancestor wrapper remains as a fallback for generated child protos without an `expr_id`, but context-less and duplicate wrappers are avoided. > 4: Shared error-unwrap helper Added a shared Arrow external-error unwrapping helper and reused it in decimal division, wide-decimal arithmetic, and fused decimal rescaling. > 5: `format_decimal_str` Removed Comet’s copied formatter and reused Arrow’s public `format_decimal_str`. Added comments explaining why the actual digit count is passed for already-overflowing values. > 6: Tests Added coverage for negative decimal casts, scale-up overflow, null and valid rows before the offending value, rounded multiplication overflow, fused array/scalar overflow, and large scale deltas. The `SparkArithmeticException` class-name match remains because that type is inaccessible outside `org.apache.spark` in Spark 3.4. Added a comment documenting this cross-version constraint. > 7: `Cast::evaluate` Added a comment explaining that child errors deliberately inherit the outer `Cast` query context because `CometIntegralDivide` creates its inner `CheckOverflow` without an expression ID. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
