sunchao opened a new pull request, #7716: URL: https://github.com/apache/datafusion-comet/pull/7716
## Which issue does this PR close? Closes #7581. ## Rationale for this change The native cast built an `i128` from every digit before rounding. For `'1.'` followed by 50 zeros, building that intermediate requires `10^50`, even though `DECIMAL(10,2)` only needs the unscaled integer `100`. Comet returned NULL or raised `CAST_INVALID_INPUT` under ANSI. Spark returns `1.00`. ## What changes are included in this PR? The parser validates the entire input and applies Spark's integral-digit check from #7578. It then uses the exponent and target scale to select the digits to accumulate from the integral and fractional slices. The first discarded digit determines whether to increment the magnitude. Applying the sign last gives `HALF_UP` for both signs, and checking precision after rounding catches a carry into an extra digit. With `spark.sql.legacy.allowNegativeScaleOfDecimal=true`, the 40-digit string for `10^39` now yields `1.0000E+39` at `DECIMAL(10,-35)`. At `DECIMAL(10,2)` it raises `NUMERIC_VALUE_OUT_OF_RANGE` under ANSI. With the setting off, Spark's integral-digit check rejects it first with `NUMERIC_OUT_OF_SUPPORTED_RANGE`. The compatibility guide and audit describe the new behavior. The ANSI error for a scale increase greater than 38 is tracked separately in #7576. The test cleanup in #7605 touches some of the existing assertions updated here. ## How are these changes tested? - The Parquet-backed SQL fixture compares `CAST` and `TRY_CAST` on both long fractions from #7581 and checks the ANSI error for the 40-digit integer with the legacy setting on and off. - `CometNativeCastSuite` compares the negative-scale result with Spark through the DataFrame API, which can express `DecimalType(10, -35)`. - Rust tests exercise exponent shifts, rounding on either side of the decimal point, negative ties, a carry past precision 38, and malformed trailing text. The cast tests check error handling at both native decimal widths. Checked Rust formatting and `git diff --check` locally. Runtime validation is delegated to PR CI. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
