sunchao opened a new pull request, #7716:
URL: https://github.com/apache/datafusion-comet/pull/7716

   ## Which issue does this PR close?
   
   Closes #7581.
   
   ## Rationale for this change
   
   The native cast built an `i128` from every digit before rounding. For `'1.'` 
followed by 50 zeros, building that intermediate requires `10^50`, even though 
`DECIMAL(10,2)` only needs the unscaled integer `100`. Comet returned NULL or 
raised `CAST_INVALID_INPUT` under ANSI. Spark returns `1.00`.
   
   ## What changes are included in this PR?
   
   The parser validates the entire input and applies Spark's integral-digit 
check from #7578. It then uses the exponent and target scale to select the 
digits to accumulate from the integral and fractional slices. The first 
discarded digit determines whether to increment the magnitude. Applying the 
sign last gives `HALF_UP` for both signs, and checking precision after rounding 
catches a carry into an extra digit.
   
   With `spark.sql.legacy.allowNegativeScaleOfDecimal=true`, the 40-digit 
string for `10^39` now yields `1.0000E+39` at `DECIMAL(10,-35)`. At 
`DECIMAL(10,2)` it raises `NUMERIC_VALUE_OUT_OF_RANGE` under ANSI. With the 
setting off, Spark's integral-digit check rejects it first with 
`NUMERIC_OUT_OF_SUPPORTED_RANGE`.
   
   The compatibility guide and audit describe the new behavior. The ANSI error 
for a scale increase greater than 38 is tracked separately in #7576. The test 
cleanup in #7605 touches some of the existing assertions updated here.
   
   ## How are these changes tested?
   
   - The Parquet-backed SQL fixture compares `CAST` and `TRY_CAST` on both long 
fractions from #7581 and checks the ANSI error for the 40-digit integer with 
the legacy setting on and off.
   - `CometNativeCastSuite` compares the negative-scale result with Spark 
through the DataFrame API, which can express `DecimalType(10, -35)`.
   - Rust tests exercise exponent shifts, rounding on either side of the 
decimal point, negative ties, a carry past precision 38, and malformed trailing 
text. The cast tests check error handling at both native decimal widths.
   
   Checked Rust formatting and `git diff --check` locally. Runtime validation 
is delegated to PR CI.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to