snmvaughan commented on issue #6966:
URL:
https://github.com/apache/datafusion-comet/issues/6966#issuecomment-6669265107
Two details from the #7522 review, found by @comphead with a small Java
program and by reading `Decimal.set` and `changePrecision` at v4.1.3:
- The slowdown starts well before the exception. On a release without
SPARK-60119, `new BigDecimal("1e-50000000").setScale(2, HALF_UP)` returns
`0.00` after about 12 seconds on JDK 17, while `1e-600000000` throws
`BigInteger would overflow supported range`.
- The int boundary is off by one. `'1e-2147483647'` cast to `DECIMAL(10,-1)`
throws `Underflow` as well, because `setScale` needs the difference
(2147483648) to fit in an int. `parse_string_to_decimal` computes a
`scale_adjustment` of -2147483648, which fits in an `i32`, so the native cast
returns 0.
Both gaps close on Spark 4.0.5, 4.1.4 and 4.2.1, where SPARK-60119 makes
these cases return 0. #7187 and #7522 cover the negative-scale difference that
overflows an int.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
