FelixYBW commented on PR #6707: URL: https://github.com/apache/incubator-gluten/pull/6707#issuecomment-2272402260
> ## What changes were proposed in this pull request? > > Gluten Result: Round(0.19324999999999998, 4) = 0.1933 > > Vanilla Spark Result: Round(0.19324999999999998, 4) = 0.1932 > > * Currently gluten round has std::nextafter to offset the small error coming from multiply result. However, for high precision numbers the nextafter leads to a bigger delta than the expected value. > * This eventually causes wrong results for high precision rounding (starts occurring beyond 15 decimals). > * Currently we use double in the best case for intermediate calculations. Now moved to long double for holding the intermediate results more accurately. > * In the future, we can use boost multi-precision to support arbitrary precision at runtime, which will more closely match the Java BigDecimal and give better performance. > > Also fixes: #5366. > > ## How was this patch tested? > * Unit Tests > * Local Spark shell > > After the fix: Gluten Result: Round(0.19324999999999998, 4) = 0.1932 Vanilla Spark Result: Round(0.19324999999999998, 4) = 0.1932 > > The following query was also tested for the attached issue: (This can be flaky since we still use double which will lose some precision) `select round(avg(cast(col as double)), 4) as topic_2 from VALUES (0.188), (0.194), (0.194), (0.194), (0.194), (0.194), (0.194), (0.194) AS tab(col);` Excellent description! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
