jiangjiangtian commented on PR #6707: URL: https://github.com/apache/incubator-gluten/pull/6707#issuecomment-2287728672
> > > @rui-mo I found an issue, but I can't tell whether the issue is from round. > > > ``` > > > SELECT round(3/32*CAST(5.92 AS DECIMAL(20,2)),2); > > > ``` > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > gluten returns 0.56, but vanilla returns 0.55. The SQLs below have the same results between gluten and vanilla, which are 0.5549999999999999 and 0.55: > > > ``` > > > SELECT 3/32*CAST(5.92 AS DECIMAL(20,2)); > > > SELECT round(0.5549999999999999, 2); > > > ``` > > > > > > I find that the argument passed to round is 0.555. I will check it. > > I think there will always be an issue with decimals because of how c++ represents floating point, there is a negligible loss when representing high precision using `double`. Java's `bigdecimal` does not pose this and can scale upto whatever the memory will support with dynamic precision. As long as we continue to use `double` in c++ as an alternative to `bigdecimal`, we will see issues with arithmetic. One of the long term fixes could be using mpfr/boost throughout the code but it might require significant effort. Thanks! Thanks for your reply! I have a minimal example that doesn't produce the right answer: ``` SELECT round(cast(0.5549999999999999 as double), 2); ``` gluten returns 0.56, vanilla returns 0.55. Besides, I run the following SQL and I get 0.1933: ``` SELECT round(cast(0.19324999999999998 AS DOUBLE), 4); ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
