ArnavBalyan commented on PR #6707: URL: https://github.com/apache/incubator-gluten/pull/6707#issuecomment-2285340553
> > @rui-mo I found an issue, but I can't tell whether the issue is from round. > > ``` > > SELECT round(3/32*CAST(5.92 AS DECIMAL(20,2)),2); > > ``` > > > > > > > > > > > > > > > > > > > > > > > > gluten returns 0.56, but vanilla returns 0.55. The SQLs below have the same results between gluten and vanilla, which are 0.5549999999999999 and 0.55: > > ``` > > SELECT 3/32*CAST(5.92 AS DECIMAL(20,2)); > > SELECT round(0.5549999999999999, 2); > > ``` > > I find that the argument passed to round is 0.555. I will check it. I think there will always be an issue with decimals because of how c++ represents floating point, there is a negligible loss when representing high precision with double. Java's bigdecimal does not pose this and can scale upto whatever the memory will support with dynamic precision. As long as we continue to use double in c++ as an alternative to bigdecimal, we will see issues with arithmetic. One of the long term fixes could be using mpfr/boost throughout the code but it might require significant effort. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
