dwsmith1983 opened a new issue, #7672: URL: https://github.com/apache/datafusion-comet/issues/7672
### Describe the bug `checkSparkAnswerWithTolerance` in `CometTestBase.scala` skips the comparison of a float or double pair whenever either side is NaN. It only compares values when `!actual.isNaN && !expected.isNaN`, so a NaN from Comet against a number from Spark, or the other way round, passes. A SQL file test with `query tolerance=...` uses this check, so it cannot catch a kernel that returns NaN where Spark does not. ### Steps to reproduce A `query tolerance=1e-6` over `ln(x)` where Comet returns NaN on some rows and Spark returns a value passes. The plain `query` form of the same test fails with "Results do not match". ### Expected behavior The pair should match only when both sides are NaN, or both are numbers within the tolerance: `if (actual.isNaN || expected.isNaN) assert(actual.isNaN && expected.isNaN)`. ### Additional context Found while reproducing the wrong `ln` results in #7671. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
