kosiew commented on issue #17557: URL: https://github.com/apache/datafusion/issues/17557#issuecomment-3414111562
Did more digging: https://github.com/apache/datafusion/blob/9bfa2ae/datafusion/common/src/scalar/mod.rs#L515-L528 DataFusion defers floating-point ordering for ScalarValue to Rust’s total_cmp. That order places negative-signed NaNs at the very bottom, finite values in the middle, and positive-signed NaNs at the very top. NaN > any non-NaN? Yes for the canonical (positive-signed) NaN: it sorts after all finite numbers, so > and < treat it as greater than any numeric value (including infinities). You can see this in the lexicographic sort test where NaNs appear after the finite values once nulls are handled. https://github.com/apache/datafusion/blob/9bfa2ae/datafusion/physical-plan/src/sorts/sort.rs#L1935-L2035 -NaN < negative non-NaN? Yes. Because negative-signed NaNs come even before negative infinity in the total order, they compare as less than every finite negative value. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
