ShayanGho commented on PR #26020: URL: https://github.com/apache/datafusion/pull/26020#issuecomment-5976990922
A note on the approach. The first idea was to normalize the partition columns right before `evaluate_partition_ranges`. That merges the two zeros into one partition, but the `SortExec` under the window has already ordered `-0.0` before `0.0`. So with `PARTITION BY a ORDER BY b` the merged partition is not in `b` order, and with `PARTITION BY a, b` equal keys are not adjacent. I also considered rewriting the key in the planner as `CASE WHEN a = 0 THEN 0 ELSE a END`. It fixes plain floats without a new expression, but it does not cover nested floats and it reports no ordering, so an input already sorted on the key gets sorted again. Wrapping the key in an expression that calls `normalize_float_zero` makes the sort, the repartition and the boundary detection see the same values, covers nested types, and keeps an existing ordering on a single float key. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
