chenhao-db commented on PR #45730: URL: https://github.com/apache/spark/pull/45730#issuecomment-2026676267
Things get a bit more complex after #45549. > For example, map type is not orderable, but can be used as grouping keys. It can only be used as grouping keys after #45549. However, with the map normalization and comparison code introduced in this PR, the map type can also be orderable. > what is the behavior today? Before #45549, the SQL snippet in my PR description would fail with an `INTERNAL_ERROR`. After #45549, it would be successfully executed. When a column is used as a window partition key, it is translated into a local `Sort` over a `HashPartitioning`. Although the current type check code of `SortOrder` rejects map type, it doesn't run when the `SortOrder` is constructed from a window partition key. Then, the `Sort` node will successfully compare map inputs, because #45549 supports map comparison. There are still some issues with sorting maps, because the map normalization code is only inserted into aggregate, but not sort. But I think they are fixable, and the map type is indeed orderable. However, if a type is not orderable, it cannot be a window partition key, so the code change in this PR is still necessary. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
