Raz Luvaton created SPARK-56946:
-----------------------------------
Summary: Add SortOrigin like ShuffleOrigin for potential
optimization
Key: SPARK-56946
URL: https://issues.apache.org/jira/browse/SPARK-56946
Project: Spark
Issue Type: Improvement
Components: Optimizer
Affects Versions: 4.1.1
Reporter: Raz Luvaton
There is `ShuffleOrigin` to indicate where the shuffle was coming from, this is
used to see if we can optimize away the shuffle if not requested by the user
I want to have the same but for `Sort` since everythink that is being added
automaticlly by `EnsureRequirements` should have some kind of tag so we can
create optimizations that remove those if not needed, like:
- if operator has no `requireChildOrdering` and child is `Sort` and the sort
came from `EnsureRequirements` then we can remove the sort
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
