Bo Zhang created SPARK-55035:
--------------------------------
Summary: Eager shuffle dependency cleanup should be performed on
child executions
Key: SPARK-55035
URL: https://issues.apache.org/jira/browse/SPARK-55035
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.2.0
Reporter: Bo Zhang
We implemented eager shuffle cleanup in SPARK-47764. However, in queries with
multiple layers of executions, shuffle dependencies for only the root execution
is cleaned up.
We should fix this and cleanup the shuffle dependencies for the child
executions, based on the shuffle cleanup mode of the root execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
