Cheng Su created SPARK-34602:
--------------------------------
Summary: Improve PlanStabilitySuite to output # of operators
information
Key: SPARK-34602
URL: https://issues.apache.org/jira/browse/SPARK-34602
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 3.2.0
Reporter: Cheng Su
Based on discussion from
[https://github.com/apache/spark/pull/31708#issuecomment-789444406,] plan check
is really good to catch plan difference now per PR. However, it's quite hard to
debug given the physical query plan can be very long and verbose for
complicated query. It would be good if we can print out the information for
important operators, e.g. # of shuffles, sub-queries, reused shuffle, join,
aggregation, etc.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
