Github user gatorsmile commented on a diff in the pull request:
https://github.com/apache/spark/pull/20402#discussion_r164159613
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/QueryExecution.scala ---
@@ -66,7 +54,16 @@ class QueryExecution(val sparkSession: SparkSession, val
logical: LogicalPlan) {
lazy val analyzed: LogicalPlan = {
SparkSession.setActiveSession(sparkSession)
- sparkSession.sessionState.analyzer.execute(logical)
+ val plan = sparkSession.sessionState.analyzer.execute(logical)
+ try {
+ sparkSession.sessionState.analyzer.checkAnalysis(plan)
+ EliminateBarriers(plan)
--- End diff --
In the future, we can re-visit all the rules we put in the optimizer
`Finish Analysis` batch. It might make sense to introduce a dedicate batch
here.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
