rdblue commented on a change in pull request #30806:
URL: https://github.com/apache/spark/pull/30806#discussion_r544466723
##########
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/SparkOptimizer.scala
##########
@@ -39,6 +39,9 @@ class SparkOptimizer(
// TODO: move SchemaPruning into catalyst
SchemaPruning :: V2ScanRelationPushDown :: PruneFileSourcePartitions :: Nil
+ override def dataSourceRewriteRules: Seq[Rule[LogicalPlan]] =
+ V2Writes :: Nil
Review comment:
I would probably put this in the early pushdown batch, even though the
name doesn't match. The rewrite batch needs to run before this so that writes
created by it run `V2Writes` afterward. That's the same reason why early
pushdown runs after plan rewrites.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
