RussellSpitzer commented on a change in pull request #2460:
URL: https://github.com/apache/iceberg/pull/2460#discussion_r612561568
##########
File path:
spark3-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteUpdate.scala
##########
@@ -59,7 +62,7 @@ case class RewriteUpdate(spark: SparkSession) extends
Rule[LogicalPlan] with Rew
// so the first job uses DynamicFileFilter and the second one uses the
underlying scan plan
// both jobs share the same SparkMergeScan instance to ensure they
operate on same files
val matchingRowsPlanBuilder = scanRelation => Filter(cond, scanRelation)
- val scanPlan = buildDynamicFilterScanPlan(spark, r.table, r.output,
mergeBuilder, cond, matchingRowsPlanBuilder)
+ val scanPlan = buildDynamicFilterScanPlan(spark, r, r.output,
mergeBuilder, cond, matchingRowsPlanBuilder)
Review comment:
here I think we would be ok with using reflection to determine the
buildDynamicFileterScanPlan to use, we would look to see which method signature
is valid and use that. Since the return type is the same I think we are more
less safe if we do that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
