aokolnychyi commented on a change in pull request #2017:
URL: https://github.com/apache/iceberg/pull/2017#discussion_r551262767
##########
File path:
spark3-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDelete.scala
##########
@@ -66,117 +75,120 @@ object RewriteDelete extends Rule[LogicalPlan] with
PredicateHelper with Logging
private val FILE_NAME_COL = "_file"
private val ROW_POS_COL = "_pos"
- override def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
- // don't rewrite deletes that can be answered by passing filters to
deleteWhere in SupportsDelete
- case d @ DeleteFromTable(r: DataSourceV2Relation, Some(cond)) if
isMetadataDelete(r, cond) =>
- d
+ private case class MergeTable(
+ table: Table with SupportsMerge,
+ operation: String) extends Table with SupportsRead with SupportsWrite {
Review comment:
We should be careful as `MergeTable` won't extend all interfaces the
original table implements. Some of them are optional like
`SupportsMetadataColumns`. Some rules won't be triggered.
##########
File path:
spark3-extensions/src/main/scala/org/apache/spark/sql/catalyst/optimizer/RewriteDelete.scala
##########
@@ -66,117 +75,120 @@ object RewriteDelete extends Rule[LogicalPlan] with
PredicateHelper with Logging
private val FILE_NAME_COL = "_file"
private val ROW_POS_COL = "_pos"
- override def apply(plan: LogicalPlan): LogicalPlan = plan resolveOperators {
- // don't rewrite deletes that can be answered by passing filters to
deleteWhere in SupportsDelete
- case d @ DeleteFromTable(r: DataSourceV2Relation, Some(cond)) if
isMetadataDelete(r, cond) =>
- d
+ private case class MergeTable(
+ table: Table with SupportsMerge,
+ operation: String) extends Table with SupportsRead with SupportsWrite {
Review comment:
We should be careful as `MergeTable` won't extend all interfaces the
original table implements. Some of them are optional like
`SupportsMetadataColumns`. Some rules may not be triggered.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
