aokolnychyi commented on a change in pull request #2132:
URL: https://github.com/apache/iceberg/pull/2132#discussion_r562258312
##########
File path:
spark3-extensions/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/DynamicFileFilter.scala
##########
@@ -42,12 +42,11 @@ case class DynamicFileFilter(
}
}
-case class DynamicFileFilterWithCountCheck(
+case class DynamicFileFilterWithCardinalityCheck(
Review comment:
I think `WithCardinalityCheck` is slightly better.
##########
File path:
spark3-extensions/src/main/scala/org/apache/spark/sql/catalyst/utils/RewriteRowLevelOperationHelper.scala
##########
@@ -87,7 +87,8 @@ trait RewriteRowLevelOperationHelper extends PredicateHelper
with Logging {
protected val FILE_NAME_COL = "_file"
protected val ROW_POS_COL = "_pos"
- protected val AFFECTED_FILES_ACC_NAME = "affectedFiles"
+ // `internal.metrics` prefix ensures the accumulator state is not tracked by
Spark UI
+ protected val AFFECTED_FILES_ACC_NAME =
"internal.metrics.merge.affectedFiles"
Review comment:
I guess this one is important.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
