Github user abehm commented on a diff in the pull request:
https://github.com/apache/spark/pull/23096#discussion_r235151635
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/rules/RuleExecutor.scala
---
@@ -88,15 +92,18 @@ abstract class RuleExecutor[TreeType <: TreeNode[_]]
extends Logging {
val startTime = System.nanoTime()
val result = rule(plan)
val runTime = System.nanoTime() - startTime
+ val effective = !result.fastEquals(plan)
- if (!result.fastEquals(plan)) {
+ if (effective) {
queryExecutionMetrics.incNumEffectiveExecution(rule.ruleName)
queryExecutionMetrics.incTimeEffectiveExecutionBy(rule.ruleName, runTime)
planChangeLogger.log(rule.ruleName, plan, result)
}
queryExecutionMetrics.incExecutionTimeBy(rule.ruleName,
runTime)
queryExecutionMetrics.incNumExecution(rule.ruleName)
+ tracker.foreach(_.recordRuleInvocation(rule.ruleName, runTime,
effective))
--- End diff --
Doesn't this make the query-local and the global metrics inconsistent when
tracker is None?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
