Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/12729#discussion_r61995819
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -572,6 +588,65 @@ class CodegenContext {
}
/**
+ * Perform a function which generates a sequence of ExprCodes with a
given mapping between
+ * expressions and common expressions, instead of using the mapping in
current context.
+ */
+ def withSubExprEliminationExprs(
+ newSubExprEliminationExprs: Map[Expression,
SubExprEliminationState])(
+ f: => Seq[ExprCode]): Seq[ExprCode] = {
+ val oldsubExprEliminationExprs = subExprEliminationExprs
+ subExprEliminationExprs.clear
+ newSubExprEliminationExprs.foreach(subExprEliminationExprs += _)
+
+ val genCodes = f
+
+ // Restore previous subExprEliminationExprs
+ subExprEliminationExprs.clear
+ oldsubExprEliminationExprs.foreach(subExprEliminationExprs += _)
+ genCodes
+ }
+
+ /**
+ * Checks and sets up the state and codegen for subexpression
elimination. This finds the
+ * common subexpressions, generates the code snippets that evaluate
those expressions and
+ * populates the mapping of common subexpressions to the generated code
snippets. The generated
+ * code snippets will be returned and should be inserted into generated
codes before these
+ * common subexpressions actually are used first time.
+ */
+ def subexpressionEliminationForWholeStageCodegen(expressions:
Seq[Expression]): SubExprCodes = {
+ // Create a clear EquivalentExpressions and SubExprEliminationState
mapping
+ val equivalentExpressions: EquivalentExpressions = new
EquivalentExpressions
+ val subExprEliminationExprs = mutable.HashMap.empty[Expression,
SubExprEliminationState]
+
+ // Add each expression tree and compute the common subexpressions.
+ expressions.foreach(equivalentExpressions.addExprTree(_, true, false))
+
+ // Get all the expressions that appear at least twice and set up the
state for subexpression
+ // elimination.
+ val commonExprs =
equivalentExpressions.getAllEquivalentExprs.filter(_.size > 1)
+ val codes = commonExprs.map { e =>
+ val expr = e.head
+ val fnName = freshName("evalExpr")
+ val isNull = s"${fnName}IsNull"
+ val value = s"${fnName}Value"
+
+ // Generate the code for this expression tree.
+ val code = expr.genCode(this)
+ val effectiveCode =
+ s"""
+ | ${code.code.trim}
+ | boolean $isNull = ${code.isNull};
--- End diff --
yes. Can save two variables. Thanks.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
