Github user srowen commented on a diff in the pull request:
https://github.com/apache/spark/pull/11274#discussion_r207080008
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegen.scala
---
@@ -114,27 +117,71 @@ trait CodegenSupport extends SparkPlan {
}
/**
- * Consume the columns generated from it's child, call doConsume() or
emit the rows.
+ * Returns source code to evaluate all the variables, and clear the
code of them, to prevent
+ * them to be evaluated twice.
+ */
+ protected def evaluateVariables(variables: Seq[ExprCode]): String = {
+ val evaluate = variables.filter(_.code !=
"").map(_.code.trim).mkString("\n")
+ variables.foreach(_.code = "")
+ evaluate
+ }
+
+ /**
+ * Returns source code to evaluate the variables for required
attributes, and clear the code
+ * of evaluated variables, to prevent them to be evaluated twice..
*/
+ protected def evaluateRequiredVariables(
+ attributes: Seq[Attribute],
+ variables: Seq[ExprCode],
+ required: AttributeSet): String = {
+ var evaluateVars = ""
+ variables.zipWithIndex.foreach { case (ev, i) =>
+ if (ev.code != "" && required.contains(attributes(i))) {
--- End diff --
@davies I was just reviewing build warnings, and it flags this line.
`ev.code` is a `Block` rather than `String`. Should it be `ev.code.nonEmpty &&
...` instead?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
