Kimahriman commented on a change in pull request #32559:
URL: https://github.com/apache/spark/pull/32559#discussion_r636836171
##########
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala
##########
@@ -83,20 +83,35 @@ class EquivalentExpressions {
* Adds only expressions which are common in each of given expressions, in a
recursive way.
* For example, given two expressions `(a + (b + (c + 1)))` and `(d + (e +
(c + 1)))`,
* the common expression `(c + 1)` will be added into `equivalenceMap`.
+ *
+ * Note that as we don't know in advance if any child node of an expression
will be common
+ * across all given expressions, we count all child nodes when looking
through the given
+ * expressions. But when we call `addExprTree` to add common expressions
into the map, we
+ * will add recursively the child nodes. So we need to filter the child
expressions first.
+ * For example, if `((a + b) + c)` and `(a + b)` are common expressions, we
only add
+ * `((a + b) + c)`.
*/
private def addCommonExprs(
exprs: Seq[Expression],
addFunc: Expression => Boolean = addExpr): Unit = {
val exprSetForAll = mutable.Set[Expr]()
Review comment:
I don't mean add it directly to that here. I'm just thinking of a really
stupid example, `when((col + 1) > 0, col + 1).otherwise(1 + col)`. Wouldn't
`col + 1` and `1 + col` resolve as a common expression because they're
semantically equal, but only `col + 1` is added to `equivalenceMap`, so during
codegen `1 + col` wouldn't be resolved to the subexpression?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
