cloud-fan commented on code in PR #46143:
URL: https://github.com/apache/spark/pull/46143#discussion_r2525685446
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala:
##########
@@ -404,6 +405,16 @@ abstract class Expression extends TreeNode[Expression] {
} else {
""
}
+
+ /**
+ * A developer API for marking if an expression is likely to be expensive.
+ * The current only consumer of this is the pushdown optimizer.
+ * By default an expression is expensive if any of it's children are
expensive.
+ */
+ @DeveloperApi
Review Comment:
We never add such annotation to catalyst plans/expressions, as they are very
internal and we guarantee no binary compatibility.
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala:
##########
@@ -404,6 +405,16 @@ abstract class Expression extends TreeNode[Expression] {
} else {
""
}
+
+ /**
+ * A developer API for marking if an expression is likely to be expensive.
+ * The current only consumer of this is the pushdown optimizer.
+ * By default an expression is expensive if any of it's children are
expensive.
+ */
+ @DeveloperApi
+ def expensive: Boolean = _expensive
+
+ private lazy val _expensive: Boolean = children.map(_.expensive).exists(_ ==
true)
Review Comment:
```suggestion
private lazy val hasExpensiveChild: Boolean = children.exists(_.expensive)
```
##########
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala:
##########
@@ -650,6 +650,7 @@ case class StringToMap(text: Expression, pairDelim:
Expression, keyValueDelim: E
pairDelim = newSecond,
keyValueDelim = newThird
)
+
Review Comment:
unnecessary change
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
