zml1206 commented on code in PR #12477:
URL: https://github.com/apache/gluten/pull/12477#discussion_r3548052437
##########
gluten-core/src/main/scala/org/apache/gluten/extension/caller/CallerInfo.scala:
##########
@@ -79,10 +79,14 @@ object CallerInfo {
}
private def inBloomFilterStatFunctionCall(stack: Seq[StackTraceElement]):
Boolean = {
- val res = stack.exists(
- _.getClassName.equals("org.apache.spark.sql.DataFrameStatFunctions")
- && stack.exists(_.getMethodName.equals("bloomFilter")))
- res
+ // Two independent existence checks over the stack. The previous shape
used a nested
+ // `stack.exists(_.getMethodName.equals(...))` inside the outer predicate,
which re-walked
+ // the whole stack for every candidate frame - O(n^2) in stack depth.
Splitting the checks
+ // makes each a single pass and preserves the "same stack contains both
frames" semantics.
+ val hasStatFunctionsClass =
+
stack.exists(_.getClassName.equals("org.apache.spark.sql.DataFrameStatFunctions"))
+ val hasBloomFilterMethod =
stack.exists(_.getMethodName.equals("bloomFilter"))
+ hasStatFunctionsClass && hasBloomFilterMethod
Review Comment:
hasBloomFilterMethod is eagerly evaluated even when hasStatFunctionsClass is
false. Could we rely on && short-circuiting here?
```
val hasStatFunctionsClass =
stack.exists(_.getClassName.equals("org.apache.spark.sql.DataFrameStatFunctions"))
hasStatFunctionsClass && stack.exists(_.getMethodName.equals("bloomFilter"))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
