Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/19480#discussion_r144287183
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -798,10 +830,35 @@ class CodegenContext {
| ${makeSplitFunction(body)}
|}
""".stripMargin
- addNewFunction(name, code)
+ addNewFunctionInternal(name, code, inlineToOuterClass = false)
}
- foldFunctions(functions.map(name =>
s"$name(${arguments.map(_._2).mkString(", ")})"))
+ val outerClassFunctions = functions
+ .filter(_.subclassName.isEmpty)
+ .map(_.functionName)
+
+ val innerClassFunctions = functions
+ .filter(_.subclassName.isDefined)
+ .foldLeft(Map.empty[(String, String), Seq[String]]) { case (acc,
f) =>
+ val key = (f.subclassName.get, f.subclassInstance.get)
+ acc.updated(key, acc.getOrElse(key, Seq.empty[String]) ++
Seq(f.functionName))
+ }
+ .map { case ((subclassName, subclassInstance), subclassFunctions)
=>
--- End diff --
Yeah, I think it can only happen at the first or last sub-class functions.
Seems the functions might be only included in two sub-classes. Most of
functions will be wrapped in one function call.
I'm not sure the proper threshold for this, maybe 5?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
