Github user maropu commented on a diff in the pull request:
https://github.com/apache/spark/pull/19811#discussion_r156313853
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala
---
@@ -203,18 +267,35 @@ class CodegenContext {
def declareMutableStates(): String = {
// It's possible that we add same mutable state twice, e.g. the
`mergeExpressions` in
// `TypedAggregateExpression`, we should call `distinct` here to
remove the duplicated ones.
- mutableStates.distinct.map { case (javaType, variableName, _) =>
+ val inlinedStates = mutableStates.distinct.map { case (javaType,
variableName, _) =>
s"private $javaType $variableName;"
- }.mkString("\n")
+ }
+
+ val arrayStates = mutableStateArrayIdx.keys.map { case (javaType,
arrayName) =>
+ val length = mutableStateArrayIdx((javaType, arrayName)) + 1
+ if (javaType.matches("^.*\\[\\]$")) {
+ // initializer had an one-dimensional array variable
+ val baseType = javaType.substring(0, javaType.length - 2)
+ s"private $javaType[] $arrayName = new $baseType[$length][];"
+ } else {
+ // initializer had a scalar variable
+ s"private $javaType[] $arrayName = new $javaType[$length];"
+ }
+ }
+
+ (inlinedStates ++ arrayStates).mkString("\n")
}
def initMutableStates(): String = {
// It's possible that we add same mutable state twice, e.g. the
`mergeExpressions` in
// `TypedAggregateExpression`, we should call `distinct` here to
remove the duplicated ones.
val initCodes = mutableStates.distinct.map(_._3 + "\n")
--- End diff --
(This is not related to this pr thought....) `val initCodes =
mutableStates.map(_._3).distinct.map(_ + "\n")`? The current code cause too
many empty lines in gen'd code when having many mutable states. @ueshin
recently touched this line and WDYT?
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
