pgandhi999 commented on a change in pull request #24149: [SPARK-27207] : Ensure
aggregate buffers are initialized again for So…
URL: https://github.com/apache/spark/pull/24149#discussion_r268288825
##########
File path:
sql/core/src/test/scala/org/apache/spark/sql/TypedImperativeAggregateSuite.scala
##########
@@ -299,5 +319,86 @@ object TypedImperativeAggregateSuite {
}
}
+ /**
+ * Calculate the max value with object aggregation buffer. This stores class
MaxValue
+ * in aggregation buffer.
+ */
+ private case class TypedMax2(
+ child: Expression,
+ nullable: Boolean = false,
+ mutableAggBufferOffset: Int = 0,
+ inputAggBufferOffset: Int = 0)
+ extends TypedImperativeAggregate[MaxValue] with ImplicitCastInputTypes {
+
+
+ var maxValueBuffer: MaxValue = null
+ override def createAggregationBuffer(): MaxValue = {
+ // Returns Int.MinValue if all inputs are null
+ maxValueBuffer = new MaxValue(Int.MinValue)
Review comment:
@cloud-fan I am still looking more into it, but for some reason, calling
merge() without invoking initialize() does not cause any visible exception on
normal UDAF functions, but it fails with a Null Pointer Exception when I test
it with the test case described in SPARK-24935(PR #24144 ). My guess is that
for the above test case, since, two different aggregation buffer instances are
created(SketchState and UnionState), the exception shows up. Will investigate
more on it and get back to you soon. Thank you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
