This is an automated email from the ASF dual-hosted git repository.
gurwls223 pushed a commit to branch branch-3.2
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-3.2 by this push:
new 1621575 [SPARK-36789][SQL] Use the correct constant type as the null
value holder in array functions
1621575 is described below
commit 16215755b7312b9e28a82f10e08d6a23bf255695
Author: Wenchen Fan <[email protected]>
AuthorDate: Fri Sep 17 16:49:54 2021 +0900
[SPARK-36789][SQL] Use the correct constant type as the null value holder
in array functions
### What changes were proposed in this pull request?
In array functions, we use constant 0 as the placeholder when adding a null
value to an array buffer. This PR makes sure the constant 0 matches the type of
the array element.
### Why are the changes needed?
Fix a potential bug. Somehow we can hit this bug sometimes after
https://github.com/apache/spark/pull/33955 .
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
existing tests
Closes #34029 from cloud-fan/minor.
Authored-by: Wenchen Fan <[email protected]>
Signed-off-by: Hyukjin Kwon <[email protected]>
(cherry picked from commit 4145498826f288e610b00033d9fc2063fd1acc9f)
Signed-off-by: Hyukjin Kwon <[email protected]>
---
.../apache/spark/sql/catalyst/expressions/collectionOperations.scala | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
index 73a45d7..ba000a3 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
@@ -3336,7 +3336,9 @@ trait ArraySetLike {
@transient protected lazy val nullValueHolder = et match {
case ByteType => "(byte) 0"
case ShortType => "(short) 0"
- case LongType => "(long) 0"
+ case LongType => "0L"
+ case FloatType => "0.0f"
+ case DoubleType => "0.0"
case _ => "0"
}
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
