This is an automated email from the ASF dual-hosted git repository.
uros-b pushed a commit to branch branch-4.3
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/branch-4.3 by this push:
new b037b8cd7c2c [SPARK-58168][SQL] Do no use Array[Nothing] in Flatten
b037b8cd7c2c is described below
commit b037b8cd7c2c517c4f215532a03cbc0029d15fcc
Author: Emil Ejbyfeldt <[email protected]>
AuthorDate: Sun Aug 23 13:40:00 2026 +0000
[SPARK-58168][SQL] Do no use Array[Nothing] in Flatten
### What changes were proposed in this pull request?
### Why are the changes needed?
On Scala3 the erasure of `Array[Nothing]` is no longer `[]Object` but
`[]scala.runtime.Nothing$` which causes class cast errors like
```
arraycopy: element type mismatch: can not cast one of the elements of
java.lang.Object[] to the type of the destination array, scala.runtime.Nothing$
```
when used with a standard library built with Scala3. This is also related
https://github.com/scala/scala3/issues/25896 and making Spark usable on Scala
3.8.0 and later.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Existing tests ensure no regressions. That it makes the code more
compatible with Scala 3.8 was done by running a local build against the tests
in https://github.com/unum-io/tyda (a Scala 3 library that can use Spark as a
backend).
### Was this patch authored or co-authored using generative AI tooling?
No.
Closes #57299 from eejbyfeldt/SPARK-58168.
Authored-by: Emil Ejbyfeldt <[email protected]>
Signed-off-by: Uros Bojanic <[email protected]>
(cherry picked from commit 923e31286927328497859a909ab965d0920e2622)
Signed-off-by: Uros Bojanic <[email protected]>
---
.../apache/spark/sql/catalyst/expressions/collectionOperations.scala | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
index 941772ae157e..492044ff4e38 100644
---
a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
+++
b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala
@@ -3338,7 +3338,7 @@ case class Flatten(child: Expression) extends
UnaryExpression
throw QueryExecutionErrors.arrayFunctionWithElementsExceedLimitError(
prettyName, numberOfElements)
}
- val flattenedData = new Array(numberOfElements.toInt)
+ val flattenedData = new Array[Any](numberOfElements.toInt)
var position = 0
for (ad <- arrayData) {
val arr = ad.toObjectArray(elementType)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
