dbtsai commented on pull request #28577:
URL: https://github.com/apache/spark/pull/28577#issuecomment-631033792
All the tests are passing in Scala 2.12 with JDK8 and JDK11 builds.
I also tested the code in the description, and it works as expected.
- Scala 2.12 with JDK8
```
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.4.5.14-apple-SNAPSHOT
/_/
Using Scala version 2.12.10 (OpenJDK 64-Bit Server VM, Java 1.8.0_252)
Type in expressions to have them evaluated.
Type :help for more information.
scala> :paste
// Entering paste mode (ctrl-D to finish)
class NotSerializableClass(val x: Int)
val ns = new NotSerializableClass(42)
val topLevelValue = "someValue"
val func = (j: Int) => {
(1 to j).flatMap { x =>
(1 to x).map { y => y + topLevelValue }
}
}
// Exiting paste mode, now interpreting.
defined class NotSerializableClass
ns: NotSerializableClass = NotSerializableClass@2769d577
topLevelValue: String = someValue
func: Int => scala.collection.immutable.IndexedSeq[String] =
$Lambda$1751/481549862@25297d52
scala> sc.parallelize(0 to 2).map(func).collect
res0: Array[scala.collection.immutable.IndexedSeq[String]] = Array(Vector(),
Vector(1someValue), Vector(1someValue, 1someValue, 2someValue))
```
- Scala 2.12 with JDK11
```
Welcome to
____ __
/ __/__ ___ _____/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 2.4.5.14-jdk11-apple-SNAPSHOT
/_/
Using Scala version 2.12.10 (OpenJDK 64-Bit Server VM, Java 11.0.7)
Type in expressions to have them evaluated.
Type :help for more information.
scala> :paste
// Entering paste mode (ctrl-D to finish)
class NotSerializableClass(val x: Int)
val ns = new NotSerializableClass(42)
val topLevelValue = "someValue"
val func = (j: Int) => {
(1 to j).flatMap { x =>
(1 to x).map { y => y + topLevelValue }
}
}
// Exiting paste mode, now interpreting.
defined class NotSerializableClass
ns: NotSerializableClass = NotSerializableClass@199f2854
topLevelValue: String = someValue
func: Int => scala.collection.immutable.IndexedSeq[String] =
$Lambda$1852/0x0000000800c2a040@5c9cbc69
scala> sc.parallelize(0 to 2).map(func).collect
res0: Array[scala.collection.immutable.IndexedSeq[String]] = Array(Vector(),
Vector(1someValue), Vector(1someValue, 1someValue, 2someValue))
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
