This is an automated email from the ASF dual-hosted git repository.
yao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new baf461b5cb44 [SPARK-48708][CORE] Remove three unnecessary type
registrations from `KryoSerializer`
baf461b5cb44 is described below
commit baf461b5cb443e008059d3e8d6fa3d4971b9b698
Author: yangjie01 <[email protected]>
AuthorDate: Thu Jun 27 18:12:06 2024 +0800
[SPARK-48708][CORE] Remove three unnecessary type registrations from
`KryoSerializer`
### What changes were proposed in this pull request?
This pr aims to remove three unnecessary type registrations from
`KryoSerializer`, they include `None`, `Nil`, and
`scala.collection.immutable.$colon$colon` .
### Why are the changes needed?
The types that have been cleaned up are already registered in
`ScalaKryoInstantiator`, so there is no need to register them again in Spark.
https://github.com/twitter/chill/blame/0a34e50f742ab74598c81f153cfe8ad90bf3a859/chill-scala/src/main/scala/com/twitter/chill/ScalaKryoInstantiator.scala#L238-L241
```scala
None.getClass, // None
classOf[Queue[_]],
Nil.getClass, // Nil
classOf[::[_]], // scala.collection.immutable.$colon$colon
```
### Does this PR introduce _any_ user-facing change?
No
### How was this patch tested?
Exist test: "Bug: SPARK-10251" in
`org.apache.spark.serializer.KryoSerializerSuite`
### Was this patch authored or co-authored using generative AI tooling?
No
Closes #47080 from LuciferYang/clean-kryo-register.
Authored-by: yangjie01 <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
---
core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala | 3 ---
1 file changed, 3 deletions(-)
diff --git
a/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala
b/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala
index 2ca30cf4cd62..ec5d53e91b3e 100644
--- a/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala
+++ b/core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala
@@ -227,10 +227,7 @@ class KryoSerializer(conf: SparkConf)
// scalastyle:on
- kryo.register(None.getClass)
- kryo.register(Nil.getClass)
kryo.register(Utils.classForName("scala.collection.immutable.ArraySeq$ofRef"))
-
kryo.register(Utils.classForName("scala.collection.immutable.$colon$colon"))
kryo.register(Utils.classForName("scala.collection.immutable.Map$EmptyMap$"))
kryo.register(Utils.classForName("scala.math.Ordering$Reverse"))
kryo.register(Utils.classForName("scala.reflect.ClassTag$GenericClassTag"))
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
