[ https://issues.apache.org/jira/browse/SPARK-37556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Sean R. Owen reassigned SPARK-37556: ------------------------------------ Assignee: Daniel Dai > Deser void class fail with Java serialization > --------------------------------------------- > > Key: SPARK-37556 > URL: https://issues.apache.org/jira/browse/SPARK-37556 > Project: Spark > Issue Type: Bug > Components: Spark Core > Affects Versions: 3.2.0 > Reporter: Daniel Dai > Assignee: Daniel Dai > Priority: Major > > Spark code contains Java void type cannot be serialized with JavaSerializer. > For example: > {code:java} > class Foo extends Serializable { > Class k = Void.TYPE; > } > {code} > Spark will throw error: > {code:java} > java.lang.Void; local class name incompatible with stream class name "void" > at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:703) > at > java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1940) > at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1806) > at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1771) > ...... > {code} > All other primitive type works. The reason is when SPARK-8730 try to fix the > primitive type deserialization, it introduces the [following > code|https://github.com/apache/spark/blob/v3.2.0/core/src/main/scala/org/apache/spark/serializer/JavaSerializer.scala#L80]: > {code:java} > private object JavaDeserializationStream { > val primitiveMappings = Map[String, Class[_]]( > "boolean" -> classOf[Boolean], > "byte" -> classOf[Byte], > "char" -> classOf[Char], > "short" -> classOf[Short], > "int" -> classOf[Int], > "long" -> classOf[Long], > "float" -> classOf[Float], > "double" -> classOf[Double], > "void" -> classOf[Void] > ) > } > {code} > However, classOf[Void] is not the equivalence of other types. It's point to > Void.class not Void.Type. The equivalence for void should be classOf[Unit]: > {code:java} > scala> classOf[Long] > val res0: Class[Long] = long > scala> classOf[Double] > val res1: Class[Double] = double > scala> classOf[Byte] > val res2: Class[Byte] = byte > scala> classOf[Void] > val res3: Class[Void] = class java.lang.Void <--- this is wrong > scala> classOf[Unit] > val res4: Class[Unit] = void <---- this is right > {code} -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org