This is an automated email from the ASF dual-hosted git repository.
rexxiong pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 211046d0e [CELEBORN-2025] RpcFailure Scala 2.13 serialization is
incompatible
211046d0e is described below
commit 211046d0ec93f9d7432118dfbdc7c8f9524c5b70
Author: sychen <[email protected]>
AuthorDate: Thu Jun 5 22:05:56 2025 +0800
[CELEBORN-2025] RpcFailure Scala 2.13 serialization is incompatible
### What changes were proposed in this pull request?
### Why are the changes needed?
Spark4 uses scala 2.13, but it cannot be connected to the celeborn master
compiled with scala 2.12.
For example, the first node of `celeborn.master.endpoints` configured by
client is not a leader, and RpcFailure will be returned at this time.
```
https://github.com/scala/bug/issues/11207
https://users.scala-lang.org/t/serialversionuid-change-between-scala-2-12-6-and-2-12-7/3478
```
```java
java.io.InvalidClassException:
org.apache.celeborn.common.rpc.netty.RpcFailure; local class incompatible:
stream classdesc serialVersionUID = 2793139166962436434, local class
serialVersionUID = -1724324816907181707
at
java.base/java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:598)
~[?:?]
```
```bash
scala -cp /tmp/celeborn-client-spark-3-shaded_2.12-0.5.4.jar
```
```scala
:paste -raw
package org.apache.celeborn {
class Y {
def printId = {
val clazz =
classOf[org.apache.celeborn.common.rpc.netty.RpcFailure]
val uid =
java.io.ObjectStreamClass.lookup(clazz).getSerialVersionUID
println(s"Scala version:
${scala.util.Properties.versionNumberString}")
println(s"serialVersionUID: $uid")
}
}
}
new org.apache.celeborn.Y().printId
```
2.11
```
Scala version: 2.11.12
serialVersionUID: 2793139166962436434
```
2.12
```
Scala version: 2.12.19
serialVersionUID: 2793139166962436434
```
2.13
```
Scala version: 2.13.16
serialVersionUID: -1724324816907181707
```
### Does this PR introduce _any_ user-facing change?
If we used the master compiled with 2.13 before, it may be incompatible.
### How was this patch tested?
local test
```
Scala version: 2.13.16
serialVersionUID: 2793139166962436434
```
Closes #3309 from cxzl25/CELEBORN-2025.
Authored-by: sychen <[email protected]>
Signed-off-by: Shuang <[email protected]>
---
.../main/scala/org/apache/celeborn/common/rpc/netty/NettyRpcEnv.scala | 1 +
1 file changed, 1 insertion(+)
diff --git
a/common/src/main/scala/org/apache/celeborn/common/rpc/netty/NettyRpcEnv.scala
b/common/src/main/scala/org/apache/celeborn/common/rpc/netty/NettyRpcEnv.scala
index b2de46559..d857bd4b1 100644
---
a/common/src/main/scala/org/apache/celeborn/common/rpc/netty/NettyRpcEnv.scala
+++
b/common/src/main/scala/org/apache/celeborn/common/rpc/netty/NettyRpcEnv.scala
@@ -576,6 +576,7 @@ private[celeborn] object RequestMessage {
/**
* A response that indicates some failure happens in the receiver side.
*/
+@SerialVersionUID(2793139166962436434L)
private[celeborn] case class RpcFailure(e: Throwable)
/**
