(incubator-celeborn) branch main updated: [CELEBORN-1327] Support Spark 3.5 with JDK21

[chengpan]

This is an automated email from the ASF dual-hosted git repository.

chengpan pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git


The following commit(s) were added to refs/heads/main by this push:
     new 23269d77e [CELEBORN-1327] Support Spark 3.5 with JDK21
23269d77e is described below

commit 23269d77e97001321bf8371b769192f33e0acffb
Author: Curtis Howard <cur...@curtiss-mbp.lan>
AuthorDate: Wed Mar 13 23:08:09 2024 +0800

    [CELEBORN-1327] Support Spark 3.5 with JDK21
    
    ### What changes were proposed in this pull request?
    Updates Celeborn to account for 
[JDK-8303083](https://bugs.openjdk.org/browse/JDK-8303083), which affects 
JDK21.  (similar to the Apache Spark change here:
    https://github.com/apache/spark/pull/39909)
    
    ### Why are the changes needed?
    Without this change, Spark users of Celeborn will encounter a runtime error 
similar to the following:
    `Caused by: java.lang.ExceptionInInitializerError: Exception 
java.lang.IllegalStateException: java.lang.NoSuchMethodException: 
java.nio.DirectByteBuffer.<init>(long, int) [in thread "Executor task launch 
worker for task 0.0 in stage 0.0 (TID 0)"]
            at 
org.apache.celeborn.common.unsafe.Platform.<clinit>(Platform.java:135)
            ... 16 more`
    
    ### Does this PR introduce _any_ user-facing change?
    No.
    
    ### How was this patch tested?
    Tested using standalone Spark 3.5.1 (Hadoop 3.2) and the Celeborn `main` 
branch with JDK21 build changes in 
https://github.com/apache/incubator-celeborn/pull/2385.  Reproduced the runtime 
error above and confirmed the patch resolves it.
    
    Closes #2387 from curtishoward/CELEBORN-1327.
    
    Authored-by: Curtis Howard <cur...@curtiss-mbp.lan>
    Signed-off-by: Cheng Pan <cheng...@apache.org>
---
 .../main/java/org/apache/celeborn/common/unsafe/Platform.java  | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git 
a/common/src/main/java/org/apache/celeborn/common/unsafe/Platform.java 
b/common/src/main/java/org/apache/celeborn/common/unsafe/Platform.java
index ec541a770..278630e4f 100644
--- a/common/src/main/java/org/apache/celeborn/common/unsafe/Platform.java
+++ b/common/src/main/java/org/apache/celeborn/common/unsafe/Platform.java
@@ -90,7 +90,15 @@ public final class Platform {
     }
     try {
       Class<?> cls = Class.forName("java.nio.DirectByteBuffer");
-      Constructor<?> constructor = cls.getDeclaredConstructor(Long.TYPE, 
Integer.TYPE);
+      Constructor<?> constructor;
+      try {
+        constructor = cls.getDeclaredConstructor(Long.TYPE, Integer.TYPE);
+      } catch (NoSuchMethodException e) {
+        // DirectByteBuffer(long,int) was removed in
+        // 
https://github.com/openjdk/jdk/commit/a56598f5a534cc9223367e7faa8433ea38661db9
+        constructor = cls.getDeclaredConstructor(Long.TYPE, Long.TYPE);
+      }
+
       Field cleanerField = cls.getDeclaredField("cleaner");
       try {
         constructor.setAccessible(true);