This is an automated email from the ASF dual-hosted git repository.
fchen pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new f61fe1755 [CELEBORN-987][FOLLOWUP][DOC] README#Build and sbt#System
Requirements should extend to Scala 2.13 and Spark 3.5
f61fe1755 is described below
commit f61fe175515d4ba02dd8debdf9c356e1729b13b9
Author: SteNicholas <[email protected]>
AuthorDate: Sat Oct 14 09:54:22 2023 +0800
[CELEBORN-987][FOLLOWUP][DOC] README#Build and sbt#System Requirements
should extend to Scala 2.13 and Spark 3.5
### What changes were proposed in this pull request?
`README#Build` and `sbt#System Requirements` extends to Scala 2.13.
### Why are the changes needed?
`README#Build` and `sbt#System Requirements`should extend to Scala 2.13 to
align the SBT CI test results.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
SBT CI tests.
Closes #1987 from SteNicholas/CELEBORN-987.
Authored-by: SteNicholas <[email protected]>
Signed-off-by: Fu Chen <[email protected]>
---
README.md | 27 ++++++++++++++-------------
docs/developers/sbt.md | 25 +++++++++++++------------
2 files changed, 27 insertions(+), 25 deletions(-)
diff --git a/README.md b/README.md
index 8e19998be..2472ce4c1 100644
--- a/README.md
+++ b/README.md
@@ -41,8 +41,8 @@ Celeborn Worker's slot count is decided by `total usable disk
size / average shu
Celeborn worker's slot count decreases when a partition is allocated and
increments when a partition is freed.
## Build
-1. Celeborn supports Spark 2.4/3.0/3.1/3.2/3.3/3.4, Flink 1.14/1.15/1.17 and
Hadoop MapReduce 2/3.
-2. Celeborn tested under Scala 2.11/2.12 and Java 8/11/17 environment.
+1. Celeborn supports Spark 2.4/3.0/3.1/3.2/3.3/3.4/3.5, Flink 1.14/1.15/1.17
and Hadoop MapReduce 2/3.
+2. Celeborn tested under Scala 2.11/2.12/2.13 and Java 8/11/17 environment.
Build Celeborn
```shell
@@ -53,17 +53,18 @@ package apache-celeborn-${project.version}-bin.tgz will be
generated.
> **_NOTE:_** The following table indicates the compatibility of Celeborn
> Spark and Flink clients with different versions of Spark and Flink for
> various Java and Scala versions.
-| | Java 8/Scala 2.11 | Java 8/Scala 2.12 | Java 11/Scala 2.12
| Java 17/Scala 2.12 |
-|----------------|-------------------|-------------------|--------------------|--------------------|
-| Spark 2.4 | ✔ | ❌ | ❌
| ❌ |
-| Spark 3.0 | ❌ | ✔ | ✔
| ❌ |
-| Spark 3.1 | ❌ | ✔ | ✔
| ❌ |
-| Spark 3.2 | ❌ | ✔ | ✔
| ❌ |
-| Spark 3.3 | ❌ | ✔ | ✔
| ✔ |
-| Spark 3.4 | ❌ | ✔ | ✔
| ✔ |
-| Flink 1.14 | ❌ | ✔ | ✔
| ❌ |
-| Flink 1.15 | ❌ | ✔ | ✔
| ❌ |
-| Flink 1.17 | ❌ | ✔ | ✔
| ❌ |
+| | Java 8/Scala 2.11 | Java 8/Scala 2.12 | Java 11/Scala 2.12 |
Java 17/Scala 2.12 | Java 8/Scala 2.13 | Java 11/Scala 2.13 | Java 17/Scala
2.13 |
+|------------|-------------------|-------------------|--------------------|--------------------|-------------------|--------------------|--------------------|
+| Spark 2.4 | ✔ | ❌ | ❌ |
❌ | ❌ | ❌ | ❌
|
+| Spark 3.0 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
+| Spark 3.1 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
+| Spark 3.2 | ❌ | ✔ | ✔ |
❌ | ✔ | ✔ | ❌
|
+| Spark 3.3 | ❌ | ✔ | ✔ |
✔ | ✔ | ✔ | ✔
|
+| Spark 3.4 | ❌ | ✔ | ✔ |
✔ | ✔ | ✔ | ✔
|
+| Spark 3.5 | ❌ | ✔ | ✔ |
✔ | ✔ | ✔ | ✔
|
+| Flink 1.14 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
+| Flink 1.15 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
+| Flink 1.17 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
### Package Details
Build procedure will create a compressed package.
diff --git a/docs/developers/sbt.md b/docs/developers/sbt.md
index 8daa6c9a7..c949d0db6 100644
--- a/docs/developers/sbt.md
+++ b/docs/developers/sbt.md
@@ -22,21 +22,22 @@ Starting from version 0.4.0, the Celeborn project supports
building and packagin
## System Requirements
-Celeborn Service (master/worker) supports Scala 2.11/2.12 and Java 8/11/17.
+Celeborn Service (master/worker) supports Scala 2.11/2.12/2.13 and Java
8/11/17.
The following table indicates the compatibility of Celeborn Spark and Flink
clients with different versions of Spark and Flink for various Java and Scala
versions:
-| | Java 8/Scala 2.11 | Java 8/Scala 2.12 | Java 11/Scala 2.12
| Java 17/Scala 2.12 |
-|----------------|-------------------|-------------------|--------------------|--------------------|
-| Spark 2.4 | ✔ | ❌ | ❌
| ❌ |
-| Spark 3.0 | ❌ | ✔ | ✔
| ❌ |
-| Spark 3.1 | ❌ | ✔ | ✔
| ❌ |
-| Spark 3.2 | ❌ | ✔ | ✔
| ❌ |
-| Spark 3.3 | ❌ | ✔ | ✔
| ✔ |
-| Spark 3.4 | ❌ | ✔ | ✔
| ✔ |
-| Flink 1.14 | ❌ | ✔ | ✔
| ❌ |
-| Flink 1.15 | ❌ | ✔ | ✔
| ❌ |
-| Flink 1.17 | ❌ | ✔ | ✔
| ❌ |
+| | Java 8/Scala 2.11 | Java 8/Scala 2.12 | Java 11/Scala 2.12 |
Java 17/Scala 2.12 | Java 8/Scala 2.13 | Java 11/Scala 2.13 | Java 17/Scala
2.13 |
+|------------|-------------------|-------------------|--------------------|--------------------|-------------------|--------------------|--------------------|
+| Spark 2.4 | ✔ | ❌ | ❌ |
❌ | ❌ | ❌ | ❌
|
+| Spark 3.0 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
+| Spark 3.1 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
+| Spark 3.2 | ❌ | ✔ | ✔ |
❌ | ✔ | ✔ | ❌
|
+| Spark 3.3 | ❌ | ✔ | ✔ |
✔ | ✔ | ✔ | ✔
|
+| Spark 3.4 | ❌ | ✔ | ✔ |
✔ | ✔ | ✔ | ✔
|
+| Spark 3.5 | ❌ | ✔ | ✔ |
✔ | ✔ | ✔ | ✔
|
+| Flink 1.14 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
+| Flink 1.15 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
+| Flink 1.17 | ❌ | ✔ | ✔ |
❌ | ❌ | ❌ | ❌
|
## Useful SBT commands
