This is an automated email from the ASF dual-hosted git repository.
angerszhuuuu pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/incubator-celeborn.git
The following commit(s) were added to refs/heads/main by this push:
new 0db215073 [CELEBORN-808] Remove unnecessary RssShuffleManager in 0.4.0
0db215073 is described below
commit 0db2150731adbe6d4e5f3e38fde0af6df0943eb8
Author: Angerszhuuuu <[email protected]>
AuthorDate: Thu Jul 27 17:47:44 2023 +0800
[CELEBORN-808] Remove unnecessary RssShuffleManager in 0.4.0
### What changes were proposed in this pull request?
As title
### Why are the changes needed?
### Does this PR introduce _any_ user-facing change?
### How was this patch tested?
Closes #1731 from AngersZhuuuu/CELEBORN-808.
Authored-by: Angerszhuuuu <[email protected]>
Signed-off-by: Angerszhuuuu <[email protected]>
---
.../spark/shuffle/celeborn/RssShuffleManager.java | 27 ----------------------
.../spark/shuffle/celeborn/RssShuffleManager.java | 27 ----------------------
docs/migration.md | 4 ++++
3 files changed, 4 insertions(+), 54 deletions(-)
diff --git
a/client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/RssShuffleManager.java
b/client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/RssShuffleManager.java
deleted file mode 100644
index c12a908cd..000000000
---
a/client-spark/spark-2/src/main/java/org/apache/spark/shuffle/celeborn/RssShuffleManager.java
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.spark.shuffle.celeborn;
-
-import org.apache.spark.SparkConf;
-
-public class RssShuffleManager extends SparkShuffleManager {
-
- public RssShuffleManager(SparkConf conf, boolean isDriver) {
- super(conf, isDriver);
- }
-}
diff --git
a/client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/RssShuffleManager.java
b/client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/RssShuffleManager.java
deleted file mode 100644
index c12a908cd..000000000
---
a/client-spark/spark-3/src/main/java/org/apache/spark/shuffle/celeborn/RssShuffleManager.java
+++ /dev/null
@@ -1,27 +0,0 @@
-/*
- * Licensed to the Apache Software Foundation (ASF) under one or more
- * contributor license agreements. See the NOTICE file distributed with
- * this work for additional information regarding copyright ownership.
- * The ASF licenses this file to You under the Apache License, Version 2.0
- * (the "License"); you may not use this file except in compliance with
- * the License. You may obtain a copy of the License at
- *
- * http://www.apache.org/licenses/LICENSE-2.0
- *
- * Unless required by applicable law or agreed to in writing, software
- * distributed under the License is distributed on an "AS IS" BASIS,
- * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- * See the License for the specific language governing permissions and
- * limitations under the License.
- */
-
-package org.apache.spark.shuffle.celeborn;
-
-import org.apache.spark.SparkConf;
-
-public class RssShuffleManager extends SparkShuffleManager {
-
- public RssShuffleManager(SparkConf conf, boolean isDriver) {
- super(conf, isDriver);
- }
-}
diff --git a/docs/migration.md b/docs/migration.md
index 2001bef1f..c41a73088 100644
--- a/docs/migration.md
+++ b/docs/migration.md
@@ -68,3 +68,7 @@ license: |
- Since 0.3.0, Celeborn's spark shuffle manager change from
`org.apache.spark.shuffle.celeborn.RssShuffleManager` to
`org.apache.spark.shuffle.celeborn.SparkShuffleManager`. User can set spark
property `spark.shuffle.manager` to
`org.apache.spark.shuffle.celeborn.SparkShuffleManager` to use Celeborn remote
shuffle service.
In 0.3.0, Celeborn still support
`org.apache.spark.shuffle.celeborn.RssShuffleManager`, it will be removed in
0.4.0.
+
+## Upgrading from 0.3 to 0.4
+
+ - Since 0.4.0, Celeborn won't support
`org.apache.spark.shuffle.celeborn.RssShuffleManager`.
