This is an automated email from the ASF dual-hosted git repository.
zuston pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git
The following commit(s) were added to refs/heads/master by this push:
new 0b66472ec [#1373][FOLLOWUP] fix(spark3):Add client type when request
shuffle assignment (#1580)
0b66472ec is described below
commit 0b66472ec7cb982290a08306ac46c8c136fc5286
Author: dingshun3016 <[email protected]>
AuthorDate: Fri Mar 15 10:38:33 2024 +0800
[#1373][FOLLOWUP] fix(spark3):Add client type when request shuffle
assignment (#1580)
### What changes were proposed in this pull request?
add client type when request shuffle assignment
### Why are the changes needed?
Fix: (#1373)
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
Not necessary.
---
.../src/main/java/org/apache/spark/shuffle/RssShuffleManager.java | 2 ++
1 file changed, 2 insertions(+)
diff --git
a/client-spark/spark3/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java
b/client-spark/spark3/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java
index d0995734b..1f4a4cd0f 100644
---
a/client-spark/spark3/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java
+++
b/client-spark/spark3/src/main/java/org/apache/spark/shuffle/RssShuffleManager.java
@@ -1233,6 +1233,8 @@ public class RssShuffleManager extends
RssShuffleManagerBase {
int estimateTaskConcurrency,
Set<String> faultyServerIds) {
Set<String> assignmentTags =
RssSparkShuffleUtils.getAssignmentTags(sparkConf);
+ ClientUtils.validateClientType(clientType);
+ assignmentTags.add(clientType);
long retryInterval =
sparkConf.get(RssSparkConfig.RSS_CLIENT_ASSIGNMENT_RETRY_INTERVAL);
int retryTimes =
sparkConf.get(RssSparkConfig.RSS_CLIENT_ASSIGNMENT_RETRY_TIMES);
faultyServerIds.addAll(failuresShuffleServerIds);