(incubator-uniffle) branch master updated: [#1309] fix(spark): WriteBufferManager in Spark2 does not use a reassigned shuffle server. (#1310)

[roryqi]

This is an automated email from the ASF dual-hosted git repository.

roryqi pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-uniffle.git


The following commit(s) were added to refs/heads/master by this push:
     new d5de2993f [#1309] fix(spark): WriteBufferManager in Spark2 does not 
use a reassigned shuffle server. (#1310)
d5de2993f is described below

commit d5de2993f58060b7f2d719509eaed967ff137ad7
Author: yl09099 <33595968+yl09...@users.noreply.github.com>
AuthorDate: Sat Nov 11 22:46:41 2023 +0800

    [#1309] fix(spark): WriteBufferManager in Spark2 does not use a reassigned 
shuffle server. (#1310)
    
    ### What changes were proposed in this pull request?
    
    In Spark2, after a write failure, ShuffleBlockInfo is not built using the 
new shuffle server after the shuffle server is reassigned.
    
    ### Why are the changes needed?
    
    Fix: #1309
    
    ### Does this PR introduce _any_ user-facing change?
    
    No.
    
    ### How was this patch tested?
    
    Existing UT.
---
 .../src/main/java/org/apache/spark/shuffle/writer/RssShuffleWriter.java | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 
a/client-spark/spark2/src/main/java/org/apache/spark/shuffle/writer/RssShuffleWriter.java
 
b/client-spark/spark2/src/main/java/org/apache/spark/shuffle/writer/RssShuffleWriter.java
index ac1e83ed0..6c95de78a 100644
--- 
a/client-spark/spark2/src/main/java/org/apache/spark/shuffle/writer/RssShuffleWriter.java
+++ 
b/client-spark/spark2/src/main/java/org/apache/spark/shuffle/writer/RssShuffleWriter.java
@@ -208,7 +208,7 @@ public class RssShuffleWriter<K, V, C> extends 
ShuffleWriter<K, V> {
             taskAttemptId,
             bufferOptions,
             rssHandle.getDependency().serializer(),
-            rssHandle.getPartitionToServers(),
+            shuffleHandleInfo.getPartitionToServers(),
             context.taskMemoryManager(),
             shuffleWriteMetrics,
             RssSparkConfig.toRssConf(sparkConf),