leixm commented on code in PR #1141:
URL:
https://github.com/apache/incubator-uniffle/pull/1141#discussion_r1360529179
##########
client-spark/spark3/src/main/java/org/apache/spark/shuffle/reader/RssShuffleReader.java:
##########
@@ -114,7 +115,7 @@ public RssShuffleReader(
this.taskIdBitmap = taskIdBitmap;
this.hadoopConf = hadoopConf;
this.readMetrics = readMetrics;
- this.partitionToShuffleServers = rssShuffleHandle.getPartitionToServers();
+ this.partitionToShuffleServers = allPartitionToServers;
Review Comment:
Here too, can we put the information maintained by ShuffleManagerService
into rssShuffleHandle, so that it is transparent to RssShuffleReader and
RssShuffleWriter.
##########
client-spark/spark3/src/main/java/org/apache/spark/shuffle/writer/RssShuffleWriter.java:
##########
@@ -109,7 +113,10 @@ public RssShuffleWriter(
RssShuffleManager shuffleManager,
SparkConf sparkConf,
ShuffleWriteClient shuffleWriteClient,
- RssShuffleHandle<K, V, C> rssHandle) {
+ RssShuffleHandle<K, V, C> rssHandle,
+ ShuffleHandleInfo shuffleHandleInfo,
+ TaskContext context,
+ ShuffleManagerClient shuffleManagerClient) {
Review Comment:
Is it possible to put shuffleHandleInfo into rssHandle? This will make the
code cleaner. @yl09099
##########
client-spark/spark3/src/main/java/org/apache/spark/shuffle/writer/RssShuffleWriter.java:
##########
@@ -109,7 +113,10 @@ public RssShuffleWriter(
RssShuffleManager shuffleManager,
SparkConf sparkConf,
ShuffleWriteClient shuffleWriteClient,
- RssShuffleHandle<K, V, C> rssHandle) {
+ RssShuffleHandle<K, V, C> rssHandle,
+ ShuffleHandleInfo shuffleHandleInfo,
+ TaskContext context,
+ ShuffleManagerClient shuffleManagerClient) {
Review Comment:
@yl09099 Why change this constructor to add TaskContext and
ShuffleManagerClient?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
