cloud-fan commented on a change in pull request #25620: [SPARK-25341][Core]
Support rolling back a shuffle map stage and re-generate the shuffle files
URL: https://github.com/apache/spark/pull/25620#discussion_r319119240
##########
File path:
core/src/main/java/org/apache/spark/shuffle/api/ShuffleExecutorComponents.java
##########
@@ -39,17 +39,15 @@
/**
* Called once per map task to create a writer that will be responsible for
persisting all the
* partitioned bytes written by that map task.
- * @param shuffleId Unique identifier for the shuffle the map task is a
part of
- * @param mapId Within the shuffle, the identifier of the map task
+ * @param shuffleId Unique identifier for the shuffle the map task is a part
of
* @param mapTaskAttemptId Identifier of the task attempt. Multiple attempts
of the same map task
- * with the same (shuffleId, mapId) pair can be
distinguished by the
- * different values of mapTaskAttemptId.
+ * with the same (shuffleId, mapId) pair can be
distinguished by the
+ * different values of mapTaskAttemptId.
* @param numPartitions The number of partitions that will be written by the
map task. Some of
-* these partitions may be empty.
+ * these partitions may be empty.
*/
ShuffleMapOutputWriter createMapOutputWriter(
int shuffleId,
- int mapId,
long mapTaskAttemptId,
Review comment:
shall we name it `mapId`? To be consistent with the codebase.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
