cloud-fan commented on a change in pull request #25620: [SPARK-25341][Core]
Support rolling back a shuffle map stage and re-generate the shuffle files
URL: https://github.com/apache/spark/pull/25620#discussion_r325782752
##########
File path:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
##########
@@ -694,9 +700,10 @@ object ShuffleBlockFetcherIterator {
* A request to fetch blocks from a remote BlockManager.
* @param address remote BlockManager to fetch from.
* @param blocks Sequence of tuple, where the first element is the block id,
- * and the second element is the estimated size, used to
calculate bytesInFlight.
+ * and the second element is the estimated size, used to
calculate bytesInFlight,
+ * the third element is the mapIndex.
*/
- case class FetchRequest(address: BlockManagerId, blocks: Seq[(BlockId,
Long)]) {
+ case class FetchRequest(address: BlockManagerId, blocks: Seq[(BlockId, Long,
Int)]) {
Review comment:
Now it's a tuple3 with int and long elements. I think it's better to create
a class for it to make the code easier to read.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
