xuanyuanking commented on a change in pull request #24892: [SPARK-25341][Core]
Support rolling back a shuffle map stage and re-generate the shuffle files
URL: https://github.com/apache/spark/pull/24892#discussion_r299787099
##########
File path:
core/src/main/scala/org/apache/spark/network/BlockTransferService.scala
##########
@@ -55,14 +55,26 @@ abstract class BlockTransferService extends ShuffleClient
with Closeable with Lo
def hostName: String
/**
- * Fetch a sequence of blocks from a remote node asynchronously,
+ * Fetch a sequence of shuffle blocks from a remote node asynchronously,
* available only after [[init]] is invoked.
*
* Note that this API takes a sequence so the implementation can batch
requests, and does not
* return a future so the underlying implementation can invoke
onBlockFetchSuccess as soon as
* the data of a block is fetched, rather than waiting for all blocks to be
fetched.
*/
override def fetchBlocks(
+ host: String,
+ port: Int,
+ execId: String,
+ shuffleGenerationId: Int,
+ blockIds: Array[String],
+ listener: BlockFetchingListener,
+ tempFileManager: DownloadFileManager): Unit
+
+ /**
+ * Fetch a sequence of shuffle blocks.
Review comment:
Ah yes, done in 7259500.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
