Github user squito commented on a diff in the pull request:
https://github.com/apache/spark/pull/19583#discussion_r147525074
--- Diff:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
---
@@ -241,6 +243,21 @@ final class ShuffleBlockFetcherIterator(
logError(s"Failed to get block(s) from
${req.address.host}:${req.address.port}", e)
results.put(new FailureFetchResult(BlockId(blockId), address, e))
}
+
+ override def shouldRetry(t: Throwable): Boolean = {
--- End diff --
I don't understand the changes to `ShuffleBlockFetcherIterator` at all (I
guess they're incomplete from the TODOs??)
couldn't your `shouldRetry` just check if the epoch has been updated?
(you'd have to modify it so it has some way to get the current epoch)
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
