otterc commented on a change in pull request #31934:
URL: https://github.com/apache/spark/pull/31934#discussion_r600664075
##########
File path:
common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java
##########
@@ -738,14 +723,14 @@ public void onFailure(String streamId, Throwable
throwable) throws IOException {
Map<Integer, AppShufflePartitionInfo> shufflePartitions =
mergeManager.partitions.get(partitionInfo.appShuffleId);
if (shufflePartitions != null &&
shufflePartitions.containsKey(partitionInfo.reduceId)) {
- logger.debug("{} shuffleId {} reduceId {} set encountered failure",
+ logger.debug("{} shuffleId {} reduceId {} encountered failure",
partitionInfo.appShuffleId.appId,
partitionInfo.appShuffleId.shuffleId,
partitionInfo.reduceId);
partitionInfo.setCurrentMapIndex(-1);
- partitionInfo.setEncounteredFailure(true);
}
}
}
+ isWriting = false;
Review comment:
I can move this to `if` scope and that would not change the behavior or
cause any issues. The only reason I had it outside because it was consistent
with where this flag is unset in `onComplete`. I understand that is a very
trivial thing so can move this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
