cloud-fan commented on a change in pull request #27280:
[SPARK-30290][CORE][followup] fix splitting shuffle fetch requests
URL: https://github.com/apache/spark/pull/27280#discussion_r368320210
##########
File path:
core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala
##########
@@ -418,9 +464,6 @@ final class ShuffleBlockFetcherIterator(
} else {
blocks
}
- // update metrics
- numBlocksToFetch += result.size
Review comment:
when we put the merged blocks back to the input list, it's kind of rollback
this method. I think it's better to not update mutable stats here if it can be
rollbacked.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
