squito commented on a change in pull request #25299: [SPARK-27651][Core] Avoid the network when shuffle blocks are fetched from the same host URL: https://github.com/apache/spark/pull/25299#discussion_r313060387
########## File path: core/src/main/scala/org/apache/spark/storage/ShuffleBlockFetcherIterator.scala ########## @@ -98,8 +101,12 @@ final class ShuffleBlockFetcherIterator( /** Local blocks to fetch, excluding zero-sized blocks. */ private[this] val localBlocks = scala.collection.mutable.LinkedHashSet[BlockId]() - /** Remote blocks to fetch, excluding zero-sized blocks. */ - private[this] val remoteBlocks = new HashSet[BlockId]() + /** Host local blockIds to fetch by executors, excluding zero-sized blocks by executors. */ Review comment: nit: "by executors" is repeated, I'd delete the second one. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
