dongjoon-hyun commented on a change in pull request #33378:
URL: https://github.com/apache/spark/pull/33378#discussion_r670975574
##########
File path:
core/src/main/scala/org/apache/spark/storage/PushBasedFetchHelper.scala
##########
@@ -197,9 +197,12 @@ private class PushBasedFetchHelper(
localShuffleMergerBlockMgrId)
}
} else {
- logDebug(s"Asynchronous fetch the push-merged-local blocks without
cached merged dirs")
- hostLocalDirManager.getHostLocalDirs(localShuffleMergerBlockMgrId.host,
- localShuffleMergerBlockMgrId.port, Array(SHUFFLE_MERGER_IDENTIFIER)) {
+ // Push-based shuffle is only enabled when the ESS is enabled. If ESS is
not enabled, then
Review comment:
For now, this file has another instance of ESS. It would be great if you
can switch it together, @otterc .
```
$ git grep ' ESS '
core/src/main/scala/org/apache/spark/storage/PushBasedFetchHelper.scala: *
shuffle blocks that belong to the common reduce partition and were merged by
the ESS to that
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
