viirya commented on a change in pull request #29916:
URL: https://github.com/apache/spark/pull/29916#discussion_r498924302
##########
File path: core/src/main/scala/org/apache/spark/shuffle/ShuffleManager.scala
##########
@@ -24,8 +24,12 @@ import org.apache.spark.{ShuffleDependency, TaskContext}
* and on each executor, based on the spark.shuffle.manager setting. The
driver registers shuffles
* with it, and executors (or tasks running locally in the driver) can ask to
read and write data.
*
- * NOTE: this will be instantiated by SparkEnv so its constructor can take a
SparkConf and
+ * NOTE:
+ * 1. This will be instantiated by SparkEnv so its constructor can take a
SparkConf and
* boolean isDriver as parameters.
+ * 2. This contains a method ShuffleBlockResolver which interacts with
External Shuffle Service
+ * when it is enabled. Need to pay attention to that, if implementing a custom
ShuffleManager, to
+ * make sure the custom ShuffleManager could co-exist with External Shuffle
Service.
Review comment:
Good to see this doc improvement.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
