Ngone51 commented on a change in pull request #31876:
URL: https://github.com/apache/spark/pull/31876#discussion_r618106897
##########
File path: core/src/main/scala/org/apache/spark/internal/config/package.scala
##########
@@ -1217,6 +1217,14 @@ package object config {
.stringConf
.createWithDefault(classOf[LocalDiskShuffleDataIO].getName)
+ private[spark] val SHUFFLE_LOCATION_PLUGIN_CLASS =
+ ConfigBuilder("spark.shuffle.sort.location.plugin.class")
+ .doc("Qualified name of the class that used to initiate plugin location
instance. " +
+ "If not specified, Spark will use its native location (a.k.a
BlockManagerId) by default.")
+ .version("3.2.0")
+ .stringConf
+ .createOptional
Review comment:
I used the `optional` intentionally.
I tried the way of the default value (`BlockManagerId`) previously. And I
realized that we wouldn't expect to load the `BlockManager` using the
reflection, which is low efficiency, but create it directly as it's a built-in
class in Spark. As a result, the default value won't be used at all (or it
would be only used to see if it's not the default value by comparison, but
this's meaningless to do so).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
