Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/20785#discussion_r174980995
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2434,7 +2434,8 @@ private[spark] object Utils extends Logging {
*/
def getSparkOrYarnConfig(conf: SparkConf, key: String, default: String):
String = {
val sparkValue = conf.get(key, default)
- if (conf.get(SparkLauncher.SPARK_MASTER, null) == "yarn") {
+ if (conf.get(SparkLauncher.SPARK_MASTER, null) == "yarn"
--- End diff --
`YarnConfiguration` can only configure one `spark.shuffle.service.port`
value.
We can gradually upgrade the shuffle service if get
`spark.shuffle.service.port` value from `SparkConf` because we can set
different values ââfor different applications.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
