[GitHub] spark pull request #20785: [SPARK-23640][CORE] Fix hadoop config may overrid...

[wangyum]

Github user wangyum commented on a diff in the pull request:

    https://github.com/apache/spark/pull/20785#discussion_r174980995
  
    --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
    @@ -2434,7 +2434,8 @@ private[spark] object Utils extends Logging {
        */
       def getSparkOrYarnConfig(conf: SparkConf, key: String, default: String): 
String = {
         val sparkValue = conf.get(key, default)
    -    if (conf.get(SparkLauncher.SPARK_MASTER, null) == "yarn") {
    +    if (conf.get(SparkLauncher.SPARK_MASTER, null) == "yarn"
    --- End diff --
    
    `YarnConfiguration` can only configure one `spark.shuffle.service.port` 
value.
    We can gradually upgrade the shuffle service if get 
`spark.shuffle.service.port` value from `SparkConf` because we can set 
different values ​​for different applications.


---

---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org