Github user wangyum commented on a diff in the pull request:
https://github.com/apache/spark/pull/20785#discussion_r174979402
--- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala ---
@@ -2434,7 +2434,8 @@ private[spark] object Utils extends Logging {
*/
def getSparkOrYarnConfig(conf: SparkConf, key: String, default: String):
String = {
val sparkValue = conf.get(key, default)
- if (conf.get(SparkLauncher.SPARK_MASTER, null) == "yarn") {
+ if (conf.get(SparkLauncher.SPARK_MASTER, null) == "yarn"
--- End diff --
Assuming that `--conf spark.shuffle.service.port = 7338` is configured,
7338 is displayed on the tab of the environment, but 7337 is actually used.
So my idea is get value from `SparkConf ` if key starting with `spark.`
except for `spark.hadoop.`.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
