[GitHub] spark pull request #22164: [SPARK-23679][YARN] Setting RM_HA_URLS for AmIpFi...
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22164 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22164: [SPARK-23679][YARN] Setting RM_HA_URLS for AmIpFi...
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22164#discussion_r213405174
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala
---
@@ -126,4 +136,21 @@ private[spark] class YarnRMClient extends Logging {
}
}
+ private def getUrlByRmId(conf: Configuration, rmId: String): String = {
--- End diff --
Ah, right, it would be hard to use that class in the client case.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22164: [SPARK-23679][YARN] Setting RM_HA_URLS for AmIpFi...
Github user jerryshao commented on a diff in the pull request:
https://github.com/apache/spark/pull/22164#discussion_r213168025
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala
---
@@ -126,4 +136,21 @@ private[spark] class YarnRMClient extends Logging {
}
}
+ private def getUrlByRmId(conf: Configuration, rmId: String): String = {
--- End diff --
For the Spark usage, I think it may not be so useful to use
`AmFilterInitializer`, because we need to pass the filter parameters to driver
either from RPC (client mode) or from configuration (cluster mode), in either
way we should know how to set each parameter, so from my understanding using
`AmFilterInitializer` seems not so useful.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] spark pull request #22164: [SPARK-23679][YARN] Setting RM_HA_URLS for AmIpFi...
Github user vanzin commented on a diff in the pull request:
https://github.com/apache/spark/pull/22164#discussion_r213102817
--- Diff:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnRMClient.scala
---
@@ -126,4 +136,21 @@ private[spark] class YarnRMClient extends Logging {
}
}
+ private def getUrlByRmId(conf: Configuration, rmId: String): String = {
--- End diff --
The code looks ok, but it also looks similar to this:
https://github.com/apache/hadoop/blob/branch-2.6/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-web-proxy/src/main/java/org/apache/hadoop/yarn/server/webproxy/amfilter/AmFilterInitializer.java
I'm wondering if we could just call that class instead, somehow? It seems
available in 2.6 which is the oldest version we support.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
