Github user gerashegalov commented on a diff in the pull request:
https://github.com/apache/spark/pull/20327#discussion_r176597452
--- Diff:
resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala
---
@@ -136,6 +135,39 @@ class YarnClusterSuite extends BaseYarnClusterSuite {
checkResult(finalState, result)
}
+ private def testClusterDriverBind(
+ uiEnabled: Boolean,
+ localHost: String,
+ localIp: String,
+ success: Boolean): Unit = {
+ val result = File.createTempFile("result", null, tempDir)
+ val finalState = runSpark(false,
mainClassName(YarnClusterDriver.getClass),
+ appArgs = Seq(result.getAbsolutePath()),
+ extraConf = Map(
+ "spark.yarn.appMasterEnv.SPARK_LOCAL_HOSTNAME" -> localHost,
+ "spark.yarn.appMasterEnv.SPARK_LOCAL_IP" -> localIp,
+ "spark.ui.enabled" -> uiEnabled.toString
+ ))
+ if (success) {
+ checkResult(finalState, result, "success")
+ } else {
+ finalState should be (SparkAppHandle.State.FAILED)
+ }
+ }
+
+ test("yarn-cluster driver should be able to bind listeners to MM_HOST") {
--- End diff --
> it assumes that both the RM and the Spark app have the same configuration
w.r.t. which interfaces they're binding to.
RM is not actively involved here. The driver executes on the NM. The launch
context of the driver prescribes to assign `SPARK_LOCAL_IP` and
`SPARK_LOCAL_HOSTNAME` on the worker node. Then AM rpcs the tracking URL
computed on the NM back to RM.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
