Github user benoyantony commented on a diff in the pull request:
https://github.com/apache/spark/pull/2276#discussion_r17310796
--- Diff:
yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala
---
@@ -189,7 +189,11 @@ private[spark] class ApplicationMaster(args:
ApplicationMasterArguments,
if (sc == null) {
finish(FinalApplicationStatus.FAILED, "Timed out waiting for
SparkContext.")
} else {
- registerAM(sc.ui.appUIHostPort, securityMgr)
+ var uiAddress = sc.ui.appUIHostPort
+ if (yarnConf.get("yarn.http.policy").equals("HTTPS_ONLY")) {
--- End diff --
Per @vanzin , alpha means pre-branch-2 hadoop. Before YARN-1203(Hadoop
2.2), we cannot pass AM URLS with scheme. There are stable releases before
YARN-1203. So for stable releases without YARN-1203, we have to pass host:port.
There are 4 cases:
Alpha and yarn.http.policy !=HTTPS_ONLY : uiAddress = host:port
Alpha and yarn.http.policy ==HTTPS_ONLY : uiAddress = scheme:host:port
(strip off scheme since alpha will choke on scheme.)
stable and yarn.http.policy !=HTTPS_ONLY : uiAddress = host:port
stable and yarn.http.policy ==HTTPS_ONLY : uiAddress = scheme:host:port
(we are taking a bet that yarn.http.policy==HTTPS_ONLY occurs only for stable
release after Yarn-1203 since this property was introduced by Yarn-1277 . Both
these changes are released in Hadoop-2.2. )
So I think, we need to do the conditional logic for both stable and alpha.
We also need to strip off the scheme for alpha.
Please let me know , if there is a better solution.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
