[jira] [Commented] (YARN-3714) AM proxy filter can not get proper default proxy address if RM-HA is enabled
[
https://issues.apache.org/jira/browse/YARN-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14587181#comment-14587181
]
Hadoop QA commented on YARN-3714:
-
\\
\\
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 16m 59s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 41s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 48s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 23s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 13s | There were no new checkstyle
issues. |
| {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that
end in whitespace. |
| {color:green}+1{color} | install | 1m 32s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 2m 14s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 2m 2s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 0m 24s | Tests passed in
hadoop-yarn-server-web-proxy. |
| | | 42m 54s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12739710/YARN-3714.004.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2cb09e9 |
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8254/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-web-proxy test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8254/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8254/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8254/console |
This message was automatically generated.
> AM proxy filter can not get proper default proxy address if RM-HA is enabled
>
>
> Key: YARN-3714
> URL: https://issues.apache.org/jira/browse/YARN-3714
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Attachments: YARN-3714.001.patch, YARN-3714.002.patch,
> YARN-3714.003.patch, YARN-3714.004.patch
>
>
> Default proxy address could not be got without setting
> {{yarn.resourcemanager.webapp.address._rm-id_}} and/or
> {{yarn.resourcemanager.webapp.https.address._rm-id_}} explicitly if RM-HA is
> enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3714) AM proxy filter can not get proper default proxy address if RM-HA is enabled
[
https://issues.apache.org/jira/browse/YARN-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586377#comment-14586377
]
Xuan Gong commented on YARN-3714:
-
bq. HAUtil.verifyAndSetConfiguration works only on the RM node. AMs running in
slave nodes also need to know the RM webapp addresses.
Thanks for the explanation. That makes sense.
The patch looks good overall. One small nit:
in {code}
public static List getRMHAWebappAddresses(
final YarnConfiguration conf) {
{code}
We could check whether RM_WEBAPP_ADDRESS has been set with RM-ids. If not, we
only need to check whether RM_HOSTNAME has been set with RM-ids instead of
calling {code}
HAUtil.verifyAndSetRMHAIdsList(conf);
{code} ?
> AM proxy filter can not get proper default proxy address if RM-HA is enabled
>
>
> Key: YARN-3714
> URL: https://issues.apache.org/jira/browse/YARN-3714
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Attachments: YARN-3714.001.patch, YARN-3714.002.patch,
> YARN-3714.003.patch
>
>
> Default proxy address could not be got without setting
> {{yarn.resourcemanager.webapp.address._rm-id_}} and/or
> {{yarn.resourcemanager.webapp.https.address._rm-id_}} explicitly if RM-HA is
> enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3714) AM proxy filter can not get proper default proxy address if RM-HA is enabled
[
https://issues.apache.org/jira/browse/YARN-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14586263#comment-14586263
]
Hadoop QA commented on YARN-3714:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 22m 43s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 9m 15s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 11m 13s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 26s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 2m 35s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 50s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 40s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 4m 43s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 0m 27s | Tests passed in
hadoop-yarn-api. |
| {color:green}+1{color} | yarn tests | 2m 15s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 0m 22s | Tests passed in
hadoop-yarn-server-web-proxy. |
| | | 56m 32s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12739513/YARN-3714.003.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 4c5da9b |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/8252/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-api test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8252/artifact/patchprocess/testrun_hadoop-yarn-api.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8252/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-web-proxy test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8252/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8252/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8252/console |
This message was automatically generated.
> AM proxy filter can not get proper default proxy address if RM-HA is enabled
>
>
> Key: YARN-3714
> URL: https://issues.apache.org/jira/browse/YARN-3714
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Attachments: YARN-3714.001.patch, YARN-3714.002.patch,
> YARN-3714.003.patch
>
>
> Default proxy address could not be got without setting
> {{yarn.resourcemanager.webapp.address._rm-id_}} and/or
> {{yarn.resourcemanager.webapp.https.address._rm-id_}} explicitly if RM-HA is
> enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3714) AM proxy filter can not get proper default proxy address if RM-HA is enabled
[
https://issues.apache.org/jira/browse/YARN-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14584099#comment-14584099
]
Masatake Iwasaki commented on YARN-3714:
Thanks for the comment, [~xgong]. {{HAUtil.verifyAndSetConfiguration}} works
only on the RM node. AMs running in slave nodes also need to know the RM webapp
addresses.
> AM proxy filter can not get proper default proxy address if RM-HA is enabled
>
>
> Key: YARN-3714
> URL: https://issues.apache.org/jira/browse/YARN-3714
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Attachments: YARN-3714.001.patch
>
>
> Default proxy address could not be got without setting
> {{yarn.resourcemanager.webapp.address._rm-id_}} and/or
> {{yarn.resourcemanager.webapp.https.address._rm-id_}} explicitly if RM-HA is
> enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3714) AM proxy filter can not get proper default proxy address if RM-HA is enabled
[
https://issues.apache.org/jira/browse/YARN-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14583890#comment-14583890
]
Xuan Gong commented on YARN-3714:
-
[~iwasakims] Did you see this issue in a real cluster environment ? As far as I
know, when we start RM, and we only set up yarn.resourcemanager.hostname.rm-id,
we will set this for all the service address, including
yarn.resourcemanager.webapp.address.rm-id and
yarn.resourcemanager.webapp.https.address.rm-id.
{code}
// Set HA configuration should be done before login
this.rmContext.setHAEnabled(HAUtil.isHAEnabled(this.conf));
if (this.rmContext.isHAEnabled()) {
HAUtil.verifyAndSetConfiguration(this.conf);
}
{code}
> AM proxy filter can not get proper default proxy address if RM-HA is enabled
>
>
> Key: YARN-3714
> URL: https://issues.apache.org/jira/browse/YARN-3714
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Attachments: YARN-3714.001.patch
>
>
> Default proxy address could not be got without setting
> {{yarn.resourcemanager.webapp.address._rm-id_}} and/or
> {{yarn.resourcemanager.webapp.https.address._rm-id_}} explicitly if RM-HA is
> enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3714) AM proxy filter can not get proper default proxy address if RM-HA is enabled
[
https://issues.apache.org/jira/browse/YARN-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560658#comment-14560658
]
Hadoop QA commented on YARN-3714:
-
\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch | 14m 38s | Pre-patch trunk compilation is
healthy. |
| {color:green}+1{color} | @author | 0m 0s | The patch does not contain any
@author tags. |
| {color:green}+1{color} | tests included | 0m 0s | The patch appears to
include 1 new or modified test files. |
| {color:green}+1{color} | javac | 7m 32s | There were no new javac warning
messages. |
| {color:green}+1{color} | javadoc | 9m 38s | There were no new javadoc
warning messages. |
| {color:green}+1{color} | release audit | 0m 22s | The applied patch does
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle | 1m 15s | There were no new checkstyle
issues. |
| {color:red}-1{color} | whitespace | 0m 0s | The patch has 1 line(s) that
end in whitespace. Use git apply --whitespace=fix. |
| {color:green}+1{color} | install | 1m 33s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with
eclipse:eclipse. |
| {color:green}+1{color} | findbugs | 1m 58s | The patch does not introduce
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | yarn tests | 1m 58s | Tests passed in
hadoop-yarn-common. |
| {color:green}+1{color} | yarn tests | 0m 22s | Tests passed in
hadoop-yarn-server-web-proxy. |
| | | 39m 53s | |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL |
http://issues.apache.org/jira/secure/attachment/12735545/YARN-3714.001.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / cdbd66b |
| whitespace |
https://builds.apache.org/job/PreCommit-YARN-Build/8097/artifact/patchprocess/whitespace.txt
|
| hadoop-yarn-common test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8097/artifact/patchprocess/testrun_hadoop-yarn-common.txt
|
| hadoop-yarn-server-web-proxy test log |
https://builds.apache.org/job/PreCommit-YARN-Build/8097/artifact/patchprocess/testrun_hadoop-yarn-server-web-proxy.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/8097/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/8097/console |
This message was automatically generated.
> AM proxy filter can not get proper default proxy address if RM-HA is enabled
>
>
> Key: YARN-3714
> URL: https://issues.apache.org/jira/browse/YARN-3714
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
> Attachments: YARN-3714.001.patch
>
>
> Default proxy address could not be got without setting
> {{yarn.resourcemanager.webapp.address._rm-id_}} and/or
> {{yarn.resourcemanager.webapp.https.address._rm-id_}} explicitly if RM-HA is
> enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3714) AM proxy filter can not get proper default proxy address if RM-HA is enabled
[
https://issues.apache.org/jira/browse/YARN-3714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560500#comment-14560500
]
Masatake Iwasaki commented on YARN-3714:
In non-HA settings, if users do not explicitly set the
{{yarn.resourcemanager.webapp.address}} in configuration,
{{WebAppUtils#getResolvedRMWebAppURLWithoutScheme}} returns RM webapp address
based on the value of {{yarn.resourcemanager.hostname}} via the default value
set by yarn-default.xml.
{noformat}
yarn.resourcemanager.webapp.address
${yarn.resourcemanager.hostname}:8088
{noformat}
As a result, WebAppUtils#getProxyHostsAndPortsForAmFilter can return proper
proxy address.
This does not apply to {{yarn.resourcemanager.hostname._rm-id_}} in HA mode.
> AM proxy filter can not get proper default proxy address if RM-HA is enabled
>
>
> Key: YARN-3714
> URL: https://issues.apache.org/jira/browse/YARN-3714
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Masatake Iwasaki
>Assignee: Masatake Iwasaki
>Priority: Minor
>
> Default proxy address could not be got without setting
> {{yarn.resourcemanager.webapp.address._rm-id_}} and/or
> {{yarn.resourcemanager.webapp.https.address._rm-id_}} explicitly if RM-HA is
> enabled.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
