[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189957#comment-16189957 ] Robert Kanter commented on YARN-7207: - [~aw], did you have any more comments? Otherwise, I think [~yufeigu] can commit this. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16184437#comment-16184437 ] Robert Kanter commented on YARN-7207: - I agree that it's a bad idea and ideally the user should fix that kind of misconfiguration. However, users don't always do what's best for them :) Even if this is covering up a host misconfiguration, if we have a simple fix to make Yarn not susceptible to the issue, I think that's a good thing. YARN-7263 should help warn the user of this misconfiguration, but really they should be using some other monitoring software they're hopefully running to alert them about it. In any case, it does seem unnecessary to constantly be doing this lookup when the value shouldn't change. Thousands of fewer calls sounds like a good thing to me. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183572#comment-16183572 ] Allen Wittenauer commented on YARN-7207: Actually, let me expand on that a bit, because we're running directly into "better practices" in a space that many may not understand the details. A process requests a host resolution of a name/ip that is associated with the machine that the process is running on (localhost, whatever hostname() returns, etc, etc). That resolution should be going through the local cache (nscd, sssd, lookupd, whatever). That cache should be configured such that it resolves through files (e.g., /etc/hosts) and then through DNS. /etc/hosts SHOULD have all known names and IPs for the local machine, eliminating the need for any DNS lookup. A misconfigured machine will either by not having a cache or having the cache misconfigured ask DNS or some other naming service first. This will *definitely* impact system performance. But it's also a misconfiguration; this won't just impact YARN but pretty much every single process on the box. Need to write to syslog? Yup, gonna ask DNS > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183566#comment-16183566 ] Allen Wittenauer commented on YARN-7207: bq. Single call of getLocalHost is pretty slow due to some DNS issue DNS calls for localhost shouldn't happen on a properly configured machine. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183554#comment-16183554 ] Yufei Gu commented on YARN-7207: There are different symptoms. Single call of getLocalHost is pretty slow due to some DNS issue, which may cause issue in different components, not only in RM. I don't think this change will hide anything. It is still observable in that situation. There is one case we observed is that performance of single call is fine, you won't notice the slowness at all, then it gets significant slow while you call it thousands of time with multiple threads. This change helps a lot in that case. Yes, this patch hides the symptom in that case. To solve it, I filed YARN-7263. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183527#comment-16183527 ] Allen Wittenauer commented on YARN-7207: If resolving the local hostname is slow, then that's a symptom of a misconfigured host. e.g., putting dns before files in nsswitch. Are we actually helping the user by hiding it? > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183314#comment-16183314 ] Yufei Gu commented on YARN-7207: File YARN-7263 for #2. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16182871#comment-16182871 ] Robert Kanter commented on YARN-7207: - +1 LGTM > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180579#comment-16180579 ] Hadoop QA commented on YARN-7207: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 44s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 33s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 1 new + 357 unchanged - 0 fixed = 358 total (was 357) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 50m 28s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 78m 14s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | | | hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore | | | hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSAppStarvation | | Timed out junit tests | org.apache.hadoop.yarn.server.resourcemanager.TestRMStoreCommands | | | org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-7207 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12889025/YARN-7207.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux df705a0d0c2d 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / d08b8c8 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17637/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/17637/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/j
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16180473#comment-16180473 ] Yufei Gu commented on YARN-7207: Thanks [~rkanter] for the review. Uploaded patch v2 for your comments. 1. Fixed 2. To do that, we need to pass a {{Configuration}} object into constructor of {{RMContextImpl}}. That does't look good to me since {{ResourceManager#serviceInit()}} intentionally set YARN configuration to rmContext after validation of configuration. In addition, to check the slowness of {{getLocalHost()}} needs multithread loops and each loop calls {{getLocalHost()}} thousands of times. We probably want to do that separately any way. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch, YARN-7207.002.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16179957#comment-16179957 ] Robert Kanter commented on YARN-7207: - Looks good overall, a few comments: # In {{TestRMAppAttemptTransitions}}, it's fine to replace the duplicate code with a call to {{getAppProxyUrl}}, but the original code here had an {{Assert.fail()}} when there was a {{URISyntaxException}}. Now, that's ignored and it returns "N/A". We should have the test code do something like this to maintain that check: {code:java} String url = rmContext.getAppProxyUrl(conf, appAttempt.getAppAttemptId().getApplicationId()); Assert.assertNotEquals("N/A", url); {code} # Instead of calling {{getProxyHostAndPort}} every time we call {{getAppProxyUrl}}, perhaps it would be better to simply populate the value of {{proxyHostAndPort}} in the constructor for {{RMContextImpl}}? {{getAppProxyUrl}} could then use the variable directly, and we don't have to worry about any race conditions. #- This would also make it very easy to add a warning message about the host name being slow because we could just do it when populating {{proxyHostAndPort}}, which should happen during startup when the {{RMContextImpl}} is being created. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16178721#comment-16178721 ] Yufei Gu commented on YARN-7207: What did I do in my first patch? # Cache the RM proxy server host name and port globally # Remove duplicate code to get RM proxy server host name and port I didn't put the pre-check into the patch. We could put it in another JIRA. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > Attachments: YARN-7207.001.patch > > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174213#comment-16174213 ] Hadoop QA commented on YARN-7207: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 10s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 31s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 2s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 32s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 28s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager: The patch generated 4 new + 359 unchanged - 0 fixed = 363 total (was 359) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 45m 30s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 19s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 67m 58s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation | | Timed out junit tests | org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-7207 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12888188/YARN-7207.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 7d6d306b9e59 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 53047f9 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17563/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/17563/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17563/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-reso
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16169636#comment-16169636 ] Yufei Gu commented on YARN-7207: Thanks [~miklos.szeg...@cloudera.com], nice suggestion! > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7207) Cache the local host name when getting application list in RM
[ https://issues.apache.org/jira/browse/YARN-7207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16168732#comment-16168732 ] Miklos Szegedi commented on YARN-7207: -- Thank you [~yufeigu] for reporting this. Please also run a pre-check host name resolution when resource manager starts up and if it is very slow, it would be nice to place a warning message into the logs. > Cache the local host name when getting application list in RM > - > > Key: YARN-7207 > URL: https://issues.apache.org/jira/browse/YARN-7207 > Project: Hadoop YARN > Issue Type: Improvement > Components: RM >Affects Versions: 3.1.0 >Reporter: Yufei Gu >Assignee: Yufei Gu > > {{getLocalHostName()}} is invoked for generating the report for each > application, which means it is called 1000 times for each > {{getApplications()}} if there are 1000 apps in RM. Some user got a > performance issue when {{getLocalHostName()}} is slow under some network envs. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org