[jira] [Commented] (YARN-10251) Show extended resources on legacy RM UI.
[
https://issues.apache.org/jira/browse/YARN-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146731#comment-17146731
]
Hadoop QA commented on YARN-10251:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m
47s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
14s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
44s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
52s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
21s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} branch-3.2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
57s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
12s{color} | {color:green} branch-3.2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 2m
23s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 52s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 new +
44 unchanged - 0 fixed = 45 total (was 44) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 1s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
31s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}337m 2s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
32s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}428m 46s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.server.resourcemanager.TestAMAuthorization |
| | hadoop.yarn.server.resourcemanager.TestRM |
| | hadoop.yarn.server.resourcemanager.TestNodeBlacklistingOnAMFailures |
| | hadoop.yarn.server.resourcemanager.resource.TestResourceProfiles |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterService |
| |
[jira] [Commented] (YARN-9903) Support reservations continue looking for Node Labels
[
https://issues.apache.org/jira/browse/YARN-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146725#comment-17146725
]
Hadoop QA commented on YARN-9903:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m
36s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
1s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 30m
4s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
49s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
46s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
51s{color} | {color:green} branch-3.2 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 4s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
38s{color} | {color:green} branch-3.2 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
47s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} branch-3.2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 41s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
49s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}455m 0s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
33s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}540m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.nodelabels.TestRMNodeLabelsManager |
| |
hadoop.yarn.server.resourcemanager.scheduler.TestSchedulingWithAllocationRequestId
|
| | hadoop.yarn.server.resourcemanager.TestRM |
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestQueueParsing |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
|
| | hadoop.yarn.server.resourcemanager.scheduler.capacity.TestChildQueueOrder
|
| |
hadoop.yarn.server.resourcemanager.metrics.TestCombinedSystemMetricsPublisher |
| |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
|
| | hadoop.yarn.server.resourcemanager.placement.TestPlacementManager |
| | hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl |
| | hadoop.yarn.server.resourcemanager.TestClientRMService |
| |
[jira] [Commented] (YARN-10229) [Federation] Client should be able to submit application to RM directly using normal client conf
[ https://issues.apache.org/jira/browse/YARN-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146714#comment-17146714 ] Young Chen commented on YARN-10229: --- Thanks for the update [~BilwaST]! "In that case can we add this logic inside AMRMProxyService#processApplicationStartRequest. Starting of this method we can check this and return if its not submitted to Router. Thoughts?" ^ This approach sounds good to me. With this change I think ContainerManagerImpl can avoid having any changes at all. Other than that, the patch looks good to me! > [Federation] Client should be able to submit application to RM directly using > normal client conf > > > Key: YARN-10229 > URL: https://issues.apache.org/jira/browse/YARN-10229 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy, federation >Affects Versions: 3.1.1 >Reporter: JohnsonGuo >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10229.001.patch, YARN-10229.002.patch, > YARN-10229.003.patch, YARN-10229.004.patch, YARN-10229.005.patch, > YARN-10229.006.patch > > > Scenario： When enable the yarn federation feature with multi yarn clusters, > one can submit their job to yarn-router by *modified* their client > configuration with yarn router address. > But if one still wants to submit their jobs via the original client (before > enable federation) to RM directly, it will encounter the AMRMToken exception. > That means once enable federation ,if some one want to submit job, they have > to modify the client conf. > > one possible solution for this Scenario is: > In NodeManger, when the client ApplicationMaster request comes: > * get the client job.xml from HDFS "". > * parse the "yarn.resourcemanager.scheduler.address" parameter in job.xml > * if the value of the parameter is "localhost:8049"(AMRM address),then do > the AMRMToken valid process > * if the value of the parameter is "rm:port"(rm address),then skip the > AMRMToken valid process > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10328) Too many ZK Curator NodeExists exception logs in YARN Service AM logs
[
https://issues.apache.org/jira/browse/YARN-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146625#comment-17146625
]
Hadoop QA commented on YARN-10328:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 0m
57s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 36s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 19m
57s{color} | {color:green} hadoop-yarn-services-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
38s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 86m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://builds.apache.org/job/PreCommit-YARN-Build/26226/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10328 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13006539/YARN-10328.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 65cdf16dca88 4.15.0-91-generic #92-Ubuntu SMP Fri Feb 28
11:09:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 2c03524fa4b |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/26226/testReport/ |
| Max. process+thread count | 779 (vs. ulimit of 5500) |
| modules |
[jira] [Updated] (YARN-10251) Show extended resources on legacy RM UI.
[ https://issues.apache.org/jira/browse/YARN-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-10251: -- Attachment: YARN-10251.branch-3.2.006.patch > Show extended resources on legacy RM UI. > > > Key: YARN-10251 > URL: https://issues.apache.org/jira/browse/YARN-10251 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Eric Payne >Assignee: Eric Payne >Priority: Major > Attachments: Legacy RM UI With Not All Resources Shown.png, Updated > NodesPage UI With GPU columns.png, Updated RM UI With All Resources > Shown.png.png, YARN-10251.003.patch, YARN-10251.004.patch, > YARN-10251.005.patch, YARN-10251.006.patch, YARN-10251.branch-2.10.001.patch, > YARN-10251.branch-2.10.002.patch, YARN-10251.branch-2.10.003.patch, > YARN-10251.branch-2.10.005.patch, YARN-10251.branch-3.2.004.patch, > YARN-10251.branch-3.2.005.patch, YARN-10251.branch-3.2.006.patch > > > It would be great to update the legacy RM UI to include GPU resources in the > overview and in the per-app sections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10251) Show extended resources on legacy RM UI.
[
https://issues.apache.org/jira/browse/YARN-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146567#comment-17146567
]
Hadoop QA commented on YARN-10251:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
23s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 57s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
1s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
21s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
4s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 2s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 new +
46 unchanged - 0 fixed = 47 total (was 46) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 13s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
31s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
40s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 97m 12s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
28s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}181m 22s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.security.TestDelegationTokenRenewer |
| |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://builds.apache.org/job/PreCommit-YARN-Build/26222/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10251 |
| JIRA Patch URL |
[jira] [Commented] (YARN-10251) Show extended resources on legacy RM UI.
[
https://issues.apache.org/jira/browse/YARN-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146566#comment-17146566
]
Hadoop QA commented on YARN-10251:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
45s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 23m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
18m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
0s{color} | {color:blue} Used deprecated FindBugs config; considering switching
to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
10s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m
6s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 6s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server: The patch generated 1 new +
46 unchanged - 0 fixed = 47 total (was 46) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 26s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
37s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
49s{color} | {color:green} hadoop-yarn-server-common in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 96m
51s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}180m 16s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://builds.apache.org/job/PreCommit-YARN-Build/26221/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10251 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13006466/YARN-10251.006.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
|
[jira] [Commented] (YARN-10278) CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework need some review
[ https://issues.apache.org/jira/browse/YARN-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146565#comment-17146565 ] Eric Payne commented on YARN-10278: --- Yes. Here's my +1. I'll commit on Monday. > CapacityScheduler test framework > ProportionalCapacityPreemptionPolicyMockFramework need some review > --- > > Key: YARN-10278 > URL: https://issues.apache.org/jira/browse/YARN-10278 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-10278.001.patch, YARN-10278.002.patch > > > This test framework class mocks a bit too heavily, and simulates CS internal > behaviour with the mock methods over a point it is reasonably maintainable, > any internal change in CS is a major headscratch. > A lot of tests depend on this class, so we should approach it carefully, but > I think it's wroth to examine this class if it can be made a bit more > resilient to changes, and easier to maintain. Or at least document it better. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10278) CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework need some review
[ https://issues.apache.org/jira/browse/YARN-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146563#comment-17146563 ] Szilard Nemeth commented on YARN-10278: --- Thanks [~epayne] for rhe confirmation. Given that [~shuzirra] gave a non binding +1 and you also said it looks good for you, can you please commit this? Thanks. > CapacityScheduler test framework > ProportionalCapacityPreemptionPolicyMockFramework need some review > --- > > Key: YARN-10278 > URL: https://issues.apache.org/jira/browse/YARN-10278 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-10278.001.patch, YARN-10278.002.patch > > > This test framework class mocks a bit too heavily, and simulates CS internal > behaviour with the mock methods over a point it is reasonably maintainable, > any internal change in CS is a major headscratch. > A lot of tests depend on this class, so we should approach it carefully, but > I think it's wroth to examine this class if it can be made a bit more > resilient to changes, and easier to maintain. Or at least document it better. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10278) CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework need some review
[ https://issues.apache.org/jira/browse/YARN-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146561#comment-17146561 ] Eric Payne commented on YARN-10278: --- bq. I think I would target branch-3.3 along with trunk. That sounds fine. > CapacityScheduler test framework > ProportionalCapacityPreemptionPolicyMockFramework need some review > --- > > Key: YARN-10278 > URL: https://issues.apache.org/jira/browse/YARN-10278 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-10278.001.patch, YARN-10278.002.patch > > > This test framework class mocks a bit too heavily, and simulates CS internal > behaviour with the mock methods over a point it is reasonably maintainable, > any internal change in CS is a major headscratch. > A lot of tests depend on this class, so we should approach it carefully, but > I think it's wroth to examine this class if it can be made a bit more > resilient to changes, and easier to maintain. Or at least document it better. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10229) [Federation] Client should be able to submit application to RM directly using normal client conf
[
https://issues.apache.org/jira/browse/YARN-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146552#comment-17146552
]
Hadoop QA commented on YARN-10229:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
40s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 12s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
18s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager:
The patch generated 0 new + 166 unchanged - 1 fixed = 166 total (was 167)
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 52s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 22m
25s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
31s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 80m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://builds.apache.org/job/PreCommit-YARN-Build/26223/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10229 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13006537/YARN-10229.006.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 539c2cf1a226 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 2c03524fa4b |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/26223/testReport/ |
| Max. process+thread count | 414 (vs. ulimit of
[jira] [Commented] (YARN-9809) NMs should supply a health status when registering with RM
[ https://issues.apache.org/jira/browse/YARN-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146522#comment-17146522 ] Eric Badger commented on YARN-9809: --- Thanks, [~Jim_Brennan]! [~eyang], would you take another look? > NMs should supply a health status when registering with RM > -- > > Key: YARN-9809 > URL: https://issues.apache.org/jira/browse/YARN-9809 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Eric Badger >Assignee: Eric Badger >Priority: Major > Attachments: YARN-9809.001.patch, YARN-9809.002.patch, > YARN-9809.003.patch, YARN-9809.004.patch, YARN-9809.005.patch, > YARN-9809.006.patch, YARN-9809.007.patch > > > Currently if the NM registers with the RM and it is unhealthy, it can be > scheduled many containers before the first heartbeat. After the first > heartbeat, the RM will mark the NM as unhealthy and kill all of the > containers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9903) Support reservations continue looking for Node Labels
[
https://issues.apache.org/jira/browse/YARN-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146497#comment-17146497
]
Jim Brennan commented on YARN-9903:
---
[~epayne] I have uploaded a patch for branch-3.2 and I have verified that it
applies and builds cleanly all the way back to branch-2.10.
> Support reservations continue looking for Node Labels
> -
>
> Key: YARN-9903
> URL: https://issues.apache.org/jira/browse/YARN-9903
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Tarun Parimi
>Assignee: Jim Brennan
>Priority: Major
> Attachments: YARN-9903-branch-3.2.001.patch, YARN-9903.001.patch,
> YARN-9903.002.patch, YARN-9903.003.patch, YARN-9903.004.patch
>
>
> YARN-1769 brought in reservations continue looking feature which improves the
> several resource reservation scenarios. However, it is not handled currently
> when nodes have a label assigned to them. This is useful and in many cases
> necessary even for Node Labels. So we should look to support this for node
> labels also.
> For example, in AbstractCSQueue.java, we have the below TODO.
> {code:java}
> // TODO, now only consider reservation cases when the node has no label
> if (this.reservationsContinueLooking && nodePartition.equals(
> RMNodeLabelsManager.NO_LABEL) && Resources.greaterThan( resourceCalculator,
> clusterResource, resourceCouldBeUnreserved, Resources.none())) {
> {code}
> cc [~sunilg]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10328) Too many ZK Curator NodeExists exception logs in YARN Service AM logs
[
https://issues.apache.org/jira/browse/YARN-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146485#comment-17146485
]
Bilwa S T commented on YARN-10328:
--
cc [~eyang]
> Too many ZK Curator NodeExists exception logs in YARN Service AM logs
> -
>
> Key: YARN-10328
> URL: https://issues.apache.org/jira/browse/YARN-10328
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: YARN-10328.001.patch
>
>
> Following debug logs are printed everytime when component is started.
> {code:java}
> [pool-6-thread-3] DEBUG zk.CuratorService - path already present:
> /registry/users/server/services/yarn-service/default-worker/components
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for
> /registry/users/hetuserver/services/yarn-service/default-worker/components
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:128)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1480)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:740)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:723)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:720)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:484)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:474)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:454)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44)
> at
> org.apache.hadoop.registry.client.impl.zk.CuratorService.zkMkPath(CuratorService.java:587)
> at
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.mknode(RegistryOperationsService.java:99)
> at
> org.apache.hadoop.yarn.service.registry.YarnRegistryViewForProviders.putComponent(YarnRegistryViewForProviders.java:146)
> at
> org.apache.hadoop.yarn.service.registry.YarnRegistryViewForProviders.putComponent(YarnRegistryViewForProviders.java:128)
> at
> org.apache.hadoop.yarn.service.component.instance.ComponentInstance.updateServiceRecord(ComponentInstance.java:511)
> at
> org.apache.hadoop.yarn.service.component.instance.ComponentInstance.updateContainerStatus(ComponentInstance.java:449)
> at
> org.apache.hadoop.yarn.service.component.instance.ComponentInstance$ContainerStatusRetriever.run(ComponentInstance.java:620)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10328) Too many ZK Curator NodeExists exception logs in YARN Service AM logs
[
https://issues.apache.org/jira/browse/YARN-10328?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bilwa S T updated YARN-10328:
-
Attachment: YARN-10328.001.patch
> Too many ZK Curator NodeExists exception logs in YARN Service AM logs
> -
>
> Key: YARN-10328
> URL: https://issues.apache.org/jira/browse/YARN-10328
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bilwa S T
>Assignee: Bilwa S T
>Priority: Major
> Attachments: YARN-10328.001.patch
>
>
> Following debug logs are printed everytime when component is started.
> {code:java}
> [pool-6-thread-3] DEBUG zk.CuratorService - path already present:
> /registry/users/server/services/yarn-service/default-worker/components
> org.apache.zookeeper.KeeperException$NodeExistsException: KeeperErrorCode =
> NodeExists for
> /registry/users/hetuserver/services/yarn-service/default-worker/components
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:128)
> at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
> at org.apache.zookeeper.ZooKeeper.create(ZooKeeper.java:1480)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:740)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl$11.call(CreateBuilderImpl.java:723)
> at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:109)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.pathInForeground(CreateBuilderImpl.java:720)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.protectedPathInForeground(CreateBuilderImpl.java:484)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:474)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:454)
> at
> org.apache.curator.framework.imps.CreateBuilderImpl.forPath(CreateBuilderImpl.java:44)
> at
> org.apache.hadoop.registry.client.impl.zk.CuratorService.zkMkPath(CuratorService.java:587)
> at
> org.apache.hadoop.registry.client.impl.zk.RegistryOperationsService.mknode(RegistryOperationsService.java:99)
> at
> org.apache.hadoop.yarn.service.registry.YarnRegistryViewForProviders.putComponent(YarnRegistryViewForProviders.java:146)
> at
> org.apache.hadoop.yarn.service.registry.YarnRegistryViewForProviders.putComponent(YarnRegistryViewForProviders.java:128)
> at
> org.apache.hadoop.yarn.service.component.instance.ComponentInstance.updateServiceRecord(ComponentInstance.java:511)
> at
> org.apache.hadoop.yarn.service.component.instance.ComponentInstance.updateContainerStatus(ComponentInstance.java:449)
> at
> org.apache.hadoop.yarn.service.component.instance.ComponentInstance$ContainerStatusRetriever.run(ComponentInstance.java:620)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9903) Support reservations continue looking for Node Labels
[
https://issues.apache.org/jira/browse/YARN-9903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jim Brennan updated YARN-9903:
--
Attachment: YARN-9903-branch-3.2.001.patch
> Support reservations continue looking for Node Labels
> -
>
> Key: YARN-9903
> URL: https://issues.apache.org/jira/browse/YARN-9903
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Tarun Parimi
>Assignee: Jim Brennan
>Priority: Major
> Attachments: YARN-9903-branch-3.2.001.patch, YARN-9903.001.patch,
> YARN-9903.002.patch, YARN-9903.003.patch, YARN-9903.004.patch
>
>
> YARN-1769 brought in reservations continue looking feature which improves the
> several resource reservation scenarios. However, it is not handled currently
> when nodes have a label assigned to them. This is useful and in many cases
> necessary even for Node Labels. So we should look to support this for node
> labels also.
> For example, in AbstractCSQueue.java, we have the below TODO.
> {code:java}
> // TODO, now only consider reservation cases when the node has no label
> if (this.reservationsContinueLooking && nodePartition.equals(
> RMNodeLabelsManager.NO_LABEL) && Resources.greaterThan( resourceCalculator,
> clusterResource, resourceCouldBeUnreserved, Resources.none())) {
> {code}
> cc [~sunilg]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10229) [Federation] Client should be able to submit application to RM directly using normal client conf
[ https://issues.apache.org/jira/browse/YARN-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146473#comment-17146473 ] Bilwa S T commented on YARN-10229: -- Hi [~youchen] I have attached a patch. Please take a look at it > [Federation] Client should be able to submit application to RM directly using > normal client conf > > > Key: YARN-10229 > URL: https://issues.apache.org/jira/browse/YARN-10229 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy, federation >Affects Versions: 3.1.1 >Reporter: JohnsonGuo >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10229.001.patch, YARN-10229.002.patch, > YARN-10229.003.patch, YARN-10229.004.patch, YARN-10229.005.patch, > YARN-10229.006.patch > > > Scenario： When enable the yarn federation feature with multi yarn clusters, > one can submit their job to yarn-router by *modified* their client > configuration with yarn router address. > But if one still wants to submit their jobs via the original client (before > enable federation) to RM directly, it will encounter the AMRMToken exception. > That means once enable federation ,if some one want to submit job, they have > to modify the client conf. > > one possible solution for this Scenario is: > In NodeManger, when the client ApplicationMaster request comes: > * get the client job.xml from HDFS "". > * parse the "yarn.resourcemanager.scheduler.address" parameter in job.xml > * if the value of the parameter is "localhost:8049"(AMRM address),then do > the AMRMToken valid process > * if the value of the parameter is "rm:port"(rm address),then skip the > AMRMToken valid process > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10229) [Federation] Client should be able to submit application to RM directly using normal client conf
[ https://issues.apache.org/jira/browse/YARN-10229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bilwa S T updated YARN-10229: - Attachment: YARN-10229.006.patch > [Federation] Client should be able to submit application to RM directly using > normal client conf > > > Key: YARN-10229 > URL: https://issues.apache.org/jira/browse/YARN-10229 > Project: Hadoop YARN > Issue Type: Bug > Components: amrmproxy, federation >Affects Versions: 3.1.1 >Reporter: JohnsonGuo >Assignee: Bilwa S T >Priority: Major > Attachments: YARN-10229.001.patch, YARN-10229.002.patch, > YARN-10229.003.patch, YARN-10229.004.patch, YARN-10229.005.patch, > YARN-10229.006.patch > > > Scenario： When enable the yarn federation feature with multi yarn clusters, > one can submit their job to yarn-router by *modified* their client > configuration with yarn router address. > But if one still wants to submit their jobs via the original client (before > enable federation) to RM directly, it will encounter the AMRMToken exception. > That means once enable federation ,if some one want to submit job, they have > to modify the client conf. > > one possible solution for this Scenario is: > In NodeManger, when the client ApplicationMaster request comes: > * get the client job.xml from HDFS "". > * parse the "yarn.resourcemanager.scheduler.address" parameter in job.xml > * if the value of the parameter is "localhost:8049"(AMRM address),then do > the AMRMToken valid process > * if the value of the parameter is "rm:port"(rm address),then skip the > AMRMToken valid process > > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10251) Show extended resources on legacy RM UI.
[
https://issues.apache.org/jira/browse/YARN-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146410#comment-17146410
]
Eric Payne commented on YARN-10251:
---
bq. Not sure why the precommit build did not run for patch 006?
{noformat}
YARN-10251 patch is being downloaded at Fri Jun 26 15:32:50 UTC 2020 from
https://issues.apache.org/jira/secure/attachment/13006467/Screen%20Shot%202020-06-25%20at%203.40.06%20PM.png
-> Downloaded
ERROR: Unsure how to process YARN-10251.
{noformat}
It's trying to treat the screenshot as a patch. I'll delete it and try again.
> Show extended resources on legacy RM UI.
>
>
> Key: YARN-10251
> URL: https://issues.apache.org/jira/browse/YARN-10251
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Major
> Attachments: Legacy RM UI With Not All Resources Shown.png, Updated
> NodesPage UI With GPU columns.png, Updated RM UI With All Resources
> Shown.png.png, YARN-10251.003.patch, YARN-10251.004.patch,
> YARN-10251.005.patch, YARN-10251.006.patch, YARN-10251.branch-2.10.001.patch,
> YARN-10251.branch-2.10.002.patch, YARN-10251.branch-2.10.003.patch,
> YARN-10251.branch-2.10.005.patch, YARN-10251.branch-3.2.004.patch,
> YARN-10251.branch-3.2.005.patch
>
>
> It would be great to update the legacy RM UI to include GPU resources in the
> overview and in the per-app sections.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10251) Show extended resources on legacy RM UI.
[ https://issues.apache.org/jira/browse/YARN-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne updated YARN-10251: -- Attachment: (was: Screen Shot 2020-06-25 at 3.40.06 PM.png) > Show extended resources on legacy RM UI. > > > Key: YARN-10251 > URL: https://issues.apache.org/jira/browse/YARN-10251 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Eric Payne >Assignee: Eric Payne >Priority: Major > Attachments: Legacy RM UI With Not All Resources Shown.png, Updated > NodesPage UI With GPU columns.png, Updated RM UI With All Resources > Shown.png.png, YARN-10251.003.patch, YARN-10251.004.patch, > YARN-10251.005.patch, YARN-10251.006.patch, YARN-10251.branch-2.10.001.patch, > YARN-10251.branch-2.10.002.patch, YARN-10251.branch-2.10.003.patch, > YARN-10251.branch-2.10.005.patch, YARN-10251.branch-3.2.004.patch, > YARN-10251.branch-3.2.005.patch > > > It would be great to update the legacy RM UI to include GPU resources in the > overview and in the per-app sections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10251) Show extended resources on legacy RM UI.
[
https://issues.apache.org/jira/browse/YARN-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145864#comment-17145864
]
Eric Payne edited comment on YARN-10251 at 6/26/20, 3:37 PM:
-
Thanks very much for the review, [~Jim_Brennan]. I attached patch 006 for
trunk. I'll work on getting the backport patches up later.
{quote}AppInfo
- (nit) move the initialization of usageReport before the if statement so you
can use it throughout that condition. Makes the diff a little bigger, but I
think it’s worth it in this case.{quote}
For the sake of sparing the reviewers, I don't generally like to make changes
that refactor the code if they are not directly related to the bug fix/feature.
But since you suggested it and are on board, I did so in this case.
{quote}NodesPage
- The new entries for GPUs Used and GPUs Avail are still using ".vcores” for
the set selector. Is this correct? Shouldn't we use ".gpus"?
{code:java}
.th(".vcores", "GPUs Used")
.th(".vcores", "GPUs Avail");
{code}
{quote}
OOPS! Good catch.
{quote}RmAppsBlock
- I'm not sure I understand this logic:{quote}
In the AppsBlock on the RM UI, the Allocated and Reserved columns contain a
number if the app is not completed and "N/A" if the app is completed.
When {{AppInfo}} retrieves the {{ApplicationResourceUsageReport}}, this
behavior is handle automatically because the memory and vcores field is -1 if
the app is completed and has a number otherwise. However, the extended
resources can be 0 if there are no resources of that type used, or it will be 0
if the app has completed, not -1. I felt that it is less risky to put the
additional check for completed app in RMAppsBlock rather than try to change the
behavior in the retrieval of {{ApplicationResourceUsageReport}}.
was (Author: eepayne):
Thanks very much for the review, [~Jim_Brennan]. I attached patch 006 for
trunk. I'll work on getting the backport patches up later.
{quote}AppInfo
- (nit) move the initialization of usageReport before the if statement so you
can use it throughout that condition. Makes the diff a little bigger, but I
think it’s worth it in this case.{quote}
For the sake of sparing the reviewers, I don't generally like to make changes
that refactor the code if they are not directly related to the bug fix/feature.
But since you suggested it and are on board, I did so in this case.
{quote}NodesPage
- The new entries for GPUs Used and GPUs Avail are still using ".vcores” for
the set selector. Is this correct? Shouldn't we use ".gpus"?
{code:java}
.th(".vcores", "GPUs Used")
.th(".vcores", "GPUs Avail");
{code}
{quote}
OOPS! Good catch.
{quote}RmAppsBlock
- I'm not sure I understand this logic:{quote}
In the AppsBlock on the RM UI, the Allocated and Reserved columns contain a
number if the app is not completed and "N/A" if the app is completed.
!Screen Shot 2020-06-25 at 3.40.06 PM.png!
When {{AppInfo}} retrieves the {{ApplicationResourceUsageReport}}, this
behavior is handle automatically because the memory and vcores field is -1 if
the app is completed and has a number otherwise. However, the extended
resources can be 0 if there are no resources of that type used, or it will be 0
if the app has completed, not -1. I felt that it is less risky to put the
additional check for completed app in RMAppsBlock rather than try to change the
behavior in the retrieval of {{ApplicationResourceUsageReport}}.
> Show extended resources on legacy RM UI.
>
>
> Key: YARN-10251
> URL: https://issues.apache.org/jira/browse/YARN-10251
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Eric Payne
>Assignee: Eric Payne
>Priority: Major
> Attachments: Legacy RM UI With Not All Resources Shown.png, Updated
> NodesPage UI With GPU columns.png, Updated RM UI With All Resources
> Shown.png.png, YARN-10251.003.patch, YARN-10251.004.patch,
> YARN-10251.005.patch, YARN-10251.006.patch, YARN-10251.branch-2.10.001.patch,
> YARN-10251.branch-2.10.002.patch, YARN-10251.branch-2.10.003.patch,
> YARN-10251.branch-2.10.005.patch, YARN-10251.branch-3.2.004.patch,
> YARN-10251.branch-3.2.005.patch
>
>
> It would be great to update the legacy RM UI to include GPU resources in the
> overview and in the per-app sections.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6526) Refactoring SQLFederationStateStore by avoiding to recreate a connection at every call
[ https://issues.apache.org/jira/browse/YARN-6526?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146407#comment-17146407 ] Hudson commented on YARN-6526: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18384 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18384/]) YARN-6526. Refactoring SQLFederationStateStore by avoiding to recreate a (brahma: rev 2c03524fa4be754aa95889d4ac0f5d57dca8cda8) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/utils/FederationStateStoreUtils.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/impl/SQLFederationStateStore.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/TestSQLFederationStateStore.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/HSQLDBFederationStateStore.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/TestZookeeperFederationStateStore.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/federation/store/metrics/FederationStateStoreClientMetrics.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/test/java/org/apache/hadoop/yarn/server/federation/store/impl/FederationStateStoreBaseTest.java > Refactoring SQLFederationStateStore by avoiding to recreate a connection at > every call > -- > > Key: YARN-6526 > URL: https://issues.apache.org/jira/browse/YARN-6526 > Project: Hadoop YARN > Issue Type: Sub-task > Components: federation >Reporter: Giovanni Matteo Fumarola >Assignee: Bilwa S T >Priority: Major > Fix For: 3.4.0 > > Attachments: YARN-6526.001.patch, YARN-6526.002.patch, > YARN-6526.003.patch, YARN-6526.004.patch, YARN-6526.005.patch, > YARN-6526.006.patch, YARN-6526.007.patch, YARN-6526.008.patch > > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9809) NMs should supply a health status when registering with RM
[ https://issues.apache.org/jira/browse/YARN-9809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146402#comment-17146402 ] Jim Brennan commented on YARN-9809: --- Thanks for the update [~ebadger]! I am +1 (non-binding) on patch 007. > NMs should supply a health status when registering with RM > -- > > Key: YARN-9809 > URL: https://issues.apache.org/jira/browse/YARN-9809 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Eric Badger >Assignee: Eric Badger >Priority: Major > Attachments: YARN-9809.001.patch, YARN-9809.002.patch, > YARN-9809.003.patch, YARN-9809.004.patch, YARN-9809.005.patch, > YARN-9809.006.patch, YARN-9809.007.patch > > > Currently if the NM registers with the RM and it is unhealthy, it can be > scheduled many containers before the first heartbeat. After the first > heartbeat, the RM will mark the NM as unhealthy and kill all of the > containers. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10251) Show extended resources on legacy RM UI.
[ https://issues.apache.org/jira/browse/YARN-10251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146397#comment-17146397 ] Jim Brennan commented on YARN-10251: Not sure why the precommit build did not run for patch 006? > Show extended resources on legacy RM UI. > > > Key: YARN-10251 > URL: https://issues.apache.org/jira/browse/YARN-10251 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Eric Payne >Assignee: Eric Payne >Priority: Major > Attachments: Legacy RM UI With Not All Resources Shown.png, Screen > Shot 2020-06-25 at 3.40.06 PM.png, Updated NodesPage UI With GPU columns.png, > Updated RM UI With All Resources Shown.png.png, YARN-10251.003.patch, > YARN-10251.004.patch, YARN-10251.005.patch, YARN-10251.006.patch, > YARN-10251.branch-2.10.001.patch, YARN-10251.branch-2.10.002.patch, > YARN-10251.branch-2.10.003.patch, YARN-10251.branch-2.10.005.patch, > YARN-10251.branch-3.2.004.patch, YARN-10251.branch-3.2.005.patch > > > It would be great to update the legacy RM UI to include GPU resources in the > overview and in the per-app sections. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10216) Utility to dynamically reload Configuration on the disk
[
https://issues.apache.org/jira/browse/YARN-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146359#comment-17146359
]
Hadoop QA commented on YARN-10216:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
16s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
5s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
12s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
22m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
46s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
48s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
11s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m
21s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 56s{color} | {color:orange} root: The patch generated 13 new + 289 unchanged
- 0 fixed = 302 total (was 289) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 38s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
44s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
16s{color} | {color:red} hadoop-common-project/hadoop-common generated 1 new +
0 unchanged - 0 fixed = 1 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 22s{color}
| {color:red} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 59s{color}
| {color:red} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
58s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 91m
53s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
48s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}256m 7s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-common-project/hadoop-common |
| | Inconsistent synchronization of
[jira] [Commented] (YARN-10216) Utility to dynamically reload Configuration on the disk
[
https://issues.apache.org/jira/browse/YARN-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146353#comment-17146353
]
Hadoop QA commented on YARN-10216:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
18s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 18m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
21m 47s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
21s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
52s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
27s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 17m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 17m
44s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 49s{color} | {color:orange} root: The patch generated 13 new + 289 unchanged
- 0 fixed = 302 total (was 289) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 16s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
12s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
25s{color} | {color:red} hadoop-common-project/hadoop-common generated 1 new +
0 unchanged - 0 fixed = 1 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m
41s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 9s{color}
| {color:red} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 22m
43s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 88m
24s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
55s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}253m 43s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-common-project/hadoop-common |
| | Inconsistent synchronization
[jira] [Commented] (YARN-10330) Add missing test scenarios to TestUserGroupMappingPlacementRule
[
https://issues.apache.org/jira/browse/YARN-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146311#comment-17146311
]
Hadoop QA commented on YARN-10330:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 2 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
49s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
47s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 33s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 4 unchanged - 1 fixed = 5 total (was 5) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
16m 43s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 92m 46s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}161m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://builds.apache.org/job/PreCommit-YARN-Build/26219/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10330 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13006496/YARN-10330-001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 3013fa394184 4.15.0-101-generic #102-Ubuntu SMP Mon May 11
10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 33d3df7ee2b |
| Default Java | Private Build-1.8.0_252-8u252-b09-1~18.04-b09 |
|
[jira] [Commented] (YARN-10277) CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy
[
https://issues.apache.org/jira/browse/YARN-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146267#comment-17146267
]
Hadoop QA commented on YARN-10277:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-3.3 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 32m
43s{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
48s{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} branch-3.3 passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 0s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
31s{color} | {color:green} branch-3.3 passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
45s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
41s{color} | {color:green} branch-3.3 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 30s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 12 unchanged - 0 fixed = 13 total (was 12) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 45s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}109m 10s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
34s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}186m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairSchedulerPreemption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | ClientAPI=1.40 ServerAPI=1.40 base:
https://builds.apache.org/job/PreCommit-YARN-Build/26216/artifact/out/Dockerfile
|
| JIRA Issue | YARN-10277 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/13006485/YARN-10277.branch-3.3.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 11f27a6d79e1 4.15.0-101-generic #102-Ubuntu SMP Mon May 11
10:07:26 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | branch-3.3 / fa41e38 |
| Default Java |
[jira] [Commented] (YARN-10278) CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework need some review
[ https://issues.apache.org/jira/browse/YARN-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146264#comment-17146264 ] Gergely Pollak commented on YARN-10278: --- Thank you [~snemeth] for the patch, it is already much better. There are still some improvements to do, but I think we should handle those cases in separate JIRAS. My only suggestion, which might be worth to include in this patch, is to remove the magic numbers which are used as indexes during the parsing. If we'd used constants instead of just numbers, we wouldn't need to look for the nearest comment to figure out the order of the arguments. But even without this change I think it's fine LGTM + 1 (Non-binding) > CapacityScheduler test framework > ProportionalCapacityPreemptionPolicyMockFramework need some review > --- > > Key: YARN-10278 > URL: https://issues.apache.org/jira/browse/YARN-10278 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-10278.001.patch, YARN-10278.002.patch > > > This test framework class mocks a bit too heavily, and simulates CS internal > behaviour with the mock methods over a point it is reasonably maintainable, > any internal change in CS is a major headscratch. > A lot of tests depend on this class, so we should approach it carefully, but > I think it's wroth to examine this class if it can be made a bit more > resilient to changes, and easier to maintain. Or at least document it better. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10327) Remove duplication of checking for invalid application ID in TestLogsCLI
[ https://issues.apache.org/jira/browse/YARN-10327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146251#comment-17146251 ] Hudson commented on YARN-10327: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18383 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18383/]) YARN-10327. Remove duplication of checking for invalid application ID in (snemeth: rev e0c1d8a96905bfbedbddde9000fc08ce2af1d277) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/cli/TestLogsCLI.java > Remove duplication of checking for invalid application ID in TestLogsCLI > > > Key: YARN-10327 > URL: https://issues.apache.org/jira/browse/YARN-10327 > Project: Hadoop YARN > Issue Type: Test >Reporter: Hudáky Márton Gyula >Assignee: Hudáky Márton Gyula >Priority: Trivial > Fix For: 3.4.0 > > Attachments: YARN-10327.001.patch > > > TestLogsCLI has a separate function to test for invalid application ID > (#testInvalidApplicationId) and another (#testLogsCLIWithInvalidArgs) to test > multiple invalid arguments (including application ID). One of them should be > eliminated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10318) ApplicationHistory Web UI incorrect column indexing
[ https://issues.apache.org/jira/browse/YARN-10318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146240#comment-17146240 ] Hudson commented on YARN-10318: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18382 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18382/]) YARN-10318. ApplicationHistory Web UI incorrect column indexing. (snemeth: rev 6459cc73e6c0878be51c0f2f70ec9868325dec34) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/AppsBlock.java > ApplicationHistory Web UI incorrect column indexing > --- > > Key: YARN-10318 > URL: https://issues.apache.org/jira/browse/YARN-10318 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Minor > Fix For: 3.4.0 > > Attachments: Screenshot 2020-06-25 at 10.15.32.png, > YARN-10318.001.patch, image-2020-06-16-17-14-55-921.png > > > The ApplicationHistory UI is broken due to an incorrect column indexing. This > bug was probably introduced in YARN-10038, which presumes, that the table > contains the application tag column (which is true for RM Web UI, but not for > AH Web UI). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9355) RMContainerRequestor#makeRemoteRequest has confusing log message
[
https://issues.apache.org/jira/browse/YARN-9355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146239#comment-17146239
]
Szilard Nemeth commented on YARN-9355:
--
Hi [~umittal],
Your patch does not include:
{code:java}
" ask="
+ ask.size()
{code}
The size of ask was printed previously.
I also think applicationId should be printed explicitly, e.g. "applicationId="
+ applicationId.
> RMContainerRequestor#makeRemoteRequest has confusing log message
>
>
> Key: YARN-9355
> URL: https://issues.apache.org/jira/browse/YARN-9355
> Project: Hadoop YARN
> Issue Type: Improvement
>Reporter: Szilard Nemeth
>Assignee: Umesh Mittal
>Priority: Trivial
> Labels: newbie, newbie++
> Attachments: YARN-9355.001.patch, YARN-9355.002.patch
>
>
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor#makeRemoteRequest
> has this log:
> {code:java}
> if (ask.size() > 0 || release.size() > 0) {
> LOG.info("getResources() for " + applicationId + ":" + " ask="
> + ask.size() + " release= " + release.size() + " newContainers="
> + allocateResponse.getAllocatedContainers().size()
> + " finishedContainers=" + numCompletedContainers
> + " resourcelimit=" + availableResources + " knownNMs="
> + clusterNmCount);
> }
> {code}
> The reason why "getResources()" is printed because
> org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator#getResources
> invokes makeRemoteRequest. This is not too informative and error-prone as
> name of getResources could change over time and the log will be outdated.
> Moreover, it's not a good idea to print a method name from a method below the
> current one in the stack.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10327) Remove duplication of checking for invalid application ID in TestLogsCLI
[ https://issues.apache.org/jira/browse/YARN-10327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146233#comment-17146233 ] Szilard Nemeth commented on YARN-10327: --- Thanks [~mhudaky] for working on this. Latest patch LGTM, committed to trunk. Thanks [~gandras] for the review. > Remove duplication of checking for invalid application ID in TestLogsCLI > > > Key: YARN-10327 > URL: https://issues.apache.org/jira/browse/YARN-10327 > Project: Hadoop YARN > Issue Type: Test >Reporter: Hudáky Márton Gyula >Assignee: Hudáky Márton Gyula >Priority: Trivial > Fix For: 3.4.0 > > Attachments: YARN-10327.001.patch > > > TestLogsCLI has a separate function to test for invalid application ID > (#testInvalidApplicationId) and another (#testLogsCLIWithInvalidArgs) to test > multiple invalid arguments (including application ID). One of them should be > eliminated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10327) Remove duplication of checking for invalid application ID in TestLogsCLI
[ https://issues.apache.org/jira/browse/YARN-10327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10327: -- Fix Version/s: 3.4.0 > Remove duplication of checking for invalid application ID in TestLogsCLI > > > Key: YARN-10327 > URL: https://issues.apache.org/jira/browse/YARN-10327 > Project: Hadoop YARN > Issue Type: Test >Reporter: Hudáky Márton Gyula >Assignee: Hudáky Márton Gyula >Priority: Trivial > Fix For: 3.4.0 > > Attachments: YARN-10327.001.patch > > > TestLogsCLI has a separate function to test for invalid application ID > (#testInvalidApplicationId) and another (#testLogsCLIWithInvalidArgs) to test > multiple invalid arguments (including application ID). One of them should be > eliminated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10318) ApplicationHistory Web UI incorrect column indexing
[ https://issues.apache.org/jira/browse/YARN-10318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146231#comment-17146231 ] Szilard Nemeth commented on YARN-10318: --- Thanks [~gandras] for working on this. The patch looks good to me, committed to trunk. Could you please verify if we need 3.3 / 3.2 backports? Please set the target version accordingly and upload patches if required for other branches. > ApplicationHistory Web UI incorrect column indexing > --- > > Key: YARN-10318 > URL: https://issues.apache.org/jira/browse/YARN-10318 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Minor > Attachments: Screenshot 2020-06-25 at 10.15.32.png, > YARN-10318.001.patch, image-2020-06-16-17-14-55-921.png > > > The ApplicationHistory UI is broken due to an incorrect column indexing. This > bug was probably introduced in YARN-10038, which presumes, that the table > contains the application tag column (which is true for RM Web UI, but not for > AH Web UI). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10318) ApplicationHistory Web UI incorrect column indexing
[ https://issues.apache.org/jira/browse/YARN-10318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10318: -- Fix Version/s: 3.4.0 > ApplicationHistory Web UI incorrect column indexing > --- > > Key: YARN-10318 > URL: https://issues.apache.org/jira/browse/YARN-10318 > Project: Hadoop YARN > Issue Type: Bug > Components: yarn >Reporter: Andras Gyori >Assignee: Andras Gyori >Priority: Minor > Fix For: 3.4.0 > > Attachments: Screenshot 2020-06-25 at 10.15.32.png, > YARN-10318.001.patch, image-2020-06-16-17-14-55-921.png > > > The ApplicationHistory UI is broken due to an incorrect column indexing. This > bug was probably introduced in YARN-10038, which presumes, that the table > contains the application tag column (which is true for RM Web UI, but not for > AH Web UI). -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10304) Create an endpoint for remote application log directory path query
[
https://issues.apache.org/jira/browse/YARN-10304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146222#comment-17146222
]
Hudáky Márton Gyula commented on YARN-10304:
Thanks for the effort, [~gandras].
Non-binding +1
One nit found to simplify the magic with the configurations in
TestHsWebServicesLogs, like this:
{code:java}
private Configuration usedConf;
WebServletModule() {
super();
this.usedConf = conf;
}
WebServletModule(Configuration newConf) {
super();
this.usedConf = newConf;
}
{code}
> Create an endpoint for remote application log directory path query
> --
>
> Key: YARN-10304
> URL: https://issues.apache.org/jira/browse/YARN-10304
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Andras Gyori
>Assignee: Andras Gyori
>Priority: Minor
> Attachments: YARN-10304.001.patch, YARN-10304.002.patch,
> YARN-10304.003.patch, YARN-10304.004.patch, YARN-10304.005.patch,
> YARN-10304.006.patch
>
>
> The logic of the aggregated log directory path determination (currently based
> on configuration) is scattered around the codebase and duplicated multiple
> times. By providing a separate class for creating the path for a specific
> user, it allows for an abstraction over this logic. This could be used in
> place of the previously duplicated logic, moreover, we could provide an
> endpoint to query this path.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10330) Add missing test scenarios to TestUserGroupMappingPlacementRule
[
https://issues.apache.org/jira/browse/YARN-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-10330:
Attachment: YARN-10330-001.patch
> Add missing test scenarios to TestUserGroupMappingPlacementRule
> ---
>
> Key: YARN-10330
> URL: https://issues.apache.org/jira/browse/YARN-10330
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, capacityscheduler, test
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
> Attachments: YARN-10330-001.patch
>
>
> After running {{TestUserGroupMappingPlacementRule}} with EclEmma, it turned
> out that there are at least 8-10 missing test scenarios that are not covered.
> Since we're planning to enhance mapping rule logic with extra features, it is
> crucial to have good coverage so that we can verify backward compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10216) Utility to dynamically reload Configuration on the disk
[
https://issues.apache.org/jira/browse/YARN-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146197#comment-17146197
]
Hadoop QA commented on YARN-10216:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
2s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 22m
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 3m
34s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 5m
7s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
27m 40s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 3m
23s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 2m
14s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 8m
42s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 24m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 24m
25s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
3m 38s{color} | {color:orange} root: The patch generated 13 new + 289 unchanged
- 0 fixed = 302 total (was 289) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 4m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 5s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 2m
47s{color} | {color:red} hadoop-common-project/hadoop-common generated 1 new +
0 unchanged - 0 fixed = 1 total (was 0) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 12m
12s{color} | {color:green} hadoop-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 1m 23s{color}
| {color:red} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 24m
48s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 93m
49s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
50s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}296m 30s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-common-project/hadoop-common |
| | Inconsistent synchronization
[jira] [Updated] (YARN-10216) Utility to dynamically reload Configuration on the disk
[ https://issues.apache.org/jira/browse/YARN-10216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cyrus Jackson updated YARN-10216: - Attachment: (was: YARN-10216.001.patch) > Utility to dynamically reload Configuration on the disk > --- > > Key: YARN-10216 > URL: https://issues.apache.org/jira/browse/YARN-10216 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Cyrus Jackson >Assignee: Cyrus Jackson >Priority: Major > Attachments: YARN-10216.001.patch, image-2020-04-06-09-50-51-948.png > > > There should be a way to dynamically reload the configuration properties from > the disk. The purpose of this feature is to let individual classes which are > interested in observing this configuration changes, to be notified when the > conf is reloaded from the disk. This is similar to how Hbase has done. > *Class Diagram* > !image-2020-04-06-09-50-51-948.png! > > *APPROACH DETAILS* > The approach is based on the adaption of Hbase Online Configuration. In this > case, the configuration file is monitored for any changes on the disk. If the > file has changed, the properties of Configuration is reloaded and all the > observers are notified. > The classes that implements the observers updates the necessary values if > required. > -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10277) CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy
[ https://issues.apache.org/jira/browse/YARN-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146157#comment-17146157 ] Hudson commented on YARN-10277: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #18381 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/18381/]) YARN-10277. CapacityScheduler test TestUserGroupMappingPlacementRule (snemeth: rev 33d3df7ee2b3942560e9affe1409854ac3e8bb96) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/placement/TestUserGroupMappingPlacementRule.java > CapacityScheduler test TestUserGroupMappingPlacementRule should build proper > hierarchy > -- > > Key: YARN-10277 > URL: https://issues.apache.org/jira/browse/YARN-10277 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Fix For: 3.4.0 > > Attachments: YARN-10277.001.patch, YARN-10277.002.patch, > YARN-10277.003.patch, YARN-10277.branch-3.3.001.patch > > > Since the CapacityScheduler internal implementation depends more and more on > queue being hierarchical, the test gets really hard to maintain. A lot of > test cases were failing because they used non existing queues, but the older > placement rule solution ignored missing parents, but since the leaf queue > change in CS, we must be able to get a full path for any queue, since all > queues are referenced by their full path. > This test should reflect this and instead of creating and expecting the > existance of fictional queues, it should create a proper queue hierarchy, > with a way to describe it better. > Currently we set up a bunch of mockito "when" statements to simulate the > queue behavior, but this is a hassle to maintain, and easy to miss a few > method. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10327) Remove duplication of checking for invalid application ID in TestLogsCLI
[ https://issues.apache.org/jira/browse/YARN-10327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17145010#comment-17145010 ] Andras Gyori edited comment on YARN-10327 at 6/26/20, 9:26 AM: --- Thank you for working on this issue [~mhudaky]. The patch looks good to me, +1 non binding. Following the same logic, there are invalidOpts as well, that could belong to testLogsCLIWithInvalidArgs method. However, for this issue, I think it is enough. was (Author: gandras): Thank you for working on this issue [~mhudaky]. The patch looks good to me. Following the same logic, there are invalidOpts as well, that could belong to testLogsCLIWithInvalidArgs method. However, for this issue, I think it is enough. > Remove duplication of checking for invalid application ID in TestLogsCLI > > > Key: YARN-10327 > URL: https://issues.apache.org/jira/browse/YARN-10327 > Project: Hadoop YARN > Issue Type: Test >Reporter: Hudáky Márton Gyula >Assignee: Hudáky Márton Gyula >Priority: Trivial > Attachments: YARN-10327.001.patch > > > TestLogsCLI has a separate function to test for invalid application ID > (#testInvalidApplicationId) and another (#testLogsCLIWithInvalidArgs) to test > multiple invalid arguments (including application ID). One of them should be > eliminated. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10278) CapacityScheduler test framework ProportionalCapacityPreemptionPolicyMockFramework need some review
[ https://issues.apache.org/jira/browse/YARN-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146148#comment-17146148 ] Szilard Nemeth commented on YARN-10278: --- [~epayne]: I think I would target branch-3.3 along with trunk. Perhaps I can check if this easily applies to branch-3.2 as well. What do you think? > CapacityScheduler test framework > ProportionalCapacityPreemptionPolicyMockFramework need some review > --- > > Key: YARN-10278 > URL: https://issues.apache.org/jira/browse/YARN-10278 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Attachments: YARN-10278.001.patch, YARN-10278.002.patch > > > This test framework class mocks a bit too heavily, and simulates CS internal > behaviour with the mock methods over a point it is reasonably maintainable, > any internal change in CS is a major headscratch. > A lot of tests depend on this class, so we should approach it carefully, but > I think it's wroth to examine this class if it can be made a bit more > resilient to changes, and easier to maintain. Or at least document it better. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10277) CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy
[ https://issues.apache.org/jira/browse/YARN-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10277: -- Attachment: YARN-10277.branch-3.3.001.patch > CapacityScheduler test TestUserGroupMappingPlacementRule should build proper > hierarchy > -- > > Key: YARN-10277 > URL: https://issues.apache.org/jira/browse/YARN-10277 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Fix For: 3.4.0 > > Attachments: YARN-10277.001.patch, YARN-10277.002.patch, > YARN-10277.003.patch, YARN-10277.branch-3.3.001.patch > > > Since the CapacityScheduler internal implementation depends more and more on > queue being hierarchical, the test gets really hard to maintain. A lot of > test cases were failing because they used non existing queues, but the older > placement rule solution ignored missing parents, but since the leaf queue > change in CS, we must be able to get a full path for any queue, since all > queues are referenced by their full path. > This test should reflect this and instead of creating and expecting the > existance of fictional queues, it should create a proper queue hierarchy, > with a way to describe it better. > Currently we set up a bunch of mockito "when" statements to simulate the > queue behavior, but this is a hassle to maintain, and easy to miss a few > method. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-10277) CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy
[ https://issues.apache.org/jira/browse/YARN-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146142#comment-17146142 ] Szilard Nemeth commented on YARN-10277: --- Thanks [~pbacsko], As discussed offline, I committed thit to myself as [~pbacsko] don't have commit privileges yet. CC: [~sunilg] > CapacityScheduler test TestUserGroupMappingPlacementRule should build proper > hierarchy > -- > > Key: YARN-10277 > URL: https://issues.apache.org/jira/browse/YARN-10277 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Fix For: 3.4.0 > > Attachments: YARN-10277.001.patch, YARN-10277.002.patch, > YARN-10277.003.patch > > > Since the CapacityScheduler internal implementation depends more and more on > queue being hierarchical, the test gets really hard to maintain. A lot of > test cases were failing because they used non existing queues, but the older > placement rule solution ignored missing parents, but since the leaf queue > change in CS, we must be able to get a full path for any queue, since all > queues are referenced by their full path. > This test should reflect this and instead of creating and expecting the > existance of fictional queues, it should create a proper queue hierarchy, > with a way to describe it better. > Currently we set up a bunch of mockito "when" statements to simulate the > queue behavior, but this is a hassle to maintain, and easy to miss a few > method. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-10277) CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy
[ https://issues.apache.org/jira/browse/YARN-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17146142#comment-17146142 ] Szilard Nemeth edited comment on YARN-10277 at 6/26/20, 9:14 AM: - Thanks [~pbacsko], As discussed offline, I committed thit to myself as [~pbacsko] don't have commit privileges yet. CC: [~sunilg] Checking if this applies to branch-3.3 easily. was (Author: snemeth): Thanks [~pbacsko], As discussed offline, I committed thit to myself as [~pbacsko] don't have commit privileges yet. CC: [~sunilg] > CapacityScheduler test TestUserGroupMappingPlacementRule should build proper > hierarchy > -- > > Key: YARN-10277 > URL: https://issues.apache.org/jira/browse/YARN-10277 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Fix For: 3.4.0 > > Attachments: YARN-10277.001.patch, YARN-10277.002.patch, > YARN-10277.003.patch > > > Since the CapacityScheduler internal implementation depends more and more on > queue being hierarchical, the test gets really hard to maintain. A lot of > test cases were failing because they used non existing queues, but the older > placement rule solution ignored missing parents, but since the leaf queue > change in CS, we must be able to get a full path for any queue, since all > queues are referenced by their full path. > This test should reflect this and instead of creating and expecting the > existance of fictional queues, it should create a proper queue hierarchy, > with a way to describe it better. > Currently we set up a bunch of mockito "when" statements to simulate the > queue behavior, but this is a hassle to maintain, and easy to miss a few > method. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10277) CapacityScheduler test TestUserGroupMappingPlacementRule should build proper hierarchy
[ https://issues.apache.org/jira/browse/YARN-10277?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Szilard Nemeth updated YARN-10277: -- Fix Version/s: 3.4.0 > CapacityScheduler test TestUserGroupMappingPlacementRule should build proper > hierarchy > -- > > Key: YARN-10277 > URL: https://issues.apache.org/jira/browse/YARN-10277 > Project: Hadoop YARN > Issue Type: Task >Reporter: Gergely Pollak >Assignee: Szilard Nemeth >Priority: Major > Fix For: 3.4.0 > > Attachments: YARN-10277.001.patch, YARN-10277.002.patch, > YARN-10277.003.patch > > > Since the CapacityScheduler internal implementation depends more and more on > queue being hierarchical, the test gets really hard to maintain. A lot of > test cases were failing because they used non existing queues, but the older > placement rule solution ignored missing parents, but since the leaf queue > change in CS, we must be able to get a full path for any queue, since all > queues are referenced by their full path. > This test should reflect this and instead of creating and expecting the > existance of fictional queues, it should create a proper queue hierarchy, > with a way to describe it better. > Currently we set up a bunch of mockito "when" statements to simulate the > queue behavior, but this is a hassle to maintain, and easy to miss a few > method. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-10330) Add missing test scenarios to TestUserGroupMappingPlacementRule
[
https://issues.apache.org/jira/browse/YARN-10330?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Bacsko updated YARN-10330:
Description:
After running {{TestUserGroupMappingPlacementRule}} with EclEmma, it turned out
that there are at least 8-10 missing test scenarios that are not covered.
Since we're planning to enhance mapping rule logic with extra features, it is
crucial to have good coverage so that we can verify backward compatibility.
was:
After running {{TestUserGroupMappingPlacementRule}} with EclEmma, it turned out
that there are at least 8-10 missing scenarios that are not covered.
Since we're planning to enhance mapping rule logic with extra features, it is
crucial to have good coverage so that we can verify backward compatibility.
> Add missing test scenarios to TestUserGroupMappingPlacementRule
> ---
>
> Key: YARN-10330
> URL: https://issues.apache.org/jira/browse/YARN-10330
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler, capacityscheduler, test
>Reporter: Peter Bacsko
>Assignee: Peter Bacsko
>Priority: Major
>
> After running {{TestUserGroupMappingPlacementRule}} with EclEmma, it turned
> out that there are at least 8-10 missing test scenarios that are not covered.
> Since we're planning to enhance mapping rule logic with extra features, it is
> crucial to have good coverage so that we can verify backward compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-10330) Add missing test scenarios to TestUserGroupMappingPlacementRule
Peter Bacsko created YARN-10330:
---
Summary: Add missing test scenarios to
TestUserGroupMappingPlacementRule
Key: YARN-10330
URL: https://issues.apache.org/jira/browse/YARN-10330
Project: Hadoop YARN
Issue Type: Bug
Components: capacity scheduler, capacityscheduler, test
Reporter: Peter Bacsko
Assignee: Peter Bacsko
After running {{TestUserGroupMappingPlacementRule}} with EclEmma, it turned out
that there are at least 8-10 missing scenarios that are not covered.
Since we're planning to enhance mapping rule logic with extra features, it is
crucial to have good coverage so that we can verify backward compatibility.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
