[jira] [Commented] (YARN-9774) Fix order of arguments for assertEquals in TestSLSUtils
[ https://issues.apache.org/jira/browse/YARN-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913956#comment-16913956 ] Hudson commented on YARN-9774: -- FAILURE: Integrated in Jenkins build Hadoop-trunk-Commit #17172 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/17172/]) YARN-9774. Fix order of arguments for assertEquals in TestSLSUtils. (aajisaka: rev 84b1982060422760702eca6f1ef515c6ad3e85a5) * (edit) hadoop-tools/hadoop-sls/src/test/java/org/apache/hadoop/yarn/sls/utils/TestSLSUtils.java > Fix order of arguments for assertEquals in TestSLSUtils > --- > > Key: YARN-9774 > URL: https://issues.apache.org/jira/browse/YARN-9774 > Project: Hadoop YARN > Issue Type: Bug > Components: test >Reporter: Nikhil Navadiya >Assignee: Nikhil Navadiya >Priority: Minor > Fix For: 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9774.001.patch > > > According to documentation of assertEquals, correct order of parameters is > (expected, actual). It seems like assertEquals statements in > testGetRackHostname method of TestSLSUtils class have wrong order of > parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9774) Fix order of arguments for assertEquals in TestSLSUtils
[ https://issues.apache.org/jira/browse/YARN-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira Ajisaka updated YARN-9774: Issue Type: Bug (was: Improvement) Summary: Fix order of arguments for assertEquals in TestSLSUtils (was: Change order of parameters for assertEquals in TestSLSUtils) > Fix order of arguments for assertEquals in TestSLSUtils > --- > > Key: YARN-9774 > URL: https://issues.apache.org/jira/browse/YARN-9774 > Project: Hadoop YARN > Issue Type: Bug > Components: test >Reporter: Nikhil Navadiya >Assignee: Nikhil Navadiya >Priority: Minor > Attachments: YARN-9774.001.patch > > > According to documentation of assertEquals, correct order of parameters is > (expected, actual). It seems like assertEquals statements in > testGetRackHostname method of TestSLSUtils class have wrong order of > parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9562) Add Java changes for the new RuncContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913942#comment-16913942
]
Hadoop QA commented on YARN-9562:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
28s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 10 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
54s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 38s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
25s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
26s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 13s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 292 new + 687 unchanged - 2 fixed = 979 total (was 689) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 58s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
6s{color} | {color:red}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
generated 15 new + 0 unchanged - 0 fixed = 15 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 49s{color}
| {color:red} hadoop-yarn-api in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 21m
22s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
42s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}100m 0s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs |
module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| | Nullcheck of NodeManager.context at line 535 of value previously
dereferenced in
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStop() At
NodeManager.java:535 of value previously dereferenced in
org.apache.hadoop.yarn.server.nodemanager.NodeManager.serviceStop() At
NodeManager.java:[line 532] |
| | Unused field:NodeManager.java |
| | Dead store to refreshHdfsCacheThread in
org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.ImageTagToManifestPlugin.serviceStart()
At
[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource
[
https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913925#comment-16913925
]
Rohith Sharma K S commented on YARN-8917:
-
[~Tao Yang] Can you look at the test failures whether they are related?
> Absolute (maximum) capacity of level3+ queues is wrongly calculated for
> absolute resource
> -
>
> Key: YARN-8917
> URL: https://issues.apache.org/jira/browse/YARN-8917
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
>Affects Versions: 3.2.1
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Critical
> Attachments: YARN-8917.001.patch, YARN-8917.002.patch
>
>
> Absolute capacity should be equal to multiply capacity by parent-queue's
> absolute-capacity,
> but currently it's calculated as dividing capacity by parent-queue's
> absolute-capacity.
> Calculation for absolute-maximum-capacity has the same problem.
> For example:
> root.a capacity=0.4 maximum-capacity=0.8
> root.a.a1 capacity=0.5 maximum-capacity=0.6
> Absolute capacity of root.a.a1 should be 0.2 but is wrongly calculated as 1.25
> Absolute maximum capacity of root.a.a1 should be 0.48 but is wrongly
> calculated as 0.75
> Moreover:
> {{childQueue.getQueueCapacities().getCapacity()}} should be changed to
> {{childQueue.getQueueCapacities().getCapacity(label)}} to avoid getting wrong
> capacity from default partition when calculating for a non-default partition.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9774) Change order of parameters for assertEquals in TestSLSUtils
[
https://issues.apache.org/jira/browse/YARN-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913914#comment-16913914
]
Hadoop QA commented on YARN-9774:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 14s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 51s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 29s{color}
| {color:red} hadoop-sls in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.yarn.sls.appmaster.TestAMSimulator |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | YARN-9774 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978344/YARN-9774.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 86d308e8ad8e 4.15.0-54-generic #58-Ubuntu SMP Mon Jun 24
10:55:24 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / f6af7d0 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/24613/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/24613/testReport/ |
| Max. process+thread count | 436 (vs. ulimit of 5500) |
| modules | C: hadoop-tools/hadoop-sls U: hadoop-tools/hadoop-sls |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/24613/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
>
[jira] [Updated] (YARN-9756) Create metric that sums total memory/vcores preempted per round
[ https://issues.apache.org/jira/browse/YARN-9756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-9756: --- Attachment: YARN-9756.002.patch > Create metric that sums total memory/vcores preempted per round > --- > > Key: YARN-9756 > URL: https://issues.apache.org/jira/browse/YARN-9756 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Affects Versions: 3.2.0, 2.9.2, 3.0.3, 2.8.5, 3.1.2 >Reporter: Eric Payne >Assignee: Manikandan R >Priority: Major > Attachments: YARN-9756.001.patch, YARN-9756.002.patch, > YARN-9756.WIP.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9756) Create metric that sums total memory/vcores preempted per round
[
https://issues.apache.org/jira/browse/YARN-9756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913860#comment-16913860
]
Manikandan R commented on YARN-9756:
Sorry. Made changes to \{{TestCapacitySchedulerSurgicalPreemption}} test case
but missed to capture in patch.
Attached .002.patch.
> Create metric that sums total memory/vcores preempted per round
> ---
>
> Key: YARN-9756
> URL: https://issues.apache.org/jira/browse/YARN-9756
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacity scheduler
>Affects Versions: 3.2.0, 2.9.2, 3.0.3, 2.8.5, 3.1.2
>Reporter: Eric Payne
>Assignee: Manikandan R
>Priority: Major
> Attachments: YARN-9756.001.patch, YARN-9756.002.patch,
> YARN-9756.WIP.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9770) Create a queue ordering policy which picks child queues with equal probability
[
https://issues.apache.org/jira/browse/YARN-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913841#comment-16913841
]
Hadoop QA commented on YARN-9770:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
15s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 26s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 58 unchanged - 0 fixed = 59 total (was 58) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 57s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 79m 58s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}130m 49s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerResizing |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | YARN-9770 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978328/YARN-9770.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 8ff498914244 4.4.0-157-generic #185-Ubuntu SMP Tue Jul 23
09:17:01 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 93daf69 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_212 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/24612/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
[jira] [Commented] (YARN-9774) Change order of parameters for assertEquals in TestSLSUtils
[ https://issues.apache.org/jira/browse/YARN-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913834#comment-16913834 ] Akira Ajisaka commented on YARN-9774: - LGTM, +1 pending Jenkins. > Change order of parameters for assertEquals in TestSLSUtils > --- > > Key: YARN-9774 > URL: https://issues.apache.org/jira/browse/YARN-9774 > Project: Hadoop YARN > Issue Type: Improvement > Components: test >Reporter: Nikhil Navadiya >Assignee: Nikhil Navadiya >Priority: Minor > Attachments: YARN-9774.001.patch > > > According to documentation of assertEquals, correct order of parameters is > (expected, actual). It seems like assertEquals statements in > testGetRackHostname method of TestSLSUtils class have wrong order of > parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9756) Create metric that sums total memory/vcores preempted per round
[
https://issues.apache.org/jira/browse/YARN-9756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913795#comment-16913795
]
Hadoop QA commented on YARN-9756:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
26s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
36s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 39s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 3 new + 91 unchanged - 0 fixed = 94 total (was 91) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 3s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 86m
18s{color} | {color:green} hadoop-yarn-server-resourcemanager in the patch
passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}145m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | YARN-9756 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12978314/YARN-9756.001.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux e23f686583a1 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 28fb4b5 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/24611/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/24611/testReport/ |
| Max. process+thread count | 886 (vs.
[jira] [Commented] (YARN-9562) Add Java changes for the new RuncContainerRuntime
[
https://issues.apache.org/jira/browse/YARN-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913750#comment-16913750
]
Eric Badger commented on YARN-9562:
---
[~eyang], [~ccondit], I've added patch 005 which adds a new parameterized class
and also parameterized the set function for container runtime data. I'm not
quite sure if this is better since there's still an unchecked assignment in
launchContainer when the we call {{getContainerRuntimeData()}}. Let me know
whether or not this is what you had in mind. If not, let me know what you're
looking for since I'm not sure how generics helps the situation. If they do
help, then I'm clearly missing something so please explain it to me :)
> Add Java changes for the new RuncContainerRuntime
> -
>
> Key: YARN-9562
> URL: https://issues.apache.org/jira/browse/YARN-9562
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Eric Badger
>Assignee: Eric Badger
>Priority: Major
> Attachments: YARN-9562.001.patch, YARN-9562.002.patch,
> YARN-9562.003.patch, YARN-9562.004.patch, YARN-9562.005.patch,
> YARN-9562.006.patch
>
>
> This JIRA will be used to add the Java changes for the new
> RuncContainerRuntime. This will work off of YARN-9560 to use much of the
> existing DockerLinuxContainerRuntime code once it is moved up into an
> abstract class that can be extended.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9562) Add Java changes for the new RuncContainerRuntime
[ https://issues.apache.org/jira/browse/YARN-9562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Badger updated YARN-9562: -- Attachment: YARN-9562.006.patch > Add Java changes for the new RuncContainerRuntime > - > > Key: YARN-9562 > URL: https://issues.apache.org/jira/browse/YARN-9562 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Badger >Assignee: Eric Badger >Priority: Major > Attachments: YARN-9562.001.patch, YARN-9562.002.patch, > YARN-9562.003.patch, YARN-9562.004.patch, YARN-9562.005.patch, > YARN-9562.006.patch > > > This JIRA will be used to add the Java changes for the new > RuncContainerRuntime. This will work off of YARN-9560 to use much of the > existing DockerLinuxContainerRuntime code once it is moved up into an > abstract class that can be extended. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9774) Change order of parameters for assertEquals in TestSLSUtils
[ https://issues.apache.org/jira/browse/YARN-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Navadiya updated YARN-9774: -- Attachment: YARN-9774.001.patch > Change order of parameters for assertEquals in TestSLSUtils > --- > > Key: YARN-9774 > URL: https://issues.apache.org/jira/browse/YARN-9774 > Project: Hadoop YARN > Issue Type: Improvement > Components: test >Reporter: Nikhil Navadiya >Assignee: Nikhil Navadiya >Priority: Minor > Attachments: YARN-9774.001.patch > > > According to documentation of assertEquals, correct order of parameters is > (expected, actual). It seems like assertEquals statements in > testGetRackHostname method of TestSLSUtils class have wrong order of > parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Assigned] (YARN-9774) Change order of parameters for assertEquals in TestSLSUtils
[ https://issues.apache.org/jira/browse/YARN-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wei-Chiu Chuang reassigned YARN-9774: - Assignee: Nikhil Navadiya > Change order of parameters for assertEquals in TestSLSUtils > --- > > Key: YARN-9774 > URL: https://issues.apache.org/jira/browse/YARN-9774 > Project: Hadoop YARN > Issue Type: Improvement > Components: test >Reporter: Nikhil Navadiya >Assignee: Nikhil Navadiya >Priority: Minor > > According to documentation of assertEquals, correct order of parameters is > (expected, actual). It seems like assertEquals statements in > testGetRackHostname method of TestSLSUtils class have wrong order of > parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9774) change order of parameters for assertEquals in TestSLSUtils
Nikhil Navadiya created YARN-9774: - Summary: change order of parameters for assertEquals in TestSLSUtils Key: YARN-9774 URL: https://issues.apache.org/jira/browse/YARN-9774 Project: Hadoop YARN Issue Type: Improvement Components: test Reporter: Nikhil Navadiya According to documentation of assertEquals, correct order of parameters is (expected, actual). It seems like assertEquals statements in testGetRackHostname method of TestSLSUtils class have wrong order of parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9774) Change order of parameters for assertEquals in TestSLSUtils
[ https://issues.apache.org/jira/browse/YARN-9774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikhil Navadiya updated YARN-9774: -- Summary: Change order of parameters for assertEquals in TestSLSUtils (was: change order of parameters for assertEquals in TestSLSUtils) > Change order of parameters for assertEquals in TestSLSUtils > --- > > Key: YARN-9774 > URL: https://issues.apache.org/jira/browse/YARN-9774 > Project: Hadoop YARN > Issue Type: Improvement > Components: test >Reporter: Nikhil Navadiya >Priority: Minor > > According to documentation of assertEquals, correct order of parameters is > (expected, actual). It seems like assertEquals statements in > testGetRackHostname method of TestSLSUtils class have wrong order of > parameters. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource
[
https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913678#comment-16913678
]
Hadoop QA commented on YARN-8917:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 9s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 27s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager:
The patch generated 1 new + 25 unchanged - 0 fixed = 26 total (was 25) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 40s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 82m 11s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
29s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}136m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestLeafQueue |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 Image:yetus/hadoop:bdbca0e |
| JIRA Issue | YARN-8917 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12944836/YARN-8917.002.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 5d442ed6e760 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 28fb4b5 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_222 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/24609/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| unit |
[jira] [Commented] (YARN-9756) Create metric that sums total memory/vcores preempted per round
[ https://issues.apache.org/jira/browse/YARN-9756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913624#comment-16913624 ] Eric Payne commented on YARN-9756: -- Thanks for the updated patch, [~maniraj...@gmail.com]! I am fine with the updates except one thing. bq. I would like you to add a small assertion to TestCapacitySchedulerSurgicalPreemption#testSimpleSurgicalPreemption such as the following: Can you also please update TestCapacitySchedulerSurgicalPreemption to access the new methods, as described above? > Create metric that sums total memory/vcores preempted per round > --- > > Key: YARN-9756 > URL: https://issues.apache.org/jira/browse/YARN-9756 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Affects Versions: 3.2.0, 2.9.2, 3.0.3, 2.8.5, 3.1.2 >Reporter: Eric Payne >Assignee: Manikandan R >Priority: Major > Attachments: YARN-9756.001.patch, YARN-9756.WIP.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9760) Support configuring application priorities on a workflow level
[ https://issues.apache.org/jira/browse/YARN-9760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913621#comment-16913621 ] Jonathan Hung commented on YARN-9760: - Hi [~eepayne], thanks for the comments. I understand the concern. The reason we implemented it this way was b/c we wanted queue admins to be able to set workflow priorities themselves (i.e. leveraging YARN queue ACLs) which would be difficult to do client-side. IMO there's not too much complexity for this feature on the RM side. But to minimize overall complexity, perhaps we can do this in a way that doesn't make protocol changes, and on the RM side we can leverage the queue mapping interface. Would this address your concerns? [~varun_saxena] how does this sound? > Support configuring application priorities on a workflow level > -- > > Key: YARN-9760 > URL: https://issues.apache.org/jira/browse/YARN-9760 > Project: Hadoop YARN > Issue Type: New Feature >Reporter: Jonathan Hung >Assignee: Varun Saxena >Priority: Major > Labels: release-blocker > > Currently priorities are submitted on an application level, but for end users > it's common to submit workloads to YARN at a workflow level. This jira > proposes a feature to store workflow id + priority mappings on RM (similar to > queue mappings). If app is submitted with a certain workflow id (as set in > application submission context) RM will override this app's priority with the > one defined in the mapping. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9770) Create a queue ordering policy which picks child queues with equal probability
[ https://issues.apache.org/jira/browse/YARN-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913594#comment-16913594 ] Jonathan Hung commented on YARN-9770: - 002: Fix findbugs and checkstyle. The CapacitySchedulerConfiguration checkstyle error is not related, not sure why it's flagging it. > Create a queue ordering policy which picks child queues with equal probability > -- > > Key: YARN-9770 > URL: https://issues.apache.org/jira/browse/YARN-9770 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Labels: release-blocker > Attachments: YARN-9770.001.patch, YARN-9770.002.patch > > > Ran some simulations with the default queue_utilization_ordering_policy: > An underutilized queue which receives an application with many (thousands) > resource requests will hog scheduler allocations for a long time (on the > order of a minute). In the meantime apps are getting submitted to all other > queues, which increases activeUsers in these queues, which drops user limit > in these queues to small values if minimum-user-limit-percent is configured > to small values (e.g. 10%). > To avoid this issue, we assign to queues with equal probability, to avoid > scenarios where queues don't get allocations for a long time. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9770) Create a queue ordering policy which picks child queues with equal probability
[ https://issues.apache.org/jira/browse/YARN-9770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9770: Attachment: YARN-9770.002.patch > Create a queue ordering policy which picks child queues with equal probability > -- > > Key: YARN-9770 > URL: https://issues.apache.org/jira/browse/YARN-9770 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Labels: release-blocker > Attachments: YARN-9770.001.patch, YARN-9770.002.patch > > > Ran some simulations with the default queue_utilization_ordering_policy: > An underutilized queue which receives an application with many (thousands) > resource requests will hog scheduler allocations for a long time (on the > order of a minute). In the meantime apps are getting submitted to all other > queues, which increases activeUsers in these queues, which drops user limit > in these queues to small values if minimum-user-limit-percent is configured > to small values (e.g. 10%). > To avoid this issue, we assign to queues with equal probability, to avoid > scenarios where queues don't get allocations for a long time. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8657) User limit calculation should be read-lock-protected within LeafQueue
[
https://issues.apache.org/jira/browse/YARN-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913593#comment-16913593
]
Hadoop QA commented on YARN-8657:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} YARN-8657 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-8657 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12941021/YARN-8657.002.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/24610/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> User limit calculation should be read-lock-protected within LeafQueue
> -
>
> Key: YARN-8657
> URL: https://issues.apache.org/jira/browse/YARN-8657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sumana Sathish
>Assignee: Wangda Tan
>Priority: Critical
> Attachments: YARN-8657.001.patch, YARN-8657.002.patch
>
>
> When async scheduling is enabled, user limit calculation could be wrong:
> It is possible that scheduler calculated a user_limit, but inside
> {{canAssignToUser}} it becomes staled.
> We need to protect user limit calculation.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9561) Add C changes for the new RuncContainerRuntime
[ https://issues.apache.org/jira/browse/YARN-9561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913561#comment-16913561 ] Eric Yang commented on YARN-9561: - A few discoveries found during the debug session of patch 003 with [~ebadger] and [~Jim_Brennan]: # When cluster is running without cgroup configured, container-executor will report exit code INVALID_CONFIG. This cause the node manager to become unhealthy. It would be nice to have more verbose logging for the root cause. # There are several check points are all using INVALID_CONFIG. It is hard to identify where the program failed. It would be nice to have more verbose logging for the possible root causes. # The current implementation works only with overlay module. It would be nice to detect absence of overlay module and document the requirements. > Add C changes for the new RuncContainerRuntime > -- > > Key: YARN-9561 > URL: https://issues.apache.org/jira/browse/YARN-9561 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Badger >Assignee: Eric Badger >Priority: Major > Attachments: YARN-9561.001.patch, YARN-9561.002.patch, > YARN-9561.003.patch > > > This JIRA will be used to add the C changes to the container-executor native > binary that are necessary for the new RuncContainerRuntime. There should be > no changes to existing code paths. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6492) Generate queue metrics for each partition
[ https://issues.apache.org/jira/browse/YARN-6492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913526#comment-16913526 ] Manikandan R commented on YARN-6492: Created YARN-9773 for the same. Will split .005 patch and attach the same in corresponding sub tasks shortly. > Generate queue metrics for each partition > - > > Key: YARN-6492 > URL: https://issues.apache.org/jira/browse/YARN-6492 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Reporter: Jonathan Hung >Assignee: Manikandan R >Priority: Major > Attachments: PartitionQueueMetrics_default_partition.txt, > PartitionQueueMetrics_x_partition.txt, PartitionQueueMetrics_y_partition.txt, > YARN-6492.001.patch, YARN-6492.002.patch, YARN-6492.003.patch, > YARN-6492.004.patch, YARN-6492.005.WIP.patch, partition_metrics.txt > > > We are interested in having queue metrics for all partitions. Right now each > queue has one QueueMetrics object which captures metrics either in default > partition or across all partitions. (After YARN-6467 it will be in default > partition) > But having the partition metrics would be very useful. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9773) PartitionQueueMetrics for Custom Resources/Resource vectors
Manikandan R created YARN-9773: -- Summary: PartitionQueueMetrics for Custom Resources/Resource vectors Key: YARN-9773 URL: https://issues.apache.org/jira/browse/YARN-9773 Project: Hadoop YARN Issue Type: Bug Reporter: Manikandan R Assignee: Manikandan R -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9773) PartitionQueueMetrics for Custom Resources/Resource vectors
[ https://issues.apache.org/jira/browse/YARN-9773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-9773: --- Parent: YARN-6492 Issue Type: Sub-task (was: Bug) > PartitionQueueMetrics for Custom Resources/Resource vectors > --- > > Key: YARN-9773 > URL: https://issues.apache.org/jira/browse/YARN-9773 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Manikandan R >Assignee: Manikandan R >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9766) YARN CapacityScheduler QueueMetrics has missing metrics for parent queues having same name
[ https://issues.apache.org/jira/browse/YARN-9766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913502#comment-16913502 ] Manikandan R commented on YARN-9766: Ok, [~tarunparimi]. Thanks. While understanding this issue in detail, had come across another related issue. Created YARN-9772 for the same. > YARN CapacityScheduler QueueMetrics has missing metrics for parent queues > having same name > -- > > Key: YARN-9766 > URL: https://issues.apache.org/jira/browse/YARN-9766 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.7.0 >Reporter: Tarun Parimi >Assignee: Tarun Parimi >Priority: Major > > In Capacity Scheduler, we enforce Leaf Queues to have unique names. But it is > not the case for Parent Queues. For example, we can have the below queue > hierarchy, where "b" is the queue name for two different queue paths root.a.b > and root.a.d.b . Since it is not a leaf queue this configuration works and > apps run fine in the leaf queues 'c' and 'e'. > * root > ** a > *** b > c > *** d > b > * e > But the jmx metrics does not show the metrics for the parent queue > "root.a.d.b" . We can see metrics only for "root.a.b" queue. > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9772) CapacitySchedulerQueueManager has incorrect list of queues
[
https://issues.apache.org/jira/browse/YARN-9772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Manikandan R updated YARN-9772:
---
Description:
CapacitySchedulerQueueManager has incorrect list of queues when there is more
than one parent queue (say at middle level) with same name.
For example,
* root
** a
*** b
c
*** d
b
* e
{{CapacitySchedulerQueueManager#getQueues}} maintains these list of queues.
While parsing "root.a.d.b", it overrides "root.a.b" with new Queue object in
the map because of similar name. After parsing all the queues, map count should
be 7, but it is 6. Any reference to queue "root.a.b" in code path is nothing
but "root.a.d.b" object. Since {{CapacitySchedulerQueueManager#getQueues}} has
been used in multiple places, will need to understand the implications in
detail. For example, {{CapapcityScheduler#getQueue}} has been used in many
places which in turn uses {{CapacitySchedulerQueueManager#getQueues}}. cc
[~eepayne], [~sunilg]
was:
CapacitySchedulerQueueManager has incorrect list of queues when there is more
than one parent queue (say at middle level) with same name.
For example,
* root
** a
*** b
c
*** d
b
* e
{{CapacitySchedulerQueueManager#getQueues}} maintains these list of queues.
While parsing "root.a.d.b", it overrides "root.a.b" with new Queue object in
the map because of similar name. After parsing all the queues, map count should
be 7, but it is 6. Any reference to queue "root.a.b" in code path is nothing
but "root.a.d.b" object. Since {{CapacitySchedulerQueueManager#getQueues}} has
been used in multiple places, will need to understand the implications in
detail. For example, {{CapapcityScheduler#getQueue}} has been used in many
places which in turn uses {{CapacitySchedulerQueueManager#getQueues. cc
[~eepayne], [~sunilg] }}
> CapacitySchedulerQueueManager has incorrect list of queues
> --
>
> Key: YARN-9772
> URL: https://issues.apache.org/jira/browse/YARN-9772
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Manikandan R
>Assignee: Manikandan R
>Priority: Major
>
> CapacitySchedulerQueueManager has incorrect list of queues when there is more
> than one parent queue (say at middle level) with same name.
> For example,
> * root
> ** a
> *** b
> c
> *** d
> b
> * e
> {{CapacitySchedulerQueueManager#getQueues}} maintains these list of queues.
> While parsing "root.a.d.b", it overrides "root.a.b" with new Queue object in
> the map because of similar name. After parsing all the queues, map count
> should be 7, but it is 6. Any reference to queue "root.a.b" in code path is
> nothing but "root.a.d.b" object. Since
> {{CapacitySchedulerQueueManager#getQueues}} has been used in multiple places,
> will need to understand the implications in detail. For example,
> {{CapapcityScheduler#getQueue}} has been used in many places which in turn
> uses {{CapacitySchedulerQueueManager#getQueues}}. cc [~eepayne], [~sunilg]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9772) CapacitySchedulerQueueManager has incorrect list of queues
Manikandan R created YARN-9772:
--
Summary: CapacitySchedulerQueueManager has incorrect list of queues
Key: YARN-9772
URL: https://issues.apache.org/jira/browse/YARN-9772
Project: Hadoop YARN
Issue Type: Bug
Reporter: Manikandan R
Assignee: Manikandan R
CapacitySchedulerQueueManager has incorrect list of queues when there is more
than one parent queue (say at middle level) with same name.
For example,
* root
** a
*** b
c
*** d
b
* e
{{CapacitySchedulerQueueManager#getQueues}} maintains these list of queues.
While parsing "root.a.d.b", it overrides "root.a.b" with new Queue object in
the map because of similar name. After parsing all the queues, map count should
be 7, but it is 6. Any reference to queue "root.a.b" in code path is nothing
but "root.a.d.b" object. Since {{CapacitySchedulerQueueManager#getQueues}} has
been used in multiple places, will need to understand the implications in
detail. For example, {{CapapcityScheduler#getQueue}} has been used in many
places which in turn uses {{CapacitySchedulerQueueManager#getQueues. cc
[~eepayne], [~sunilg] }}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9756) Create metric that sums total memory/vcores preempted per round
[ https://issues.apache.org/jira/browse/YARN-9756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913455#comment-16913455 ] Manikandan R commented on YARN-9756: [~eepayne] Thanks. Attached .001.patch for your reviews. > Create metric that sums total memory/vcores preempted per round > --- > > Key: YARN-9756 > URL: https://issues.apache.org/jira/browse/YARN-9756 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Affects Versions: 3.2.0, 2.9.2, 3.0.3, 2.8.5, 3.1.2 >Reporter: Eric Payne >Assignee: Manikandan R >Priority: Major > Attachments: YARN-9756.001.patch, YARN-9756.WIP.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9756) Create metric that sums total memory/vcores preempted per round
[ https://issues.apache.org/jira/browse/YARN-9756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manikandan R updated YARN-9756: --- Attachment: YARN-9756.001.patch > Create metric that sums total memory/vcores preempted per round > --- > > Key: YARN-9756 > URL: https://issues.apache.org/jira/browse/YARN-9756 > Project: Hadoop YARN > Issue Type: Improvement > Components: capacity scheduler >Affects Versions: 3.2.0, 2.9.2, 3.0.3, 2.8.5, 3.1.2 >Reporter: Eric Payne >Assignee: Manikandan R >Priority: Major > Attachments: YARN-9756.001.patch, YARN-9756.WIP.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9591) JAR archive expansion doesn't faithfully restore the jar file content
[
https://issues.apache.org/jira/browse/YARN-9591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913447#comment-16913447
]
Terence Yim commented on YARN-9591:
---
Ping again. Is anyone available reviewing the fix?
> JAR archive expansion doesn't faithfully restore the jar file content
> -
>
> Key: YARN-9591
> URL: https://issues.apache.org/jira/browse/YARN-9591
> Project: Hadoop YARN
> Issue Type: Bug
> Components: nodemanager
>Affects Versions: 3.1.0
>Reporter: Terence Yim
>Priority: Major
>
> Due to change in YARN-2185, jar archives are unjar using {{JarInputStream}},
> which will skip writing out the {{META-INF/MANIFEST.MF}} file.
> Before the YARN-2185 change, it was unarchive using {{JarFile}} instead, and
> the {{META-INF/MANIFEST.MF}} file was written out correctly.
> This change of behavior is causing some applications to fail.
> Suggested to change to use {{ZipInputStream}} instead to faithfully restoring
> all entries inside a jar file.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9509) Capped cpu usage with cgroup strict-resource-usage based on a mulitplier
[
https://issues.apache.org/jira/browse/YARN-9509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913439#comment-16913439
]
Hadoop QA commented on YARN-9509:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
46s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
16s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
44s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 22s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
5s{color} | {color:green} trunk passed {color} |
| {color:blue}0{color} | {color:blue} spotbugs {color} | {color:blue} 1m
40s{color} | {color:blue} Used deprecated FindBugs config; considering
switching to SpotBugs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
36s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
58s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 19s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 5 new + 219 unchanged - 0 fixed = 224 total (was 219) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
3s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
59s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 4m
8s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 18m 53s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}119m 11s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.containermanager.TestContainerManager |
| |
[jira] [Commented] (YARN-9009) Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs
[
https://issues.apache.org/jira/browse/YARN-9009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913418#comment-16913418
]
Hadoop QA commented on YARN-9009:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 13s{color}
| {color:red} https://github.com/apache/hadoop/pull/438 does not apply to
trunk. Rebase required? Wrong Branch? See
https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| GITHUB PR | https://github.com/apache/hadoop/pull/438 |
| JIRA Issue | YARN-9009 |
| Console output |
https://builds.apache.org/job/hadoop-multibranch/job/PR-438/7/console |
| versions | git=2.7.4 |
| Powered by | Apache Yetus 0.10.0 http://yetus.apache.org |
This message was automatically generated.
> Fix flaky test TestEntityGroupFSTimelineStore.testCleanLogs
> ---
>
> Key: YARN-9009
> URL: https://issues.apache.org/jira/browse/YARN-9009
> Project: Hadoop YARN
> Issue Type: Bug
> Environment: Ubuntu 18.04
> java version "1.8.0_181"
> Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
> Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)
>
> Apache Maven 3.5.4 (1edded0938998edf8bf061f1ceb3cfdeccf443fe;
> 2018-06-17T13:33:14-05:00)
>Reporter: OrDTesters
>Assignee: OrDTesters
>Priority: Minor
> Fix For: 3.0.4, 3.1.2, 3.3.0, 3.2.1
>
> Attachments: YARN-9009-trunk-001.patch
>
>
> In TestEntityGroupFSTimelineStore, testCleanLogs fails when run after
> testMoveToDone.
> testCleanLogs fails because testMoveToDone moves a file into the same
> directory that testCleanLogs cleans, causing testCleanLogs to clean 3 files,
> instead of 2 as testCleanLogs expects.
> To fix the failure of testCleanLogs, we can delete the file after the file is
> moved by testMoveToDone.
> Pull request link: [https://github.com/apache/hadoop/pull/438]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9468) Fix inaccurate documentations in Placement Constraints
[
https://issues.apache.org/jira/browse/YARN-9468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913403#comment-16913403
]
Hadoop QA commented on YARN-9468:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
3s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
28m 21s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 39s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/hadoop-multibranch/job/PR-717/8/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/717 |
| JIRA Issue | YARN-9468 |
| Optional Tests | dupname asflicense mvnsite |
| uname | Linux 4667e3873e93 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 69ddb36 |
| Max. process+thread count | 412 (vs. ulimit of 5500) |
| modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site |
| Console output |
https://builds.apache.org/job/hadoop-multibranch/job/PR-717/8/console |
| versions | git=2.7.4 maven=3.3.9 |
| Powered by | Apache Yetus 0.10.0 http://yetus.apache.org |
This message was automatically generated.
> Fix inaccurate documentations in Placement Constraints
> --
>
> Key: YARN-9468
> URL: https://issues.apache.org/jira/browse/YARN-9468
> Project: Hadoop YARN
> Issue Type: Bug
> Components: yarn
>Affects Versions: 3.2.0
>Reporter: hunshenshi
>Assignee: hunshenshi
>Priority: Major
>
> Document Placement Constraints
> *First*
> {code:java}
> zk=3,NOTIN,NODE,zk:hbase=5,IN,RACK,zk:spark=7,CARDINALITY,NODE,hbase,1,3{code}
> * place 5 containers with tag “hbase” with affinity to a rack on which
> containers with tag “zk” are running (i.e., an “hbase” container
> should{color:#ff} not{color} be placed at a rack where an “zk” container
> is running, given that “zk” is the TargetTag of the second constraint);
> The _*not*_ word in brackets should be delete.
>
> *Second*
> {code:java}
> PlacementSpec => "" | KeyVal;PlacementSpec
> {code}
> The semicolon should be replaced by colon
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9771) Add GPU in the container-executor.cfg example
Adam Antal created YARN-9771:
Summary: Add GPU in the container-executor.cfg example
Key: YARN-9771
URL: https://issues.apache.org/jira/browse/YARN-9771
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager, yarn
Affects Versions: 3.2.0
Reporter: Adam Antal
Assignee: Julia Kinga Marton
Currently the {{hadoop-yarn-project/hadoop-yarn/conf/container-executor.cfg}}
example config file does not contain a gpu section. According to the [apache
Hadoop wiki
page|https://hadoop.apache.org/docs/r3.2.0/hadoop-yarn/hadoop-yarn-site/UsingGpus.html]
this is the way how you enable using gpus:
{code:java}
[gpu]
module.enabled=true
{code}
I didn't see any other related configs just like in the case of fpga or
pluggable device, but I think we can add that short part to show that the gpu
feature works.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9579) the property of sharedcache in mapred-default.xml
[
https://issues.apache.org/jira/browse/YARN-9579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913375#comment-16913375
]
Hadoop QA commented on YARN-9579:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
44s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} dupname {color} | {color:green} 0m
0s{color} | {color:green} No case conflicting files found. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m
20s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
29m 11s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
11m 57s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 5m
21s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
24s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 51m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.1 Server=19.03.1 base:
https://builds.apache.org/job/hadoop-multibranch/job/PR-848/9/artifact/out/Dockerfile
|
| GITHUB PR | https://github.com/apache/hadoop/pull/848 |
| JIRA Issue | YARN-9579 |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient xml |
| uname | Linux 4db840610cb7 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | personality/hadoop.sh |
| git revision | trunk / 69ddb36 |
| Default Java | 1.8.0_222 |
| Test Results |
https://builds.apache.org/job/hadoop-multibranch/job/PR-848/9/testReport/ |
| Max. process+thread count | 1643 (vs. ulimit of 5500) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output |
https://builds.apache.org/job/hadoop-multibranch/job/PR-848/9/console |
| versions | git=2.7.4 maven=3.3.9 |
| Powered by | Apache Yetus 0.10.0 http://yetus.apache.org |
This message was automatically generated.
> the property of sharedcache in mapred-default.xml
> -
>
> Key: YARN-9579
> URL: https://issues.apache.org/jira/browse/YARN-9579
> Project: Hadoop
[jira] [Commented] (YARN-9479) Change String.equals to Objects.equals(String,String) to avoid possible NullPointerException
[
https://issues.apache.org/jira/browse/YARN-9479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913351#comment-16913351
]
Hadoop QA commented on YARN-9479:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 13s{color}
| {color:red} https://github.com/apache/hadoop/pull/738 does not apply to
trunk. Rebase required? Wrong Branch? See
https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| GITHUB PR | https://github.com/apache/hadoop/pull/738 |
| JIRA Issue | YARN-9479 |
| Console output |
https://builds.apache.org/job/hadoop-multibranch/job/PR-738/10/console |
| versions | git=2.7.4 |
| Powered by | Apache Yetus 0.10.0 http://yetus.apache.org |
This message was automatically generated.
> Change String.equals to Objects.equals(String,String) to avoid possible
> NullPointerException
>
>
> Key: YARN-9479
> URL: https://issues.apache.org/jira/browse/YARN-9479
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: bd2019us
>Priority: Major
> Attachments: 1.patch
>
>
> Hello,
> I found that the String "queueName" may cause potential risk of
> NullPointerException since it is immediately used after initialization and
> there is no null checker. One recommended API is
> Objects.equals(String,String) which can avoid this exception.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8657) User limit calculation should be read-lock-protected within LeafQueue
[
https://issues.apache.org/jira/browse/YARN-8657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913205#comment-16913205
]
Rohith Sharma K S commented on YARN-8657:
-
[~cheersyang] [~sunilg] [~leftnoteasy] [~bsteinbach] Any consensus on this?
> User limit calculation should be read-lock-protected within LeafQueue
> -
>
> Key: YARN-8657
> URL: https://issues.apache.org/jira/browse/YARN-8657
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Reporter: Sumana Sathish
>Assignee: Wangda Tan
>Priority: Critical
> Attachments: YARN-8657.001.patch, YARN-8657.002.patch
>
>
> When async scheduling is enabled, user limit calculation could be wrong:
> It is possible that scheduler calculated a user_limit, but inside
> {{canAssignToUser}} it becomes staled.
> We need to protect user limit calculation.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9640) Slow event processing could cause too many attempt unregister events
[
https://issues.apache.org/jira/browse/YARN-9640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913197#comment-16913197
]
Rohith Sharma K S commented on YARN-9640:
-
Patch reasonable to me. +1 lgtm. [~bibinchundatt] Can you look at the test
failures?
> Slow event processing could cause too many attempt unregister events
>
>
> Key: YARN-9640
> URL: https://issues.apache.org/jira/browse/YARN-9640
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Labels: scalability
> Attachments: YARN-9640.001.patch, YARN-9640.002.patch,
> YARN-9640.003.patch
>
>
> We found in one of our test cluster verification that the number attempt
> unregister events is about 300k+.
> # AM all containers completed.
> # AMRMClientImpl send finishApplcationMaster
> # AMRMClient check event 100ms the finish Status using
> finishApplicationMaster request.
> # AMRMClientImpl#unregisterApplicationMaster
> {code:java}
> while (true) {
> FinishApplicationMasterResponse response =
> rmClient.finishApplicationMaster(request);
> if (response.getIsUnregistered()) {
> break;
> }
> LOG.info("Waiting for application to be successfully unregistered.");
> Thread.sleep(100);
> }
> {code}
> # ApplicationMasterService finishApplicationMaster interface sends
> unregister events on every status update.
> We should send unregister event only once and cache event send , ignore and
> send not unregistered response back to AM not overloading the event queue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8917) Absolute (maximum) capacity of level3+ queues is wrongly calculated for absolute resource
[
https://issues.apache.org/jira/browse/YARN-8917?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913169#comment-16913169
]
Rohith Sharma K S commented on YARN-8917:
-
I am re-triggering jenkins to see patch still applies.
cc:/ [~Tao Yang] [~sunilg] [~leftnoteasy]
> Absolute (maximum) capacity of level3+ queues is wrongly calculated for
> absolute resource
> -
>
> Key: YARN-8917
> URL: https://issues.apache.org/jira/browse/YARN-8917
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacityscheduler
>Affects Versions: 3.2.1
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Critical
> Attachments: YARN-8917.001.patch, YARN-8917.002.patch
>
>
> Absolute capacity should be equal to multiply capacity by parent-queue's
> absolute-capacity,
> but currently it's calculated as dividing capacity by parent-queue's
> absolute-capacity.
> Calculation for absolute-maximum-capacity has the same problem.
> For example:
> root.a capacity=0.4 maximum-capacity=0.8
> root.a.a1 capacity=0.5 maximum-capacity=0.6
> Absolute capacity of root.a.a1 should be 0.2 but is wrongly calculated as 1.25
> Absolute maximum capacity of root.a.a1 should be 0.48 but is wrongly
> calculated as 0.75
> Moreover:
> {{childQueue.getQueueCapacities().getCapacity()}} should be changed to
> {{childQueue.getQueueCapacities().getCapacity(label)}} to avoid getting wrong
> capacity from default partition when calculating for a non-default partition.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9714) ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby
[ https://issues.apache.org/jira/browse/YARN-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913157#comment-16913157 ] Rohith Sharma K S commented on YARN-9714: - bq. ZooKeeper connection in ZKRMStateStore never be closed. Which version of Hadoop are you using? We are closing the ZK connection. See ZKRMStateStore#closeInternal() [~Tao Yang] Do you think this is still issue in latest version of hadoop? > ZooKeeper connection in ZKRMStateStore leaks after RM transitioned to standby > - > > Key: YARN-9714 > URL: https://issues.apache.org/jira/browse/YARN-9714 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Tao Yang >Assignee: Tao Yang >Priority: Blocker > Labels: memory-leak > Attachments: YARN-9714.001.patch, YARN-9714.002.patch > > > Recently RM full GC happened in one of our clusters, after investigating the > dump memory and jstack, I found two places in RM may cause memory leaks after > RM transitioned to standby: > # Release cache cleanup timer in AbstractYarnScheduler never be canceled. > # ZooKeeper connection in ZKRMStateStore never be closed. > To solve those leaks, we should close the connection or cancel the timer when > services are stopping. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9642) AbstractYarnScheduler#clearPendingContainerCache could run even after transitiontostandby
[ https://issues.apache.org/jira/browse/YARN-9642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913074#comment-16913074 ] Rohith Sharma K S commented on YARN-9642: - +1 LGTM.. [~bibinchundatt] Can you take a look at the test failures? Is it related? > AbstractYarnScheduler#clearPendingContainerCache could run even after > transitiontostandby > - > > Key: YARN-9642 > URL: https://issues.apache.org/jira/browse/YARN-9642 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Blocker > Labels: memory-leak > Attachments: YARN-9642.001.patch, YARN-9642.002.patch, > YARN-9642.003.patch, image-2019-06-22-16-05-24-114.png > > > The TimeTask could hold the reference of Scheduler in case of fast switch > over too. > AbstractYarnScheduler should make sure scheduled Timer cancelled on > serviceStop. > Causes memory leak too > !image-2019-06-22-16-05-24-114.png! -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8625) Aggregate Resource Allocation for each job is not present in ATS
[ https://issues.apache.org/jira/browse/YARN-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913037#comment-16913037 ] Rohith Sharma K S commented on YARN-8625: - Updated right fix version for branch-3.2 i.e 3.2.1 > Aggregate Resource Allocation for each job is not present in ATS > > > Key: YARN-8625 > URL: https://issues.apache.org/jira/browse/YARN-8625 > Project: Hadoop YARN > Issue Type: Bug > Components: ATSv2 >Affects Versions: 2.7.4 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Fix For: 2.10.0, 2.7.8, 3.0.4, 3.3.0, 2.8.6, 3.2.1, 2.9.3, 3.1.3 > > Attachments: 0001-YARN-8625.patch, 0002-YARN-8625.patch, > ApplicationHistoryServer_Rest_Api.png, ApplicationHistoryServer_UI.png, > YARN-8625-branch-2.001.patch, YARN-8625-branch-2.7.001.patch, > YARN-8625-branch-2.8.001.patch, yarn-site.xml > > > Aggregate Resource Allocation shown on RM UI for finished job is very useful > metric to understand how much resource a job has consumed. But this does not > get stored in ATS. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8625) Aggregate Resource Allocation for each job is not present in ATS
[ https://issues.apache.org/jira/browse/YARN-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8625: Fix Version/s: (was: 3.2.2) 3.2.1 > Aggregate Resource Allocation for each job is not present in ATS > > > Key: YARN-8625 > URL: https://issues.apache.org/jira/browse/YARN-8625 > Project: Hadoop YARN > Issue Type: Bug > Components: ATSv2 >Affects Versions: 2.7.4 >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Fix For: 2.10.0, 2.7.8, 3.0.4, 3.3.0, 2.8.6, 3.2.1, 2.9.3, 3.1.3 > > Attachments: 0001-YARN-8625.patch, 0002-YARN-8625.patch, > ApplicationHistoryServer_Rest_Api.png, ApplicationHistoryServer_UI.png, > YARN-8625-branch-2.001.patch, YARN-8625-branch-2.7.001.patch, > YARN-8625-branch-2.8.001.patch, yarn-site.xml > > > Aggregate Resource Allocation shown on RM UI for finished job is very useful > metric to understand how much resource a job has consumed. But this does not > get stored in ATS. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8856) TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError
[ https://issues.apache.org/jira/browse/YARN-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-8856: Fix Version/s: (was: 3.2.2) 3.2.1 > TestTimelineReaderWebServicesHBaseStorage tests failing with > NoClassDefFoundError > - > > Key: YARN-8856 > URL: https://issues.apache.org/jira/browse/YARN-8856 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jason Lowe >Assignee: Sushil Ks >Priority: Major > Fix For: 3.3.0, 3.2.1 > > Attachments: YARN-8856.001.patch > > > TestTimelineReaderWebServicesHBaseStorage has been failing in nightly builds > with NoClassDefFoundError in the tests. Sample error and stacktrace to > follow. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8856) TestTimelineReaderWebServicesHBaseStorage tests failing with NoClassDefFoundError
[ https://issues.apache.org/jira/browse/YARN-8856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913036#comment-16913036 ] Rohith Sharma K S commented on YARN-8856: - Updated right fix version for branch-3.2 i.e 3.2.1 > TestTimelineReaderWebServicesHBaseStorage tests failing with > NoClassDefFoundError > - > > Key: YARN-8856 > URL: https://issues.apache.org/jira/browse/YARN-8856 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jason Lowe >Assignee: Sushil Ks >Priority: Major > Fix For: 3.3.0, 3.2.1 > > Attachments: YARN-8856.001.patch > > > TestTimelineReaderWebServicesHBaseStorage has been failing in nightly builds > with NoClassDefFoundError in the tests. Sample error and stacktrace to > follow. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9529) Log correct cpu controller path on error while initializing CGroups.
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913033#comment-16913033 ] Rohith Sharma K S commented on YARN-9529: - Updated right fix version for branch-3.2 i.e 3.2.1 > Log correct cpu controller path on error while initializing CGroups. > > > Key: YARN-9529 > URL: https://issues.apache.org/jira/browse/YARN-9529 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 2.10.0, 3.2.0 >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Labels: 2.10.0, 3.3.0 > Fix For: 2.10.0, 3.0.4, 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9529.001.patch > > > The base cpu controller path is logged instead of the hadoop cgroup path. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9529) Log correct cpu controller path on error while initializing CGroups.
[ https://issues.apache.org/jira/browse/YARN-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-9529: Fix Version/s: (was: 3.2.2) 3.2.1 > Log correct cpu controller path on error while initializing CGroups. > > > Key: YARN-9529 > URL: https://issues.apache.org/jira/browse/YARN-9529 > Project: Hadoop YARN > Issue Type: Improvement > Components: nodemanager >Affects Versions: 2.10.0, 3.2.0 >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Labels: 2.10.0, 3.3.0 > Fix For: 2.10.0, 3.0.4, 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9529.001.patch > > > The base cpu controller path is logged instead of the hadoop cgroup path. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9282) Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'
[ https://issues.apache.org/jira/browse/YARN-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913035#comment-16913035 ] Rohith Sharma K S commented on YARN-9282: - Updated right fix version for branch-3.2 i.e 3.2.1 > Typo in javadoc of class LinuxContainerExecutor: > hadoop.security.authetication should be 'authentication' > - > > Key: YARN-9282 > URL: https://issues.apache.org/jira/browse/YARN-9282 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Charan Hebri >Priority: Trivial > Labels: newbie, newbie++ > Fix For: 2.10.0, 3.0.4, 3.3.0, 3.2.1, 2.9.3, 3.1.3 > > Attachments: YARN-9282.001.patch > > > There's a typo in the javadoc of class LinuxContainerExecutor: > "hadoop.security.authetication" should be "hadoop.security.authentication" -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9442) container working directory has group read permissions
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913034#comment-16913034 ] Rohith Sharma K S commented on YARN-9442: - Updated right fix version for branch-3.2 i.e 3.2.1 > container working directory has group read permissions > -- > > Key: YARN-9442 > URL: https://issues.apache.org/jira/browse/YARN-9442 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.2.2 >Reporter: Jim Brennan >Assignee: Jim Brennan >Priority: Minor > Fix For: 2.10.0, 3.0.4, 3.3.0, 2.8.6, 3.2.1, 2.9.3, 3.1.3 > > Attachments: YARN-9442-branch-2.8.001.patch, YARN-9442.001.patch, > YARN-9442.002.patch, YARN-9442.003.patch > > > Container working directories are currently created with permissions 0750, > owned by the user and with the group set to the node manager group. > Is there any reason why these directories need group read permissions? > I have been testing with group read permissions removed and so far I haven't > encountered any problems. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9442) container working directory has group read permissions
[ https://issues.apache.org/jira/browse/YARN-9442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-9442: Fix Version/s: (was: 3.2.2) 3.2.1 > container working directory has group read permissions > -- > > Key: YARN-9442 > URL: https://issues.apache.org/jira/browse/YARN-9442 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn >Affects Versions: 3.2.2 >Reporter: Jim Brennan >Assignee: Jim Brennan >Priority: Minor > Fix For: 2.10.0, 3.0.4, 3.3.0, 2.8.6, 3.2.1, 2.9.3, 3.1.3 > > Attachments: YARN-9442-branch-2.8.001.patch, YARN-9442.001.patch, > YARN-9442.002.patch, YARN-9442.003.patch > > > Container working directories are currently created with permissions 0750, > owned by the user and with the group set to the node manager group. > Is there any reason why these directories need group read permissions? > I have been testing with group read permissions removed and so far I haven't > encountered any problems. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9282) Typo in javadoc of class LinuxContainerExecutor: hadoop.security.authetication should be 'authentication'
[ https://issues.apache.org/jira/browse/YARN-9282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-9282: Fix Version/s: (was: 3.2.2) 3.2.1 > Typo in javadoc of class LinuxContainerExecutor: > hadoop.security.authetication should be 'authentication' > - > > Key: YARN-9282 > URL: https://issues.apache.org/jira/browse/YARN-9282 > Project: Hadoop YARN > Issue Type: Improvement >Reporter: Szilard Nemeth >Assignee: Charan Hebri >Priority: Trivial > Labels: newbie, newbie++ > Fix For: 2.10.0, 3.0.4, 3.3.0, 3.2.1, 2.9.3, 3.1.3 > > Attachments: YARN-9282.001.patch > > > There's a typo in the javadoc of class LinuxContainerExecutor: > "hadoop.security.authetication" should be "hadoop.security.authentication" -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9668) UGI conf doesn't read user overridden configurations on RM and NM startup
[ https://issues.apache.org/jira/browse/YARN-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-9668: Fix Version/s: (was: 3.2.2) 3.2.1 > UGI conf doesn't read user overridden configurations on RM and NM startup > - > > Key: YARN-9668 > URL: https://issues.apache.org/jira/browse/YARN-9668 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Fix For: 2.10.0, 3.0.4, 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9668-branch-2.001.patch, > YARN-9668-branch-2.002.patch, YARN-9668-branch-3.0.001.addendum, > YARN-9668-branch-3.2.001.patch, YARN-9668.001.patch, YARN-9668.002.patch, > YARN-9668.003.patch > > > Similar to HADOOP-15150. Configs overridden thru e.g. -D or -conf are not > passed to the UGI conf on RM or NM startup. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9559) Create AbstractContainersLauncher for pluggable ContainersLauncher logic
[ https://issues.apache.org/jira/browse/YARN-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913031#comment-16913031 ] Rohith Sharma K S commented on YARN-9559: - Updated right fix version for branch-3.2 i.e 3.2.1 > Create AbstractContainersLauncher for pluggable ContainersLauncher logic > > > Key: YARN-9559 > URL: https://issues.apache.org/jira/browse/YARN-9559 > Project: Hadoop YARN > Issue Type: Task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Fix For: 2.10.0, 3.0.4, 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9559.001.patch, YARN-9559.002.patch, > YARN-9559.003.patch, YARN-9559.004.patch, YARN-9559.005.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9559) Create AbstractContainersLauncher for pluggable ContainersLauncher logic
[ https://issues.apache.org/jira/browse/YARN-9559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith Sharma K S updated YARN-9559: Fix Version/s: (was: 3.2.2) 3.2.1 > Create AbstractContainersLauncher for pluggable ContainersLauncher logic > > > Key: YARN-9559 > URL: https://issues.apache.org/jira/browse/YARN-9559 > Project: Hadoop YARN > Issue Type: Task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Fix For: 2.10.0, 3.0.4, 3.3.0, 3.2.1, 3.1.3 > > Attachments: YARN-9559.001.patch, YARN-9559.002.patch, > YARN-9559.003.patch, YARN-9559.004.patch, YARN-9559.005.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9668) UGI conf doesn't read user overridden configurations on RM and NM startup
[ https://issues.apache.org/jira/browse/YARN-9668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913030#comment-16913030 ] Rohith Sharma K S commented on YARN-9668: - Updated right fix version for branch-3.2 i.e 3.2.1 > UGI conf doesn't read user overridden configurations on RM and NM startup > - > > Key: YARN-9668 > URL: https://issues.apache.org/jira/browse/YARN-9668 > Project: Hadoop YARN > Issue Type: Bug >Affects Versions: 2.10.0 >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Fix For: 2.10.0, 3.0.4, 3.3.0, 3.1.3, 3.2.2 > > Attachments: YARN-9668-branch-2.001.patch, > YARN-9668-branch-2.002.patch, YARN-9668-branch-3.0.001.addendum, > YARN-9668-branch-3.2.001.patch, YARN-9668.001.patch, YARN-9668.002.patch, > YARN-9668.003.patch > > > Similar to HADOOP-15150. Configs overridden thru e.g. -D or -conf are not > passed to the UGI conf on RM or NM startup. -- This message was sent by Atlassian Jira (v8.3.2#803003) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8200) Backport resource types/GPU features to branch-3.0/branch-2
[
https://issues.apache.org/jira/browse/YARN-8200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16913021#comment-16913021
]
Hadoop QA commented on YARN-8200:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m
38s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 73 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
54s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
43s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 13m
1s{color} | {color:green} branch-2 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 11m
39s{color} | {color:green} branch-2 passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 2m
52s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m
59s{color} | {color:green} branch-2 passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-project hadoop-yarn-project/hadoop-yarn
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-tests
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 10m
13s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 8m
28s{color} | {color:green} branch-2 passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
4s{color} | {color:green} branch-2 passed with JDK v1.8.0_222 {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
20s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
29s{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 12m
29s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 12m 29s{color}
| {color:red} root-jdk1.7.0_95 with JDK v1.7.0_95 generated 2 new + 1442
unchanged - 2 fixed = 1444 total (was 1444) {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 12m
10s{color} | {color:green} the patch passed with JDK v1.8.0_222 {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 12m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 12m
10s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
2m 42s{color} | {color:orange} root: The patch generated 194 new + 3977
unchanged - 73 fixed = 4171 total (was 4050) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 11m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
10s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
1s{color} | {color:red} The patch 524 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
18s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:blue}0{color} |
