[jira] [Commented] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[
https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739038#comment-16739038
]
Hadoop QA commented on YARN-9184:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m
11s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
55s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 22s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
50s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
11s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 2m
9s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 59s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
40s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
9s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 18m
48s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
14s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
[jira] [Commented] (YARN-7086) Release all containers aynchronously
[ https://issues.apache.org/jira/browse/YARN-7086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739035#comment-16739035 ] Manikandan R commented on YARN-7086: [~asuresh] [~leftnoteasy] [~sunilg] Can you please share your views? > Release all containers aynchronously > > > Key: YARN-7086 > URL: https://issues.apache.org/jira/browse/YARN-7086 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager >Reporter: Arun Suresh >Assignee: Manikandan R >Priority: Major > Attachments: YARN-7086.001.patch, YARN-7086.002.patch, > YARN-7086.Perf-test-case.patch > > > We have noticed in production two situations that can cause deadlocks and > cause scheduling of new containers to come to a halt, especially with regard > to applications that have a lot of live containers: > # When these applicaitons release these containers in bulk. > # When these applications terminate abruptly due to some failure, the > scheduler releases all its live containers in a loop. > To handle the issues mentioned above, we have a patch in production to make > sure ALL container releases happen asynchronously - and it has served us well. > Opening this JIRA to gather feedback on if this is a good idea generally (cc > [~leftnoteasy], [~jlowe], [~curino], [~kasha], [~subru], [~roniburd]) > BTW, In YARN-6251, we already have an asyncReleaseContainer() in the > AbstractYarnScheduler and a corresponding scheduler event, which is currently > used specifically for the container-update code paths (where the scheduler > realeases temp containers which it creates for the update) -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-7018) Interface for adding extra behavior to node heartbeats
[ https://issues.apache.org/jira/browse/YARN-7018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739033#comment-16739033 ] Manikandan R commented on YARN-7018: [~leftnoteasy] Can you please take a look? Thanks > Interface for adding extra behavior to node heartbeats > -- > > Key: YARN-7018 > URL: https://issues.apache.org/jira/browse/YARN-7018 > Project: Hadoop YARN > Issue Type: New Feature > Components: resourcemanager >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Major > Attachments: YARN-7018.POC.001.patch, YARN-7018.POC.002.patch, > YARN-7018.POC.003.patch, YARN-7018.POC.004.patch, YARN-7018.POC.005.patch > > > This JIRA tracks an interface for plugging in new behavior to node heartbeat > processing. Adding a formal interface for additional node heartbeat > processing would allow admins to configure new functionality that is > scheduler-independent without needing to replace the entire scheduler. For > example, both YARN-5202 and YARN-5215 had approaches where node heartbeat > processing was extended to implement new functionality that was essentially > scheduler-independent and could be implemented as a plugin with this > interface. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9189) Clarify FairScheduler submission logging
[ https://issues.apache.org/jira/browse/YARN-9189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] BELUGA BEHR updated YARN-9189: -- Attachment: YARN-9189.2.patch > Clarify FairScheduler submission logging > > > Key: YARN-9189 > URL: https://issues.apache.org/jira/browse/YARN-9189 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Affects Versions: 3.2.0 >Reporter: Patrick Bayne >Priority: Minor > Attachments: YARN-9189.1.patch, YARN-9189.2.patch > > > Logging was ambiguous for the fairscheduler. It was unclear if the "total > number applications" was referring to the global total or the queue's total. > Fixed wording/spelling of output logging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9189) Clarify FairScheduler submission logging
[
https://issues.apache.org/jira/browse/YARN-9189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738968#comment-16738968
]
Hadoop QA commented on YARN-9189:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 7s{color}
| {color:red} YARN-9189 does not apply to trunk. Rebase required? Wrong Branch?
See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-9189 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954387/YARN-9189.2.patch |
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23048/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Clarify FairScheduler submission logging
>
>
> Key: YARN-9189
> URL: https://issues.apache.org/jira/browse/YARN-9189
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: scheduler
>Affects Versions: 3.2.0
>Reporter: Patrick Bayne
>Priority: Minor
> Attachments: YARN-9189.1.patch, YARN-9189.2.patch
>
>
> Logging was ambiguous for the fairscheduler. It was unclear if the "total
> number applications" was referring to the global total or the queue's total.
> Fixed wording/spelling of output logging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[ https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738960#comment-16738960 ] Zhaohui Xin edited comment on YARN-9184 at 1/10/19 3:42 AM: Hi, [~ebadger], very goog suggestions. I attached patch, can you help me review this? :D was (Author: uranus): [~ebadger], very goog suggestions. I attached patch, can you help me review this? :D > Docker run doesn't pull down latest image if the image exists locally > -- > > Key: YARN-9184 > URL: https://issues.apache.org/jira/browse/YARN-9184 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0, 3.0.3 >Reporter: Zhaohui Xin >Assignee: Zhaohui Xin >Priority: Major > Attachments: YARN-9184.001.patch > > > See [docker run doesn't pull down latest image if the image exists > locally|https://github.com/moby/moby/issues/13331]. > So, I think we should pull image before run to make image always latest. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[ https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738960#comment-16738960 ] Zhaohui Xin commented on YARN-9184: --- [~ebadger], very goog suggestions. I attached patch, can you help me review this? :D > Docker run doesn't pull down latest image if the image exists locally > -- > > Key: YARN-9184 > URL: https://issues.apache.org/jira/browse/YARN-9184 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0, 3.0.3 >Reporter: Zhaohui Xin >Assignee: Zhaohui Xin >Priority: Major > Attachments: YARN-9184.001.patch > > > See [docker run doesn't pull down latest image if the image exists > locally|https://github.com/moby/moby/issues/13331]. > So, I think we should pull image before run to make image always latest. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9184) Docker run doesn't pull down latest image if the image exists locally
[ https://issues.apache.org/jira/browse/YARN-9184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738960#comment-16738960 ] Zhaohui Xin edited comment on YARN-9184 at 1/10/19 3:42 AM: Hi, [~ebadger], very good suggestions. I attached patch, can you help me review this? :D was (Author: uranus): Hi, [~ebadger], very goog suggestions. I attached patch, can you help me review this? :D > Docker run doesn't pull down latest image if the image exists locally > -- > > Key: YARN-9184 > URL: https://issues.apache.org/jira/browse/YARN-9184 > Project: Hadoop YARN > Issue Type: Sub-task > Components: nodemanager >Affects Versions: 3.1.0, 3.0.3 >Reporter: Zhaohui Xin >Assignee: Zhaohui Xin >Priority: Major > Attachments: YARN-9184.001.patch > > > See [docker run doesn't pull down latest image if the image exists > locally|https://github.com/moby/moby/issues/13331]. > So, I think we should pull image before run to make image always latest. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Comment Edited] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738921#comment-16738921 ] Zac Zhou edited comment on YARN-9161 at 1/10/19 2:36 AM: - The test case is broken by HDFS-14084 and resolved by YARN-9183 was (Author: yuan_zac): The test case is broken by [YARN-9183|https://issues.apache.org/jira/browse/YARN-9183] > Absolute resources of capacity scheduler doesn't support GPU and FPGA > - > > Key: YARN-9161 > URL: https://issues.apache.org/jira/browse/YARN-9161 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Attachments: YARN-9161.001.patch, YARN-9161.002.patch, > YARN-9161.003.patch > > > As the enum CapacitySchedulerConfiguration.AbsoluteResourceType only has two > elements: memory and vcores, which would filter out absolute resources > configuration of gpu and fpga in > AbstractCSQueue.updateConfigurableResourceRequirement. > This issue would cause gpu and fpga can't be allocated correctly -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738921#comment-16738921 ] Zac Zhou commented on YARN-9161: The test case is broken by [YARN-9183|https://issues.apache.org/jira/browse/YARN-9183] > Absolute resources of capacity scheduler doesn't support GPU and FPGA > - > > Key: YARN-9161 > URL: https://issues.apache.org/jira/browse/YARN-9161 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Attachments: YARN-9161.001.patch, YARN-9161.002.patch, > YARN-9161.003.patch > > > As the enum CapacitySchedulerConfiguration.AbsoluteResourceType only has two > elements: memory and vcores, which would filter out absolute resources > configuration of gpu and fpga in > AbstractCSQueue.updateConfigurableResourceRequirement. > This issue would cause gpu and fpga can't be allocated correctly -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9189) Clarify FairScheduler submission logging
[
https://issues.apache.org/jira/browse/YARN-9189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738910#comment-16738910
]
Hadoop QA commented on YARN-9189:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
9s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 33s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
28s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
27s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 27s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
28s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 4m
8s{color} | {color:red} patch has errors when building and testing our client
artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
25s{color} | {color:red} hadoop-yarn-server-resourcemanager in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 28s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 46m 37s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9189 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954378/YARN-9189.1.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 02076bcd15d8 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / c634589 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
https://builds.apache.org/job/PreCommit-YARN-Build/23046/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
|
| compile |
[jira] [Updated] (YARN-9189) Clarify FairScheduler submission logging
[ https://issues.apache.org/jira/browse/YARN-9189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Bayne updated YARN-9189: Attachment: YARN-9189.1.patch > Clarify FairScheduler submission logging > > > Key: YARN-9189 > URL: https://issues.apache.org/jira/browse/YARN-9189 > Project: Hadoop YARN > Issue Type: Improvement > Components: scheduler >Affects Versions: 3.2.0 >Reporter: Patrick Bayne >Priority: Minor > Attachments: YARN-9189.1.patch > > > Logging was ambiguous for the fairscheduler. It was unclear if the "total > number applications" was referring to the global total or the queue's total. > Fixed wording/spelling of output logging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9189) Clarify FairScheduler submission logging
Patrick Bayne created YARN-9189: --- Summary: Clarify FairScheduler submission logging Key: YARN-9189 URL: https://issues.apache.org/jira/browse/YARN-9189 Project: Hadoop YARN Issue Type: Improvement Components: scheduler Affects Versions: 3.2.0 Reporter: Patrick Bayne Logging was ambiguous for the fairscheduler. It was unclear if the "total number applications" was referring to the global total or the queue's total. Fixed wording/spelling of output logging. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9187) Backport YARN-6852 for GPU-specific native changes to branch-2
[
https://issues.apache.org/jira/browse/YARN-9187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738851#comment-16738851
]
Hadoop QA commented on YARN-9187:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} YARN-8200 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
40s{color} | {color:green} YARN-8200 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} YARN-8200 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
40s{color} | {color:green} YARN-8200 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}184m 13s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
24s{color} | {color:red} The patch generated 2 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}202m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:a716388 |
| JIRA Issue | YARN-9187 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954235/YARN-9187-YARN-8200.001.patch
|
| Optional Tests | dupname asflicense compile cc mvnsite javac unit |
| uname | Linux 6f4c84ef0078 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | YARN-8200 / 12568a2 |
| maven | version: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_181 |
| unit |
https://builds.apache.org/job/PreCommit-YARN-Build/23041/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23041/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-YARN-Build/23041/artifact/out/patch-asflicense-problems.txt
|
| Max. process+thread count | 163 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23041/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Backport YARN-6852 for GPU-specific native changes to branch-2
> --
>
> Key: YARN-9187
> URL: https://issues.apache.org/jira/browse/YARN-9187
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jonathan Hung
>Assignee: Jonathan Hung
>Priority: Major
> Attachments: YARN-9187-YARN-8200.001.patch
>
>
> YARN-6852 adds native GPU support, including
> # general native code refactoring
> # GPU specific native code
> 1 is handled by YARN-7321 in branch-2. This ticket is for handling 2 in
> branch-2.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To
[jira] [Resolved] (YARN-9183) TestAMRMTokens fails
[ https://issues.apache.org/jira/browse/YARN-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe resolved YARN-9183. -- Resolution: Done HDFS-14084 was reverted so this should now be fixed. > TestAMRMTokens fails > > > Key: YARN-9183 > URL: https://issues.apache.org/jira/browse/YARN-9183 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Akira Ajisaka >Assignee: Abhishek Modi >Priority: Blocker > > TestAMRMTokens.testMasterKeyRollOver and TestAMRMTokens.testTokenExpiry is > failing. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9183) TestAMRMTokens fails
[
https://issues.apache.org/jira/browse/YARN-9183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jason Lowe updated YARN-9183:
-
Priority: Blocker (was: Major)
I think this is much worse than just a failed unit test. A simple MapReduce
sleep job no longer succeeds with each map task failing with what looks like
the same error:
{noformat}
2019-01-09 22:59:36,423 WARN [main] org.apache.hadoop.mapred.YarnChild:
Exception running child : org.apache.hadoop.metrics2.MetricsException: Metrics
source RpcDetailedActivityForPort-1 already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:152)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:125)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:229)
at
org.apache.hadoop.ipc.metrics.RpcDetailedMetrics.create(RpcDetailedMetrics.java:55)
at org.apache.hadoop.ipc.Client.(Client.java:1341)
at org.apache.hadoop.ipc.ClientCache.getClient(ClientCache.java:57)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.(ProtobufRpcEngine.java:149)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.(ProtobufRpcEngine.java:136)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.(ProtobufRpcEngine.java:120)
at
org.apache.hadoop.ipc.ProtobufRpcEngine.getProxy(ProtobufRpcEngine.java:102)
at org.apache.hadoop.ipc.RPC.getProtocolProxy(RPC.java:624)
at
org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithAlignmentContext(NameNodeProxiesClient.java:370)
at
org.apache.hadoop.hdfs.NameNodeProxiesClient.createNonHAProxyWithClientProtocol(NameNodeProxiesClient.java:348)
at
org.apache.hadoop.hdfs.server.namenode.ha.ClientHAProxyFactory.createProxy(ClientHAProxyFactory.java:46)
at
org.apache.hadoop.hdfs.server.namenode.ha.AbstractNNFailoverProxyProvider.createProxyIfNeeded(AbstractNNFailoverProxyProvider.java:152)
at
org.apache.hadoop.hdfs.server.namenode.ha.IPFailoverProxyProvider.getProxy(IPFailoverProxyProvider.java:57)
at
org.apache.hadoop.hdfs.server.namenode.ha.IPFailoverProxyProvider.getProxy(IPFailoverProxyProvider.java:44)
at
org.apache.hadoop.io.retry.RetryInvocationHandler$ProxyDescriptor.(RetryInvocationHandler.java:197)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.(RetryInvocationHandler.java:328)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.(RetryInvocationHandler.java:322)
at org.apache.hadoop.io.retry.RetryProxy.create(RetryProxy.java:59)
at
org.apache.hadoop.hdfs.NameNodeProxiesClient.createHAProxy(NameNodeProxiesClient.java:326)
at
org.apache.hadoop.hdfs.NameNodeProxiesClient.createProxyWithClientProtocol(NameNodeProxiesClient.java:144)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:356)
at org.apache.hadoop.hdfs.DFSClient.(DFSClient.java:290)
at
org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:176)
at
org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:3312)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:124)
at
org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:3361)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:3329)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:479)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:227)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:177)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:172)
2019-01-09 22:59:36,431 INFO [main] org.apache.hadoop.mapred.Task: Running
cleanup for the task
2019-01-09 22:59:36,432 INFO [main] org.apache.hadoop.mapred.YarnChild:
Exception cleaning up: java.lang.NullPointerException
at org.apache.hadoop.mapred.Task.taskCleanup(Task.java:1458)
at org.apache.hadoop.mapred.YarnChild$3.run(YarnChild.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:197)
2019-01-09 22:59:36,544 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics
system...
2019-01-09 22:59:36,544 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system
stopped.
2019-01-09 22:59:36,545 INFO [main]
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask
[jira] [Commented] (YARN-9003) Support multi-homed network for docker container
[
https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738749#comment-16738749
]
Hadoop QA commented on YARN-9003:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
19s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
39s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 28s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 8m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 8m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 13s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 20m 28s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
18s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
37s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}102m 31s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.nodemanager.amrmproxy.TestFederationInterceptor |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9003 |
| JIRA Patch URL |
[jira] [Commented] (YARN-9180) Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2
[
https://issues.apache.org/jira/browse/YARN-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738708#comment-16738708
]
Hadoop QA commented on YARN-9180:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} YARN-8200 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m
53s{color} | {color:green} YARN-8200 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} YARN-8200 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} YARN-8200 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
38s{color} | {color:green} YARN-8200 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
27s{color} | {color:green} YARN-8200 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
30s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
48s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed.
{color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 48s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
34s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 52s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 25s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:a716388 |
| JIRA Issue | YARN-9180 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954355/YARN-9180-YARN-8200.001.patch
|
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 44105319d688 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | YARN-8200 / 0fb1a38 |
| maven | version: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_181 |
| mvninstall |
https://builds.apache.org/job/PreCommit-YARN-Build/23045/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| compile |
https://builds.apache.org/job/PreCommit-YARN-Build/23045/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| javac |
https://builds.apache.org/job/PreCommit-YARN-Build/23045/artifact/out/patch-compile-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| mvnsite |
https://builds.apache.org/job/PreCommit-YARN-Build/23045/artifact/out/patch-mvnsite-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
|
| unit |
[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738672#comment-16738672
]
Ayush Saxena commented on YARN-8498:
Yetus ran on older version.
After patch Native compilation passes for me on UBUNTU-18.04/GCC 7.3.0
{noformat}
[INFO] Reactor Summary:
[INFO]
[INFO] Apache Hadoop YARN . SUCCESS [ 3.270 s]
[INFO] Apache Hadoop YARN API . SUCCESS [ 13.977 s]
[INFO] Apache Hadoop YARN Common .. SUCCESS [ 11.779 s]
[INFO] Apache Hadoop YARN Server .. SUCCESS [ 0.610 s]
[INFO] Apache Hadoop YARN Server Common ... SUCCESS [ 7.117 s]
[INFO] Apache Hadoop YARN NodeManager . SUCCESS [ 54.988 s]
[INFO] Apache Hadoop YARN Web Proxy ... SUCCESS [ 6.875 s]
[INFO] Apache Hadoop YARN ApplicationHistoryService ... SUCCESS [ 5.056 s]
[INFO] Apache Hadoop YARN Timeline Service SUCCESS [ 3.475 s]
[INFO] Apache Hadoop YARN ResourceManager . SUCCESS [ 14.097 s]
[INFO] Apache Hadoop YARN Server Tests SUCCESS [ 3.539 s]
[INFO] Apache Hadoop YARN Client .. SUCCESS [ 4.673 s]
[INFO] Apache Hadoop YARN SharedCacheManager .. SUCCESS [ 3.990 s]
[INFO] Apache Hadoop YARN Timeline Plugin Storage . SUCCESS [ 3.146 s]
[INFO] Apache Hadoop YARN TimelineService HBase Backend ... SUCCESS [ 0.638 s]
[INFO] Apache Hadoop YARN TimelineService HBase Common SUCCESS [ 6.824 s]
[INFO] Apache Hadoop YARN TimelineService HBase Client SUCCESS [ 7.431 s]
[INFO] Apache Hadoop YARN TimelineService HBase Servers ... SUCCESS [ 0.578 s]
[INFO] Apache Hadoop YARN TimelineService HBase Server 1.2 SUCCESS [ 5.473 s]
[INFO] Apache Hadoop YARN TimelineService HBase tests . SUCCESS [ 6.380 s]
[INFO] Apache Hadoop YARN Router .. SUCCESS [ 5.389 s]
[INFO] Apache Hadoop YARN Applications SUCCESS [ 0.661 s]
[INFO] Apache Hadoop YARN DistributedShell SUCCESS [ 3.212 s]
[INFO] Apache Hadoop YARN Unmanaged Am Launcher ... SUCCESS [ 2.647 s]
[INFO] Apache Hadoop YARN Services SUCCESS [ 0.683 s]
[INFO] Apache Hadoop YARN Services Core ... SUCCESS [ 9.123 s]
[INFO] Apache Hadoop YARN Services API SUCCESS [ 4.718 s]
[INFO] Yet Another Learning Platform .. SUCCESS [ 3.505 s]
[INFO] Apache Hadoop YARN Site SUCCESS [ 0.612 s]
[INFO] Apache Hadoop YARN Registry SUCCESS [ 1.621 s]
[INFO] Apache Hadoop YARN UI .. SUCCESS [ 1.163 s]
[INFO] Apache Hadoop YARN CSI . SUCCESS [ 14.102 s]
[INFO] Apache Hadoop YARN Project . SUCCESS [ 3.415 s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 03:35 min
[INFO] Finished at: 2019-01-10T02:44:39+05:30
[INFO] Final Memory: 210M/870M
[INFO]
{noformat}
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8498-02.patch, YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error:
[jira] [Commented] (YARN-9180) Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2
[
https://issues.apache.org/jira/browse/YARN-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738682#comment-16738682
]
Hadoop QA commented on YARN-9180:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} docker {color} | {color:red} 3m
21s{color} | {color:red} Docker failed to build yetus/hadoop:a716388. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | YARN-9180 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954355/YARN-9180-YARN-8200.001.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23044/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2
> ---
>
> Key: YARN-9180
> URL: https://issues.apache.org/jira/browse/YARN-9180
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jonathan Hung
>Assignee: Jonathan Hung
>Priority: Major
> Attachments: YARN-9180-YARN-8200.001.patch,
> YARN-9180-YARN-8200.branch3.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9180) Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2
[ https://issues.apache.org/jira/browse/YARN-9180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9180: Attachment: YARN-9180-YARN-8200.001.patch > Port YARN-7033 NM recovery of assigned resources to branch-3.0/branch-2 > --- > > Key: YARN-9180 > URL: https://issues.apache.org/jira/browse/YARN-9180 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Attachments: YARN-9180-YARN-8200.001.patch, > YARN-9180-YARN-8200.branch3.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9003) Support multi-homed network for docker container
[ https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9003: Attachment: YARN-9003.006.patch > Support multi-homed network for docker container > > > Key: YARN-9003 > URL: https://issues.apache.org/jira/browse/YARN-9003 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Labels: docker > Attachments: YARN-9003.001.patch, YARN-9003.002.patch, > YARN-9003.003.patch, YARN-9003.004.patch, YARN-9003.005.patch, > YARN-9003.006.patch > > > Docker network can be defined as configuration properties - docker.network to > setup docker container to connect to a specific network in YARN service. > Docker can run multi-homed network by specifying --net=bridge > --net=private-net. This is useful to expose small number of front end > container and ports, while the rest of the infrastructure runs in private > network. This task is to add support for specifying multiple docker networks > to YARN service and docker support. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9003) Support multi-homed network for docker container
[ https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738662#comment-16738662 ] Eric Yang commented on YARN-9003: - Patch 006 fixed unit test error. > Support multi-homed network for docker container > > > Key: YARN-9003 > URL: https://issues.apache.org/jira/browse/YARN-9003 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Labels: docker > Attachments: YARN-9003.001.patch, YARN-9003.002.patch, > YARN-9003.003.patch, YARN-9003.004.patch, YARN-9003.005.patch, > YARN-9003.006.patch > > > Docker network can be defined as configuration properties - docker.network to > setup docker container to connect to a specific network in YARN service. > Docker can run multi-homed network by specifying --net=bridge > --net=private-net. This is useful to expose small number of front end > container and ports, while the rest of the infrastructure runs in private > network. This task is to add support for specifying multiple docker networks > to YARN service and docker support. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9188) Port YARN-7136 to branch-2
[
https://issues.apache.org/jira/browse/YARN-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738641#comment-16738641
]
Jonathan Hung commented on YARN-9188:
-
attached 001 patch which is the same as YARN-7136 from branch-3.0 minus an
unused {{import
org.apache.curator.shaded.com.google.common.reflect.ClassPath;}} which was
causing compilation issues.
> Port YARN-7136 to branch-2
> --
>
> Key: YARN-9188
> URL: https://issues.apache.org/jira/browse/YARN-9188
> Project: Hadoop YARN
> Issue Type: Sub-task
>Reporter: Jonathan Hung
>Assignee: Jonathan Hung
>Priority: Major
> Attachments: YARN-9188-YARN-8200.001.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9188) Port YARN-7136 to branch-2
[ https://issues.apache.org/jira/browse/YARN-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9188: Attachment: YARN-9188-YARN-8200.001.patch > Port YARN-7136 to branch-2 > -- > > Key: YARN-9188 > URL: https://issues.apache.org/jira/browse/YARN-9188 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Attachments: YARN-9188-YARN-8200.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9188) Port YARN-7136 to branch-2
[ https://issues.apache.org/jira/browse/YARN-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9188: Attachment: (was: YARN-9188-YARN-8200.001.patch) > Port YARN-7136 to branch-2 > -- > > Key: YARN-9188 > URL: https://issues.apache.org/jira/browse/YARN-9188 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Attachments: YARN-9188-YARN-8200.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9188) Port YARN-7136 to branch-2
[ https://issues.apache.org/jira/browse/YARN-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9188: Attachment: YARN-9188-YARN-8200.001.patch > Port YARN-7136 to branch-2 > -- > > Key: YARN-9188 > URL: https://issues.apache.org/jira/browse/YARN-9188 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > Attachments: YARN-9188-YARN-8200.001.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9188) Port YARN-7136 to branch-2
[ https://issues.apache.org/jira/browse/YARN-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9188: Attachment: (was: YARN-9187-YARN-8200.001.patch) > Port YARN-7136 to branch-2 > -- > > Key: YARN-9188 > URL: https://issues.apache.org/jira/browse/YARN-9188 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9188) Port YARN-7136 to branch-2
[ https://issues.apache.org/jira/browse/YARN-9188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hung updated YARN-9188: Attachment: YARN-9187-YARN-8200.001.patch > Port YARN-7136 to branch-2 > -- > > Key: YARN-9188 > URL: https://issues.apache.org/jira/browse/YARN-9188 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Jonathan Hung >Assignee: Jonathan Hung >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Created] (YARN-9188) Port YARN-7136 to branch-2
Jonathan Hung created YARN-9188: --- Summary: Port YARN-7136 to branch-2 Key: YARN-9188 URL: https://issues.apache.org/jira/browse/YARN-9188 Project: Hadoop YARN Issue Type: Sub-task Reporter: Jonathan Hung Assignee: Jonathan Hung -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9003) Support multi-homed network for docker container
[
https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738607#comment-16738607
]
Hadoop QA commented on YARN-9003:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
23s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 3m
51s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 24m
8s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 20s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 10s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 37s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
18s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}103m 23s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | TEST-cetest |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9003 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954340/YARN-9003.005.patch |
| Optional Tests | dupname asflicense
[jira] [Commented] (YARN-9003) Support multi-homed network for docker container
[
https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738581#comment-16738581
]
Hadoop QA commented on YARN-9003:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
24s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
19s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 8m
19s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
30s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
14s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 18s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
6s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 7m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 18s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Skipped patched modules with no Java source:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 19m 31s{color}
| {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
22s{color} | {color:green} hadoop-yarn-site in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
40s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 96m 32s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | TEST-cetest |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9003 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954337/YARN-9003.004.patch |
| Optional Tests | dupname asflicense
[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738530#comment-16738530
]
Hadoop QA commented on YARN-8498:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 22m
51s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 27m
3s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
42m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} cc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
15m 52s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 20m
52s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
41s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}107m 42s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-8498 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954329/YARN-8498-02.patch |
| Optional Tests | dupname asflicense compile cc mvnsite javac unit |
| uname | Linux cf461a1f1500 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 709ddb1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23037/testReport/ |
| Max. process+thread count | 307 (vs. ulimit of 1) |
| modules | C:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
U:
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager
|
| Console output |
https://builds.apache.org/job/PreCommit-YARN-Build/23037/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8498-02.patch, YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
>
[jira] [Commented] (YARN-9003) Support multi-homed network for docker container
[ https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738508#comment-16738508 ] Eric Yang commented on YARN-9003: - [~billie.rinaldi] Patch 005 added the new network configuration check, and updated documentation for multi-homed network. > Support multi-homed network for docker container > > > Key: YARN-9003 > URL: https://issues.apache.org/jira/browse/YARN-9003 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Labels: docker > Attachments: YARN-9003.001.patch, YARN-9003.002.patch, > YARN-9003.003.patch, YARN-9003.004.patch, YARN-9003.005.patch > > > Docker network can be defined as configuration properties - docker.network to > setup docker container to connect to a specific network in YARN service. > Docker can run multi-homed network by specifying --net=bridge > --net=private-net. This is useful to expose small number of front end > container and ports, while the rest of the infrastructure runs in private > network. This task is to add support for specifying multiple docker networks > to YARN service and docker support. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9003) Support multi-homed network for docker container
[ https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9003: Attachment: YARN-9003.005.patch > Support multi-homed network for docker container > > > Key: YARN-9003 > URL: https://issues.apache.org/jira/browse/YARN-9003 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Labels: docker > Attachments: YARN-9003.001.patch, YARN-9003.002.patch, > YARN-9003.003.patch, YARN-9003.004.patch, YARN-9003.005.patch > > > Docker network can be defined as configuration properties - docker.network to > setup docker container to connect to a specific network in YARN service. > Docker can run multi-homed network by specifying --net=bridge > --net=private-net. This is useful to expose small number of front end > container and ports, while the rest of the infrastructure runs in private > network. This task is to add support for specifying multiple docker networks > to YARN service and docker support. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9003) Support multi-homed network for docker container
[ https://issues.apache.org/jira/browse/YARN-9003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Yang updated YARN-9003: Attachment: YARN-9003.004.patch > Support multi-homed network for docker container > > > Key: YARN-9003 > URL: https://issues.apache.org/jira/browse/YARN-9003 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Eric Yang >Assignee: Eric Yang >Priority: Major > Labels: docker > Attachments: YARN-9003.001.patch, YARN-9003.002.patch, > YARN-9003.003.patch, YARN-9003.004.patch > > > Docker network can be defined as configuration properties - docker.network to > setup docker container to connect to a specific network in YARN service. > Docker can run multi-homed network by specifying --net=bridge > --net=private-net. This is useful to expose small number of front end > container and ports, while the rest of the infrastructure runs in private > network. This task is to add support for specifying multiple docker networks > to YARN service and docker support. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9080) Bucket Directories as part of ATS done accumulates
[ https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738443#comment-16738443 ] Prabhu Joseph commented on YARN-9080: - Hi [~rohithsharma], Can you review the patch when you get time. Thanks. > Bucket Directories as part of ATS done accumulates > -- > > Key: YARN-9080 > URL: https://issues.apache.org/jira/browse/YARN-9080 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: 0001-YARN-9080.patch, 0002-YARN-9080.patch, > 0003-YARN-9080.patch > > > Have observed older bucket directories cluster_timestamp, bucket1 and bucket2 > as part of ATS done accumulates. The cleanLogs part of EntityLogCleaner > removes only the app directories and not the bucket directories. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9116) Capacity Scheduler: add the default maximum-allocation-mb and maximum-allocation-vcores for the queues
[ https://issues.apache.org/jira/browse/YARN-9116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738441#comment-16738441 ] Aihua Xu commented on YARN-9116: Thanks [~leftnoteasy] and [~cheersyang] for the suggestion. Just to clarify the behavior: the queue's maximum-allocation will not exceed the global setting (yarn.scheduler.capacity.maximum-allocation-mb) to keep the compatibility; among the queues, the child inherits the setting from the parent and the child queue can override the parent queue with larger or smaller setting but still respecting the global setting. That sounds reasonable to me. I will work on the support of maximum-allocation-mb/vcores to parent queues first and have a follow up on the general resource type. > Capacity Scheduler: add the default maximum-allocation-mb and > maximum-allocation-vcores for the queues > -- > > Key: YARN-9116 > URL: https://issues.apache.org/jira/browse/YARN-9116 > Project: Hadoop YARN > Issue Type: Sub-task > Components: capacity scheduler >Affects Versions: 2.7.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Major > Attachments: YARN-9116.1.patch > > > YARN-1582 adds the support of maximum-allocation-mb configuration per queue > which is targeting to support larger container features on dedicated queues > (larger maximum-allocation-mb/maximum-allocation-vcores for such queue) . > While to achieve larger container configuration, we need to increase the > global maximum-allocation-mb/maximum-allocation-vcores (e.g. 120G/256) and > then override those configurations with desired values on the queues since > queue configuration can't be larger than cluster configuration. There are > many queues in the system and if we forget to configure such values when > adding a new queue, then such queue gets default 120G/256 which typically is > not what we want. > We can come up with a queue-default configuration (set to normal queue > configuration like 16G/8), so the leaf queues gets such values by default. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9080) Bucket Directories as part of ATS done accumulates
[
https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738438#comment-16738438
]
Hadoop QA commented on YARN-9080:
-
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 25m
21s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 54s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
33s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 12s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage:
The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 22s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
34s{color} | {color:green} hadoop-yarn-server-timeline-pluginstorage in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
27s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 59m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9080 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954321/0003-YARN-9080.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 0f6f6c93f7e5 3.13.0-153-generic #203-Ubuntu SMP Thu Jun 14
08:52:28 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 709ddb1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/23036/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-YARN-Build/23036/testReport/ |
| Max. process+thread count | 341 (vs. ulimit of 1) |
| modules | C:
[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738433#comment-16738433
]
Ayush Saxena commented on YARN-8498:
Thanx [~jlowe] for giving a look.
{quote}That's going to cause problems if a system ever decides to not typedef
pid_t to __pid_t, and systems are completely free to make that choice in the
future. Same goes for __WAIT_STATUS.
{quote}
That is what something happened for __WAIT_STATUS .
As of now _pid_t still seems to be present in GCC 7.But we definitely can't be
sure regarding its future too.Moreover I guess using these macros is not
helping us too in no way.
I have changed __pid_t to pid_t too in v2 as a preventive measure.
If someone wants to give a check that __pid_t is same as pid_t , can check
[here|https://github.com/trailofbits/abigen/blob/master/data/platforms/ubuntu/18.04.1/usr/include/sys/types.h#L98]
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8498-02.patch, YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error: expected ‘;’ before ‘oom_listener_status’
> __WAIT_STATUS oom_listener_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30:
> error: ‘oom_listener_status’ was not declared in this scope
> __pid_t exited1 = wait(oom_listener_status);
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ayush Saxena updated YARN-8498:
---
Attachment: YARN-8498-02.patch
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8498-02.patch, YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error: expected ‘;’ before ‘oom_listener_status’
> __WAIT_STATUS oom_listener_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30:
> error: ‘oom_listener_status’ was not declared in this scope
> __pid_t exited1 = wait(oom_listener_status);
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738362#comment-16738362
]
Jason Lowe commented on YARN-8498:
--
Just coming up to speed on this, so apologies if I'm missing something here.
To me the main issue is using private types when the code has no good reason to
do so. The [manpage for wait(2)|
http://man7.org/linux/man-pages/man2/wait.2.html] shows it returning a
{{pid_t}}, not a {{\_\_pid_t}}, so I don't understand why the code insists on
using the private {{\_\_pid_t}} type. That's going to cause problems if a
system ever decides to _not_ typedef {{pid_t}} to {{\_\_pid_t}}, and systems
are completely free to make that choice in the future. Same goes for
{{\_\_WAIT_STATUS}}. This is a private type that should not be used in user
code directly. The manpage shows that {{wait}} takes an {{int*}} argument, not
a {{\_\_WAIT_STATUS}}. This smells like someone coded directly against the
header files rather than reading the manpages, and the risk of the former is
using private, system-specific types that may not compile on other distros.
So I think fixing this for all Linux distros going forward is as simple as
changing {{\_\_pid_t}} to {{pid_t}} and {{\_\_WAIT_STATUS}} to {{int *}}.
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error: expected ‘;’ before ‘oom_listener_status’
> __WAIT_STATUS oom_listener_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30:
> error: ‘oom_listener_status’ was not declared in this scope
> __pid_t exited1 = wait(oom_listener_status);
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA
[ https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738327#comment-16738327 ] Zac Zhou commented on YARN-9161: [~sunilg], Thanks for your comments~ The failed unit tests seem not related to this patch from my understanding. > Absolute resources of capacity scheduler doesn't support GPU and FPGA > - > > Key: YARN-9161 > URL: https://issues.apache.org/jira/browse/YARN-9161 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Reporter: Zac Zhou >Assignee: Zac Zhou >Priority: Major > Attachments: YARN-9161.001.patch, YARN-9161.002.patch, > YARN-9161.003.patch > > > As the enum CapacitySchedulerConfiguration.AbsoluteResourceType only has two > elements: memory and vcores, which would filter out absolute resources > configuration of gpu and fpga in > AbstractCSQueue.updateConfigurableResourceRequirement. > This issue would cause gpu and fpga can't be allocated correctly -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9080) Bucket Directories as part of ATS done accumulates
[ https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9080: Attachment: 0003-YARN-9080.patch > Bucket Directories as part of ATS done accumulates > -- > > Key: YARN-9080 > URL: https://issues.apache.org/jira/browse/YARN-9080 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: 0001-YARN-9080.patch, 0002-YARN-9080.patch, > 0003-YARN-9080.patch > > > Have observed older bucket directories cluster_timestamp, bucket1 and bucket2 > as part of ATS done accumulates. The cleanLogs part of EntityLogCleaner > removes only the app directories and not the bucket directories. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9080) Bucket Directories as part of ATS done accumulates
[
https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738243#comment-16738243
]
Hadoop QA commented on YARN-9080:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
26s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 54s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
31s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 12s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage:
The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11) {color}
|
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 41s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 2m 41s{color}
| {color:red} hadoop-yarn-server-timeline-pluginstorage in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
25s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 55m 34s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.timeline.TestEntityGroupFSTimelineStore |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9080 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954309/0002-YARN-9080.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux 02dfd2694500 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 709ddb1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-YARN-Build/23035/artifact/out/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt
|
| unit |
[jira] [Commented] (YARN-9050) Usability improvements for scheduler activities
[
https://issues.apache.org/jira/browse/YARN-9050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738219#comment-16738219
]
Tao Yang commented on YARN-9050:
Hi [~leftnoteasy], [~cheersyang]
Design doc is attached
[here|https://docs.google.com/document/d/1pwf-n3BCLW76bGrmNPM4T6pQ3vC4dVMcN2Ud1hq1t2M/edit#heading=h.2jnaobmmfne5],
please help to review in your free time. Thanks!
> Usability improvements for scheduler activities
> ---
>
> Key: YARN-9050
> URL: https://issues.apache.org/jira/browse/YARN-9050
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: capacityscheduler
>Reporter: Tao Yang
>Assignee: Tao Yang
>Priority: Major
> Attachments: image-2018-11-23-16-46-38-138.png
>
>
> We have did some usability improvements for scheduler activities based on
> YARN3.1 in our cluster as follows:
> 1. Not available for multi-thread asynchronous scheduling. App and node
> activites maybe confused when multiple scheduling threads record activites of
> different allocation processes in the same variables like appsAllocation and
> recordingNodesAllocation in ActivitiesManager. I think these variables should
> be thread-local to make activities clear among multiple threads.
> 2. Incomplete activites for multi-node lookup machanism, since
> ActivitiesLogger will skip recording through {{if (node == null ||
> activitiesManager == null) }} when node is null which represents this
> allocation is for multi-nodes. We need support recording activities for
> multi-node lookup machanism.
> 3. Current app activites can not meet requirements of diagnostics, for
> example, we can know that node doesn't match request but hard to know why,
> especially when using placement constraints, it's difficult to make a
> detailed diagnosis manually. So I propose to improve the diagnoses of
> activites, add diagnosis for placement constraints check, update insufficient
> resource diagnosis with detailed info (like 'insufficient resource
> names:[memory-mb]') and so on.
> 4. Add more useful fields for app activities, in some scenarios we need to
> distinguish different requests but can't locate requests based on app
> activities info, there are some other fields can help to filter what we want
> such as allocation tags. We have added containerPriority, allocationRequestId
> and allocationTags fields in AppAllocation.
> 5. Filter app activities by key fields, sometimes the results of app
> activities is massive, it's hard to find what we want. We have support filter
> by allocation-tags to meet requirements from some apps, more over, we can
> take container-priority and allocation-request-id as candidates if necessary.
> 6. Aggragate app activities by diagnoses. For a single allocation process,
> activities still can be massive in a large cluster, we frequently want to
> know why request can't be allocated in cluster, it's hard to check every node
> manually in a large cluster, so that aggragation for app activities by
> diagnoses is neccessary to solve this trouble. We have added groupingType
> parameter for app-activities REST API for this, supports grouping by
> diagnositics and example like this:
> !image-2018-11-23-16-46-38-138.png!
> I think we can have a discuss about these points, useful improvements which
> can be accepted will be added into the patch. Thanks.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-9080) Bucket Directories as part of ATS done accumulates
[ https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9080: Attachment: 0002-YARN-9080.patch > Bucket Directories as part of ATS done accumulates > -- > > Key: YARN-9080 > URL: https://issues.apache.org/jira/browse/YARN-9080 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: 0001-YARN-9080.patch, 0002-YARN-9080.patch > > > Have observed older bucket directories cluster_timestamp, bucket1 and bucket2 > as part of ATS done accumulates. The cleanLogs part of EntityLogCleaner > removes only the app directories and not the bucket directories. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9080) Bucket Directories as part of ATS done accumulates
[
https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738169#comment-16738169
]
Hadoop QA commented on YARN-9080:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m
35s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
16s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
12m 59s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
18s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
20s{color} | {color:red} hadoop-yarn-server-timeline-pluginstorage in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
20s{color} | {color:red} hadoop-yarn-server-timeline-pluginstorage in the patch
failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 20s{color}
| {color:red} hadoop-yarn-server-timeline-pluginstorage in the patch failed.
{color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 12s{color} | {color:orange}
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timeline-pluginstorage:
The patch generated 1 new + 11 unchanged - 0 fixed = 12 total (was 11) {color}
|
| {color:red}-1{color} | {color:red} mvnsite {color} | {color:red} 0m
21s{color} | {color:red} hadoop-yarn-server-timeline-pluginstorage in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 9s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
21s{color} | {color:red} hadoop-yarn-server-timeline-pluginstorage in the patch
failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 0m 23s{color}
| {color:red} hadoop-yarn-server-timeline-pluginstorage in the patch failed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 1s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA Issue | YARN-9080 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954305/0001-YARN-9080.patch |
| Optional Tests | dupname asflicense compile javac javadoc mvninstall
mvnsite unit shadedclient findbugs checkstyle |
| uname | Linux f4dc46254e20 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 709ddb1 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.1.0-RC1 |
| mvninstall |
https://builds.apache.org/job/PreCommit-YARN-Build/23034/artifact/out/patch-mvninstall-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timeline-pluginstorage.txt
|
| compile |
[jira] [Updated] (YARN-9080) Bucket Directories as part of ATS done accumulates
[ https://issues.apache.org/jira/browse/YARN-9080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prabhu Joseph updated YARN-9080: Attachment: 0001-YARN-9080.patch > Bucket Directories as part of ATS done accumulates > -- > > Key: YARN-9080 > URL: https://issues.apache.org/jira/browse/YARN-9080 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Prabhu Joseph >Assignee: Prabhu Joseph >Priority: Major > Attachments: 0001-YARN-9080.patch > > > Have observed older bucket directories cluster_timestamp, bucket1 and bucket2 > as part of ATS done accumulates. The cleanLogs part of EntityLogCleaner > removes only the app directories and not the bucket directories. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-9161) Absolute resources of capacity scheduler doesn't support GPU and FPGA
[
https://issues.apache.org/jira/browse/YARN-9161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738124#comment-16738124
]
Hadoop QA commented on YARN-9161:
-
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
21s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
13s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 21m
18s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
46s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
42s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
19m 58s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
37s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
49s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
14s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
35s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
1m 32s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn: The patch
generated 2 new + 363 unchanged - 6 fixed = 365 total (was 369) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 3m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 15s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 6m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
48s{color} | {color:green} hadoop-yarn-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
26s{color} | {color:green} hadoop-yarn-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 10s{color}
| {color:red} hadoop-yarn-server-resourcemanager in the patch failed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 16m
10s{color} | {color:green} hadoop-yarn-applications-distributedshell in the
patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
43s{color} | {color:green} hadoop-yarn-submarine in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}215m 19s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests |
hadoop.yarn.server.resourcemanager.security.TestAMRMTokens |
| | hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceFair |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8f97d6f |
| JIRA
[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16738016#comment-16738016
]
Ayush Saxena commented on YARN-8498:
Thanx [~sunilg] for the review.
I guess sys/wat.h is also not having this macro in 18.04
It is peresent in ver 14.04 here
https://github.com/trailofbits/abigen/blob/master/data/platforms/ubuntu/14.04.5/usr/include/x86_64-linux-gnu/sys/wait.h#L71
But not in 18.04 here :
https://github.com/trailofbits/abigen/blob/master/data/platforms/ubuntu/18.04.1/usr/include/sys/wait.h
IIRC There used to be some problem of multiple definitions conflicts too when
stdlib.h and sys/wait.h were used together.Don't know whether that has been
resolved now.
If we want to use macro only. We can define explicitly if not defined in the
code using #ifndef _WAIT_STATUS then define it.
Not sure that will optimize anything in this case. MACRO just replaces the word
with definition.By functionality point of view I guess there shouldn't be any
as such problem.
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error: expected ‘;’ before ‘oom_listener_status’
> __WAIT_STATUS oom_listener_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30:
> error: ‘oom_listener_status’ was not declared in this scope
> __pid_t exited1 = wait(oom_listener_status);
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-8498) Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
[
https://issues.apache.org/jira/browse/YARN-8498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737970#comment-16737970
]
Sunil Govindan commented on YARN-8498:
--
[~qiyuangong] [~bibinchundatt]
Cud u pls help to verify this patch in older gcc and newer one in couple of OS.
[~leftnoteasy] [~jlowe] [~vinodkv] cud u pls help to take a look if possible.
> Yarn NodeManager OOM Listener Fails Compilation on Ubuntu 18.04
> ---
>
> Key: YARN-8498
> URL: https://issues.apache.org/jira/browse/YARN-8498
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jack Bearden
>Assignee: Ayush Saxena
>Priority: Blocker
> Attachments: YARN-8948-01.patch
>
>
> While building this project, I ran into a few compilation errors here. The
> first one was in this file:
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/impl/oom_listener_main.c
> At the very end, during the compilation of the OOM test, it fails again:
>
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:256:7:
> error: ‘__WAIT_STATUS’ was not declared in this scope
> __WAIT_STATUS mem_hog_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:257:30:
> error: ‘mem_hog_status’ was not declared in this scope
> __pid_t exited0 = wait(mem_hog_status);
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:275:21:
> error: expected ‘;’ before ‘oom_listener_status’
> __WAIT_STATUS oom_listener_status = {};
> hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/native/oom-listener/test/oom_listener_test_main.cc:276:30:
> error: ‘oom_listener_status’ was not declared in this scope
> __pid_t exited1 = wait(oom_listener_status);
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
