[jira] [Commented] (YARN-6550) Capture launch_container.sh logs to a separate log file
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451046#comment-16451046 ] Hudson commented on YARN-6550: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14057 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/14057/]) YARN-6550. Capture launch_container.sh logs to a separate log file. (xyao: rev ae5e4aac0acd60f3045b62e8d0d107f709761275) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java > Capture launch_container.sh logs to a separate log file > --- > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad >Priority: Major > Fix For: 2.9.0, 3.0.0 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch, > YARN-6550.011.patch, YARN-6550.011.patch, YARN-6550.012.patch, > YARN-6550.branch-2.001.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs to a separate log file
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16189028#comment-16189028 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 12m 32s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} branch-2 Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 11s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 35s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 23s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 37s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 6s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s{color} | {color:green} branch-2 passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 31s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 31s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 1 new + 24 unchanged - 1 fixed = 25 total (was 25) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 20s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 29 new + 161 unchanged - 0 fixed = 190 total (was 161) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 42m 24s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 22s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 75m 31s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.TestNodeManagerReboot | | Timed out junit tests | org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater | | | org.apache.hadoop.yarn.server.nodemanager.TestNodeManagerResync | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:eaf5c66 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12889396/YARN-6550.branch-2.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 012d3844b1d6 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | branch-2 / 1eecf8a | | Default Java | 1.7.0_151 | | findbugs | v3.0.0 | | javac | https://builds.apache.org/job/PreCommit-YARN-Build/17736/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17736/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/17736/artifact/pa
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs to a separate log file
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186728#comment-16186728 ] Hudson commented on YARN-6550: -- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #12999 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/12999/]) YARN-6550. Capture launch_container.sh logs to a separate log file. (wangda: rev febeead5f95c6fc245ea3735f5b538d4bb4dc8a4) * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/DefaultContainerExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/TestContainerLaunch.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/ContainerExecutor.java * (edit) hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/LinuxContainerExecutor.java > Capture launch_container.sh logs to a separate log file > --- > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch, > YARN-6550.011.patch, YARN-6550.011.patch, YARN-6550.012.patch, > YARN-6550.branch-2.001.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs to a separate log file
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16186711#comment-16186711 ] Wangda Tan commented on YARN-6550: -- Just pushed to trunk/branch-3.0. Thanks [~suma.shivaprasad] and thanks reviews from [~aw]! > Capture launch_container.sh logs to a separate log file > --- > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch, > YARN-6550.011.patch, YARN-6550.011.patch, YARN-6550.012.patch, > YARN-6550.branch-2.001.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183578#comment-16183578 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 49s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} branch-2 Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 38s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 34s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 51s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 34s{color} | {color:green} branch-2 passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 39s{color} | {color:green} branch-2 passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 44s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 48s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 48s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 1 new + 24 unchanged - 1 fixed = 25 total (was 25) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 31s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 29 new + 162 unchanged - 0 fixed = 191 total (was 162) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 47s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 50s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 46m 33s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 47s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 72m 41s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.TestNodeManagerReboot | | | hadoop.yarn.server.nodemanager.containermanager.TestContainerManager | | | hadoop.yarn.server.nodemanager.TestNodeStatusUpdater | | | hadoop.yarn.server.nodemanager.containermanager.scheduler.TestContainerSchedulerQueuing | | | hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels | | Timed out junit tests | org.apache.hadoop.yarn.server.nodemanager.TestNodeManagerShutdown | | | org.apache.hadoop.yarn.server.nodemanager.TestNodeManagerResync | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:eaf5c66 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12889396/YARN-6550.branch-2.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 5398bf427a08 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | branch-2 / c143708 | | Default Java | 1.7.0_151 | | findbugs | v3.0.0 | | javac | https://builds.apache.org/job/PreCommit-YARN-Build/17673/artifact/patchprocess/diff-compile-javac-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-ya
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16183501#comment-16183501 ] Suma Shivaprasad commented on YARN-6550: This patch was tested internally with runs of mapreduce, distributed shell apps to valudate container launches without any issues > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch, > YARN-6550.011.patch, YARN-6550.011.patch, YARN-6550.012.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16177320#comment-16177320 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 26s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 25s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 20s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 30 new + 140 unchanged - 1 fixed = 170 total (was 141) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 58s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 59s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 40m 59s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.TestContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12888594/YARN-6550.012.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux ee3b842398e3 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 164a063 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17604/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/17604/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17604/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/17604/console | | Powered by | Apache Yetus 0.6.0-
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176705#comment-16176705 ] Wangda Tan commented on YARN-6550: -- Thanks [~suma.shivaprasad] for updating the patch, +1 to latest patch, will commit next Mon if no opposite opinions. > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch, > YARN-6550.011.patch, YARN-6550.011.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175671#comment-16175671 ] Hadoop QA commented on YARN-6550: - | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 18s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 10s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 40s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 23s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 51s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 22s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 30 new + 140 unchanged - 1 fixed = 170 total (was 141) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 5s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 15m 8s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 41m 37s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12888385/YARN-6550.011.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 83bfa42067e3 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / bfd1a72 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17580/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17580/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/17580/console | | Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN >
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16175426#comment-16175426 ] Wangda Tan commented on YARN-6550: -- Thanks [~suma.shivaprasad], +1 to skip printing stack trace for non zero exit code. In general patch LGTM, only one naming suggestions in generated script for readability: - {{export STDOUT}} to {{PRELAUNCH_OUT}}, and {{STDERR}} to {{PRELAUNCH_ERR}} > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch, > YARN-6550.011.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174091#comment-16174091 ] Suma Shivaprasad commented on YARN-6550: Launch container script after patch update {noformat} #!/bin/bash set -o pipefail -e export STDOUT="/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505948297782_0008/container_1505948297782_0008_02_01/prelaunch.out" exec >"${STDOUT}" export STDERR="/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505948297782_0008/container_1505948297782_0008_02_01/prelaunch.err" exec 2>"${STDERR}" echo "Setting up env variables" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/etc/hadoop"} export JAVA_HOME=${JAVA_HOME:-"/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home"} export APP_SUBMIT_TIME_ENV="1505955689174" export NM_HOST="10.22.16.92" export LD_LIBRARY_PATH="$PWD" export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT"} export LOGNAME="sshivaprasad" export JVM_PID="$$" export HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT"} export PWD="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505948297782_0008/container_1505948297782_0008_02_01" export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT"} export LOCAL_DIRS="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505948297782_0008" export APPLICATION_WEB_PROXY_BASE="/proxy/application_1505948297782_0008" export SHELL="/bin/bash" export NM_HTTP_PORT="8042" export LOG_DIRS="/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505948297782_0008/container_1505948297782_0008_02_01" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gA=^M " export NM_PORT="57171" export USER="sshivaprasad" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT"} export CLASSPATH="$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:$PWD/*" export HADOOP_TOKEN_FILE_LOCATION="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505948297782_0008/container_1505948297782_0008_02_01/container_tokens" export LOCAL_USER_DIRS="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/" export HOME="/home/" export CONTAINER_ID="container_1505948297782_0008_02_01" export MALLOC_ARENA_MAX="" echo "Setting up job resources" mkdir -pjobSubmitDir ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505948297782_0008/filecache/10/job.splitmetainfo" "jobSubmitDir/job.splitmetainfo" ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505948297782_0008/filecache/11/job.jar" "job.jar" mkdir -pjobSubmitDir ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505948297782_0008/filecache/12/job.split" "jobSubmitDir/job.split" ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505948297782_0008/filecache/13/job.xml" "job.xml" echo "Launching container" exec /bin/bash -c "$JAVA_HOME/bin/java -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505948297782_0008/container_1505948297782_0008_02_01 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505948297782_0008/container_1505948297782_0008_02_01/stdout 2>/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505948297782_0008/container_1505948297782_0008_02_01/stderr " {noformat} > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch, > YARN-6550.011.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16174004#comment-16174004 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 34s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 16s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 44s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 17s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 30 new + 141 unchanged - 1 fixed = 171 total (was 142) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 45s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 13s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m 14s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 33m 45s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12888150/YARN-6550.010.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 2ad96f3b01ce 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a12f09b | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17555/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17555/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17555/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/17555/console | | Powered by | Apache Yetus
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173980#comment-16173980 ] Allen Wittenauer commented on YARN-6550: FYI: This: {code} set -o pipefail -e {code} ... should be at the top. Additionally, if those sets are being done, then {code} hadoop_shell_errorcode=$? if [[ "$hadoop_shell_errorcode" -ne 0 ]] then exit $hadoop_shell_errorcode fi {code} ... should never be getting reached. Additionally: {code} exec ... {code} replaces the running app, so that closing if/then clause is pointless. > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.009.patch, YARN-6550.010.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173896#comment-16173896 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 16s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 10s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 47s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 40s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 19s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 30 new + 141 unchanged - 1 fixed = 171 total (was 142) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m 6s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 35m 52s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12888133/YARN-6550.009.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 931c8a767a52 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / a12f09b | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17550/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17550/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17550/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/17550/console | | Powered by | Apache Yet
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16173835#comment-16173835 ] Suma Shivaprasad commented on YARN-6550: launch_container script after patch {noformat} #!/bin/bash export STDOUT="/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505931818595_0002/container_1505931818595_0002_01_03/prelaunch.out" exec >"${STDOUT}" export STDERR="/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505931818595_0002/container_1505931818595_0002_01_03/prelaunch.err" exec 2>"${STDERR}" set -o pipefail -e echo "Setting up env variables" export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/etc/hadoop"} export STDOUT_LOGFILE_ENV="/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505931818595_0002/container_1505931818595_0002_01_03/stdout" export HADOOP_ROOT_LOGGER="INFO,console" export JAVA_HOME=${JAVA_HOME:-"/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home"} export NM_HOST="10.22.16.92" export LD_LIBRARY_PATH="$PWD:$HADOOP_COMMON_HOME/lib/native" export STDERR_LOGFILE_ENV="/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505931818595_0002/container_1505931818595_0002_01_03/stderr" export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT"} export LOGNAME="sshivaprasad" export JVM_PID="$$" export HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT"} export PWD="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505931818595_0002/container_1505931818595_0002_01_03" export HADOOP_CLIENT_OPTS="" export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT"} export LOCAL_DIRS="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505931818595_0002" export SHELL="/bin/bash" export NM_HTTP_PORT="8042" export LOG_DIRS="/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505931818595_0002/container_1505931818595_0002_01_03" export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gA=^M " export NM_PORT="65028" export USER="sshivaprasad" export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT"} export CLASSPATH="$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:$PWD/*" export HADOOP_TOKEN_FILE_LOCATION="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505931818595_0002/container_1505931818595_0002_01_03/container_tokens" export LOCAL_USER_DIRS="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/" export HOME="/home/" export CONTAINER_ID="container_1505931818595_0002_01_03" export MALLOC_ARENA_MAX="" echo "Setting up job resources" ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505931818595_0002/filecache/13/job.xml" "job.xml" hadoop_shell_errorcode=$? if [[ "$hadoop_shell_errorcode" -ne 0 ]] then exit $hadoop_shell_errorcode fi ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1505931818595_0002/filecache/11/job.jar" "job.jar" hadoop_shell_errorcode=$? if [[ "$hadoop_shell_errorcode" -ne 0 ]] then exit $hadoop_shell_errorcode fi echo "Launching container" exec /bin/bash -c "$JAVA_HOME/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx200m -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505931818595_0002/container_1505931818595_0002_01_03 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 10.22.16.92 65054 attempt_1505931818595_0002_r_00_0 3 1>/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505931818595_0002/container_1505931818595_0002_01_03/stdout 2>/2017-08-14/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1505931818595_0002/container_1505931818595_0002_01_03/stderr " hadoop_shell_errorcode=$? if [[ "$hadoop_shell_errorcode" -ne 0 ]] then exit $hadoop_shell_errorcode fi {noformat} > Capture launch_container.sh logs > > >
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16143259#comment-16143259 ] Junping Du commented on YARN-6550: -- Update 3.0.0-beta from fixed version to target version. Normally we don't set fix version until committer commit the patch to specific branch. > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.007.patch, > YARN-6550.008.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16140940#comment-16140940 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 42s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 50s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 22s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 34s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 49s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 46s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 21s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 51 new + 118 unchanged - 1 fixed = 169 total (was 119) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 0s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m 41s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 37m 35s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12883437/YARN-6550.008.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 7ab9c6d8af8b 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / c2cb7ea | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17127/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17127/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17127/testReport/ | | modules | C: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager U: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/17127/console | | Powered by | Apache Yet
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139291#comment-16139291 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m 29s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 19m 46s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 39s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 59s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 23s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 51 new + 117 unchanged - 1 fixed = 168 total (was 118) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 22s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 15m 18s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 20s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 46m 10s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.scheduler.TestContainerSchedulerQueuing | | | hadoop.yarn.server.nodemanager.containermanager.TestContainerManager | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12883437/YARN-6550.008.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 67a958da30ff 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 7e6463d | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17104/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17104/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/17104/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanag
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139152#comment-16139152 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 42s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 45s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 17s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 49 new + 117 unchanged - 1 fixed = 166 total (was 118) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 32s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 36m 44s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12883415/YARN-6550.007.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 7077f11fedef 3.13.0-119-generic #166-Ubuntu SMP Wed May 3 12:18:55 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 7e6463d | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/17099/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/17099/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/17099/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/17099/test
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124279#comment-16124279 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 2s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 55s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 41s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 41s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 20 unchanged - 2 fixed = 20 total (was 22) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 18s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 26 new + 117 unchanged - 1 fixed = 143 total (was 118) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 51s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 36m 39s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12881542/YARN-6550.006.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 7c4db0ec46dc 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / cc59b5f | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/16867/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/16867/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/16867/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-y
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124130#comment-16124130 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 21s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 34s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 52s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 20 unchanged - 2 fixed = 20 total (was 22) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 18s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 26 new + 117 unchanged - 1 fixed = 143 total (was 118) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 15s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 11s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 37m 24s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12881542/YARN-6550.006.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 237e4adf7235 3.13.0-123-generic #172-Ubuntu SMP Mon Jun 26 18:04:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / bbbf0e2 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/16861/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/16861/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/16861/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-y
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16124043#comment-16124043 ] Suma Shivaprasad commented on YARN-6550: Updated patch with unit test failure fix and added javadocs. > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, > YARN-6550.005.patch, YARN-6550.006.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16123712#comment-16123712 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 46s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 1 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 43s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 20 unchanged - 2 fixed = 20 total (was 22) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 17s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 26 new + 117 unchanged - 1 fixed = 143 total (was 118) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 14m 14s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 16s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 36m 55s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12881502/YARN-6550.005.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 95474421f293 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 4222c97 | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/16851/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/16851/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/16851/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/16851/testRe
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120728#comment-16120728 ] Allen Wittenauer commented on YARN-6550: BTW, be aware that in sh, {} forces a subshell. which means it's going to get interpreted first > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120727#comment-16120727 ] Allen Wittenauer commented on YARN-6550: bq. Also was executing this with "bash" explicitly so the #!/bin/bash wouldnt have affect it. I'm aware. I'm just pointing out that it's Yet Another Bug in the nodemanager's code. Also, use your toolset: {code} $ shellcheck /tmp/container_launch.sh In /tmp/container_launch.sh line 6: { ^-- SC1009: The mentioned parser error was in this brace group. In /tmp/container_launch.sh line 11: partition (cd_education_status) ^-- SC1073: Couldn't parse this function. ^-- SC1065: Trying to declare parameters? Don't. Use () and refer to params as $1, $2.. In /tmp/container_launch.sh line 12: select cd_demo_sk, cd_gender, " ^-- SC1064: Expected a { to open the function definition. ^-- SC1072: Fix any mentioned problems and try again. {code} > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120645#comment-16120645 ] Suma Shivaprasad commented on YARN-6550: Yes output is correct. You mean the curl braces between highlighted in bold below? That didnt matter. I was trying out different things and would have missed it. Also was executing this with "bash" explicitly so the #!/bin/bash wouldnt have affect it. bash /tmp/unit_test_fail.sh /tmp/unit_test_fail.sh: line 11: syntax error near unexpected token `cd_education_status' /tmp/unit_test_fail.sh: line 11: `partition (cd_education_status)' {noformat} #!/bin/bash export STDOUT="/tmp/1.out" export STDERR="/tmp/1.err" { echo "Setting up env variables" export APPLICATION_WORKFLOW_CONTEXT="*{*"workflowId":"609f91c5cd83","workflowName":" insert table partition (cd_education_status) select cd_demo_sk, cd_gender, " echo "Setting up job resources" echo "Launching container" } 1> >(tee -a "${STDOUT}" >&1) 2> >(tee -a "${STDERR}" >&2) exec /bin/bash -c "" hadoop_shell_errorcode=$? if [[ "$hadoop_shell_errorcode" -ne 0 ]] then exit $hadoop_shell_errorcode fi {noformat} > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120571#comment-16120571 ] Allen Wittenauer commented on YARN-6550: are you sure your output is correct? I'm seeing missing curly braces between the two. That error would indicate that other quotes (or other bits) are missing too. Also: {code} #!/bin/bash {code} Not portable. {code} if [ $hadoop_shell_errorcode -ne 0 ] {code} use [[ and quote the variable. > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16120554#comment-16120554 ] Suma Shivaprasad commented on YARN-6550: [~aw] Thanks for checking on the bash version portability. With the updated patch to use command groups, there is a UT failure that exposed some unexpected The UT - TestContainerLaunch.testInvalidSyntax checks for failures being propagated from a bunch of invalid commands through ShellCommandExecutor. Behaviour without the patch Below is the script that the UT executes {noformat} #!/bin/bash export APPLICATION_WORKFLOW_CONTEXT="{"workflowId":"609f91c5cd83","workflowName":" insert table partition (cd_education_status) select cd_demo_sk, cd_gender, " exec /bin/bash -c "" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi {noformat} Expected error */tmp/unit_test_pass.sh: line 5: insert: command not found* /tmp/unit_test_pass.sh: line 6: syntax error near unexpected token `cd_education_status' /tmp/unit_test_pass.sh: line 6: `partition (cd_education_status)' Behaviour with the patch Script that the UT executes with the patch (command groups) -- {noformat} { echo "Setting up env variables" export APPLICATION_WORKFLOW_CONTEXT=""workflowId":"609f91c5cd83","workflowName":" insert table partition (cd_education_status) select cd_demo_sk, cd_gender, " echo "Setting up job resources" echo "Launching container" } 1> >(tee -a "${STDOUT}" >&1) 2> >(tee -a "${STDERR}" >&2) # Note that redirection doesnt matter. Having a command group causes it. exec /bin/bash -c "" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi {noformat} Error /tmp/unit_test_fail.sh: line 6: syntax error near unexpected token `cd_education_status' /tmp/unit_test_fail.sh: line 6: `partition (cd_education_status)' Please note that the "insert table" command being an invalid command is not even thrown. Given the above issues, I was exploring other ways of achieving redirection without doing it per line. Using exec with redirection seems like a more concise way to achieve this - http://tldp.org/LDP/abs/html/x17974.html I also tested the above UT script with exec and it works fine. If you dont have any objections, will update the patch to use exec with redirection instead. {noformat} export STDOUT="/tmp/1.out" export STDERR="/tmp/1.err" exec 1> >(tee -a "${STDOUT}" >&1) 2> >(tee -a "${STDERR}" >&2) export APPLICATION_WORKFLOW_CONTEXT="{"workflowId":"609f91c5cd83","workflowName":" insert table partition (cd_education_status) select cd_demo_sk, cd_gender, " exec /bin/bash -c "" hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi {noformat} Error == */tmp/unit_test_exec.sh: line 9: insert: command not found* /tmp/unit_test_exec.sh: line 10: syntax error near unexpected token `cd_education_status' /tmp/unit_test_exec.sh: line 10: `partition (cd_education_status)' > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.003.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119069#comment-16119069 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 20s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 17m 7s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 38s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 54s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 5 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 33s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 34s{color} | {color:green} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 0 new + 20 unchanged - 2 fixed = 20 total (was 22) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 21s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 28 new + 118 unchanged - 1 fixed = 146 total (was 119) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 13m 51s{color} | {color:red} hadoop-yarn-server-nodemanager in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 15s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 39m 9s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.yarn.server.nodemanager.containermanager.launcher.TestContainerLaunch | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:14b5c93 | | JIRA Issue | YARN-6550 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12880902/YARN-6550.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux 75f7ebe71ea3 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / e0c2414 | | Default Java | 1.8.0_131 | | findbugs | v3.1.0-RC1 | | findbugs | https://builds.apache.org/job/PreCommit-YARN-Build/16792/artifact/patchprocess/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager-warnings.html | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/16792/artifact/patchprocess/diff-checkstyle-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt | | unit | https://builds.apache.org/job/PreCommit-YARN-Build/16792/artifact/patchprocess/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-y
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111943#comment-16111943 ] Allen Wittenauer commented on YARN-6550: bq. could there be any portability issues across bash versions? Short answer: No, definitely not. Long answer: Hmm... How long as grouping been supported? That's a great question. After a bit research, some answers * Hadoop 3.x requires bash 3.2. Pushing aside the OSS BSDs (which don't ship bash at all), that's old enough for Mac OS X, the only modern platform that doesn't ship bash 4.x. To find something that shipped with older than bash 3.2, you'll need to go back to circa 2003 with Solaris 9. * Shell command grouping is part of the earliest POSIX spec I can find. So "real" Bourne definitely supports it. * Bourne has supported it for a very long time according to unofficial sources. (eg. O'Reilly's UNIX Power Tools book) and likely since near the beginning of UNIX! > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16111912#comment-16111912 ] Suma Shivaprasad commented on YARN-6550: Thanks [~aw] . Will work on these suggestions and update the patch. With the shell grouping feature, could there be any portability issues across bash versions? > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16110225#comment-16110225 ] Allen Wittenauer commented on YARN-6550: If you group this, you won't have to set it per line. i.e.: {code} #!/usr/bin/env bash STDERR=/tmp/err.log STDOUT=/tmp/out.log { cmd cmd cmd } 2>"${stderr}" | tee -a "${stderr}" > "${stdout}" {code} (or whatever) In shell, it's generally not useful to split stderr away from stdout when debugging. The above should split stdout into a file and stdout+stderr into another. (I'm doing this on the fly so that might not be 100% correct syntax haha) It's probably also worth pointing out that several paths will have unexpected results if they have spaces due to lack of quotes. Probably be a good idea to run the computed shell script through shellcheck to find errors like that. (Although it won't catch e.g., -Dyarn.app.container.log.dir=, since that's hard-set.) > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Affects Versions: 3.0.0-beta1 >Reporter: Wangda Tan >Assignee: Suma Shivaprasad > Fix For: 3.0.0-beta1 > > Attachments: YARN-6550.002.patch, YARN-6550.patch > > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109953#comment-16109953 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 23s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 16m 55s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 36s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 59s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 5 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 35s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 37s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 37s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 2 new + 20 unchanged - 2 fixed = 22 total (was 22) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 20s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 31 new + 117 unchanged - 1 fixed = 148 total (was 118) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 31s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 4s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager generated 4 new + 5 unchanged - 0 fixed = 9 total (was 5) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 24s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 4 new + 105 unchanged - 0 fixed = 109 total (was 105) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 14m 0s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 18s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 39m 29s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | | Unread field:field be static? At ContainerLaunch.java:[line 863] | | | Unread field:field be static? At ContainerLaunch.java:[line 862] | | | Format-string method String.format(String, Object[]) called with format string "@%s symlink "%s" "%s"" wants 3 arguments but is given 4 in org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch$WindowsShellScriptBuilder.link(Path, Path) At ContainerLaunch.java:with format string "@%s symlink "%s" "%s"" wants 3 arguments but is given 4 in org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch$WindowsShellScriptBuilder.link(Path, Path) At ContainerLaunch.java:[line 1144] | | | Format-string method String.format(String, Object[]) called with format string "@if not exist "%s" mkdir "%s"" wants 2 arguments but is given 3 in org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch$Windows
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109878#comment-16109878 ] Hadoop QA commented on YARN-6550: - | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 27s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 38s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 30s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 45s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager in trunk has 5 extant Findbugs warnings. {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 18s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 28s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 2 new + 20 unchanged - 2 fixed = 22 total (was 22) {color} | | {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange} 0m 18s{color} | {color:orange} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager: The patch generated 31 new + 117 unchanged - 1 fixed = 148 total (was 118) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m 56s{color} | {color:red} hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager generated 4 new + 5 unchanged - 0 fixed = 9 total (was 5) {color} | | {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m 16s{color} | {color:red} hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager generated 4 new + 105 unchanged - 0 fixed = 109 total (was 105) {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 13m 12s{color} | {color:green} hadoop-yarn-server-nodemanager in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 14s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 35m 0s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager | | | Unread field:field be static? At ContainerLaunch.java:[line 863] | | | Unread field:field be static? At ContainerLaunch.java:[line 862] | | | Format-string method String.format(String, Object[]) called with format string "@%s symlink "%s" "%s"" wants 3 arguments but is given 4 in org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch$WindowsShellScriptBuilder.link(Path, Path) At ContainerLaunch.java:with format string "@%s symlink "%s" "%s"" wants 3 arguments but is given 4 in org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch$WindowsShellScriptBuilder.link(Path, Path) At ContainerLaunch.java:[line 1138] | | | Format-string method String.format(String, Object[]) called with format string "@if not exist "%s" mkdir "%s"" wants 2 arguments but is given 3 in org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch$Windows
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16109638#comment-16109638 ] Suma Shivaprasad commented on YARN-6550: Example launch_container.sh with the patch {noformat} #!/bin/bash export STDOUT="/2017-07-25/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1501616662779_0002/container_1501616662779_0002_01_01/prelaunch.out" export STDERR="/2017-07-25/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1501616662779_0002/container_1501616662779_0002_01_01/prelaunch.err" echo "Setting up env variables" 1> >(tee -a $STDOUT >&1) export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-"/2017-07-25/hadoop-3.0.0-beta1-SNAPSHOT/etc/hadoop"} 2> >(tee -a $STDERR >&2) export YARN_CONTAINER_RUNTIME_TYPE="docker" 2> >(tee -a $STDERR >&2) export JAVA_HOME=${JAVA_HOME:-"/Library/Java/JavaVirtualMachines/jdk1.8.0_131.jdk/Contents/Home"} 2> >(tee -a $STDERR >&2) export YARN_CONTAINER_RUNTIME_DOCKER_IMAGE="sequenceiq/hadoop-docker" 2> >(tee -a $STDERR >&2) export APP_SUBMIT_TIME_ENV="1501617165715" 2> >(tee -a $STDERR >&2) export NM_HOST="10.22.16.92" 2> >(tee -a $STDERR >&2) export LD_LIBRARY_PATH="$PWD:$HADOOP_COMMON_HOME/lib/native" 2> >(tee -a $STDERR >&2) export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-"/2017-07-25/hadoop-3.0.0-beta1-SNAPSHOT"} 2> >(tee -a $STDERR >&2) export LOGNAME="sshivaprasad" 2> >(tee -a $STDERR >&2) export JVM_PID="$$" 2> >(tee -a $STDERR >&2) export HADOOP_MAPRED_HOME=${HADOOP_MAPRED_HOME:-"/2017-07-25/hadoop-3.0.0-beta1-SNAPSHOT"} 2> >(tee -a $STDERR >&2) export PWD="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1501616662779_0002/container_1501616662779_0002_01_01" 2> >(tee -a $STDERR >&2) export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-"/2017-07-25/hadoop-3.0.0-beta1-SNAPSHOT"} 2> >(tee -a $STDERR >&2) export LOCAL_DIRS="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1501616662779_0002" 2> >(tee -a $STDERR >&2) export APPLICATION_WEB_PROXY_BASE="/proxy/application_1501616662779_0002" 2> >(tee -a $STDERR >&2) export SHELL="/bin/bash" 2> >(tee -a $STDERR >&2) export NM_HTTP_PORT="8042" 2> >(tee -a $STDERR >&2) export LOG_DIRS="/2017-07-25/hadoop-3.0.0-beta1-SNAPSHOT/logs/userlogs/application_1501616662779_0002/container_1501616662779_0002_01_01" 2> >(tee -a $STDERR >&2) export NM_AUX_SERVICE_mapreduce_shuffle="AAA0+gA= " 2> >(tee -a $STDERR >&2) export NM_PORT="9" 2> >(tee -a $STDERR >&2) export USER="sshivaprasad" 2> >(tee -a $STDERR >&2) export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-"/2017-07-25/hadoop-3.0.0-beta1-SNAPSHOT"} 2> >(tee -a $STDERR >&2) export CLASSPATH="$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/*:$HADOOP_MAPRED_HOME/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:$PWD/*" 2> >(tee -a $STDERR >&2) export HADOOP_TOKEN_FILE_LOCATION="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1501616662779_0002/container_1501616662779_0002_01_01/container_tokens" 2> >(tee -a $STDERR >&2) export LOCAL_USER_DIRS="/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/" 2> >(tee -a $STDERR >&2) export HOME="/home/" 2> >(tee -a $STDERR >&2) export CONTAINER_ID="container_1501616662779_0002_01_01" 2> >(tee -a $STDERR >&2) export MALLOC_ARENA_MAX="" 2> >(tee -a $STDERR >&2) echo "Setting up job resources" 1> >(tee -a $STDOUT >&1) ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1501616662779_0002/filecache/11/job.jar" "job.jar" 2> >(tee -a $STDERR >&2) hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi mkdir -p jobSubmitDir 2> >(tee -a $STDERR >&2) hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1501616662779_0002/filecache/12/job.split" "jobSubmitDir/job.split" 2> >(tee -a $STDERR >&2) hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/tmp/hadoop-sshivaprasad/nm-local-dir/usercache/sshivaprasad/appcache/application_1501616662779_0002/filecache/13/job.xml" "job.xml" 2> >(tee -a $STDERR >&2) hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi mkdir -p jobSubmitDir 2> >(tee -a $STDERR >&2) hadoop_shell_errorcode=$? if [ $hadoop_shell_errorcode -ne 0 ] then exit $hadoop_shell_errorcode fi ln -sf "/tmp/hadoop-sshivaprasad/nm-local-
[jira] [Commented] (YARN-6550) Capture launch_container.sh logs
[ https://issues.apache.org/jira/browse/YARN-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15994033#comment-15994033 ] Wangda Tan commented on YARN-6550: -- +[~vinodkv]/[~sidharta-s]/[~vvasudev]. > Capture launch_container.sh logs > > > Key: YARN-6550 > URL: https://issues.apache.org/jira/browse/YARN-6550 > Project: Hadoop YARN > Issue Type: Sub-task >Reporter: Wangda Tan > > launch_container.sh which generated by NM will do a bunch of things (like > create link, etc.) while launch a process. No logs captured until {{exec}} is > called. We need capture all failures of launch_container.sh for easier > troubleshooting. -- This message was sent by Atlassian JIRA (v6.3.15#6346) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org