[jira] [Commented] (MAPREDUCE-7036) ASF License warning in hadoop-mapreduce-client
[ https://issues.apache.org/jira/browse/MAPREDUCE-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340606#comment-16340606 ] Takanobu Asanuma commented on MAPREDUCE-7036: - The failed test is not related to the 1st patch. > ASF License warning in hadoop-mapreduce-client > -- > > Key: MAPREDUCE-7036 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7036 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Minor > Attachments: MAPREDUCE-7036.1.patch > > > it occurred in MAPREDUCE-7021 and MAPREDUCE-7034. > {noformat} > Lines that start with ? in the ASF License report indicate files that do > not have an Apache license header: !? > /testptch/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/jobTokenPassword > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7036) ASF License warning in hadoop-mapreduce-client
[ https://issues.apache.org/jira/browse/MAPREDUCE-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340598#comment-16340598 ] Hadoop QA commented on MAPREDUCE-7036: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 8s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 39s{color} | {color:blue} Maven dependency ordering for branch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 17s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 57s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 11s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 37s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m 8s{color} | {color:blue} Maven dependency ordering for patch {color} | | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 55s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 38s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 25s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 28s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 34s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 36s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 26s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed. {color} | | {color:red}-1{color} | {color:red} unit {color} | {color:red}132m 55s{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 27s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}183m 49s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.mapreduce.v2.TestUberAM | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | MAPREDUCE-7036 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907807/MAPREDUCE-7036.1.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux cae7ffd1191f 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 2e58656 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | unit |
[jira] [Commented] (MAPREDUCE-7036) ASF License warning in hadoop-mapreduce-client
[ https://issues.apache.org/jira/browse/MAPREDUCE-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340487#comment-16340487 ] Takanobu Asanuma commented on MAPREDUCE-7036: - I reran Jenkins since MAPREDUCE-7041 is resolved now. > ASF License warning in hadoop-mapreduce-client > -- > > Key: MAPREDUCE-7036 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7036 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Minor > Attachments: MAPREDUCE-7036.1.patch > > > it occurred in MAPREDUCE-7021 and MAPREDUCE-7034. > {noformat} > Lines that start with ? in the ASF License report indicate files that do > not have an Apache license header: !? > /testptch/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/jobTokenPassword > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7036) ASF License warning in hadoop-mapreduce-client
[ https://issues.apache.org/jira/browse/MAPREDUCE-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma updated MAPREDUCE-7036: Attachment: MAPREDUCE-7036.1.patch > ASF License warning in hadoop-mapreduce-client > -- > > Key: MAPREDUCE-7036 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7036 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Minor > Attachments: MAPREDUCE-7036.1.patch > > > it occurred in MAPREDUCE-7021 and MAPREDUCE-7034. > {noformat} > Lines that start with ? in the ASF License report indicate files that do > not have an Apache license header: !? > /testptch/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/jobTokenPassword > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7036) ASF License warning in hadoop-mapreduce-client
[ https://issues.apache.org/jira/browse/MAPREDUCE-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takanobu Asanuma updated MAPREDUCE-7036: Attachment: (was: MAPREDUCE-7036.1.patch) > ASF License warning in hadoop-mapreduce-client > -- > > Key: MAPREDUCE-7036 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7036 > Project: Hadoop Map/Reduce > Issue Type: Test > Components: test >Reporter: Takanobu Asanuma >Assignee: Takanobu Asanuma >Priority: Minor > Attachments: MAPREDUCE-7036.1.patch > > > it occurred in MAPREDUCE-7021 and MAPREDUCE-7034. > {noformat} > Lines that start with ? in the ASF License report indicate files that do > not have an Apache license header: !? > /testptch/hadoop/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/jobTokenPassword > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7041) MR should not try to clean up at first job attempt
[ https://issues.apache.org/jira/browse/MAPREDUCE-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340415#comment-16340415 ] Hudson commented on MAPREDUCE-7041: --- SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #13561 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/13561/]) MAPREDUCE-7041. MR should not try to clean up at first job attempt. (haibochen: rev cc10852252c2d69294eabc68bd032cc630a53b18) * (edit) hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java > MR should not try to clean up at first job attempt > -- > > Key: MAPREDUCE-7041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7041 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Gergo Repas >Priority: Major > Fix For: 3.1.0, 3.0.1, 2.10.0 > > Attachments: MAPREDUCE-7041.000.patch > > > These tests fails in trunk now. MAPREDUCE-6984 may be related. > {noformat} > hadoop.mapreduce.v2.TestMROldApiJobs.testJobSucceed > hadoop.mapred.TestJobCleanup.testCustomAbort > hadoop.mapreduce.lib.output.TestJobOutputCommitter.testCustomAbort > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7041) MR should not try to clean up at first job attempt
[ https://issues.apache.org/jira/browse/MAPREDUCE-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-7041: -- Resolution: Fixed Hadoop Flags: Reviewed Fix Version/s: 2.10.0 3.0.1 3.1.0 Status: Resolved (was: Patch Available) Thanks [~tasanuma0829] for reporting the issue, and [~grepas] for the quick fix. I committed the patch in branch-2, branch-3.0 and trunk > MR should not try to clean up at first job attempt > -- > > Key: MAPREDUCE-7041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7041 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Gergo Repas >Priority: Major > Fix For: 3.1.0, 3.0.1, 2.10.0 > > Attachments: MAPREDUCE-7041.000.patch > > > These tests fails in trunk now. MAPREDUCE-6984 may be related. > {noformat} > hadoop.mapreduce.v2.TestMROldApiJobs.testJobSucceed > hadoop.mapred.TestJobCleanup.testCustomAbort > hadoop.mapreduce.lib.output.TestJobOutputCommitter.testCustomAbort > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7041) MR should not try to clean up at first job attempt
[ https://issues.apache.org/jira/browse/MAPREDUCE-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-7041: -- Summary: MR should not try to clean up at first job attempt (was: Several unit tests in hadoop-mapreduce-client-jobclient are failed) > MR should not try to clean up at first job attempt > -- > > Key: MAPREDUCE-7041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7041 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Gergo Repas >Priority: Major > Attachments: MAPREDUCE-7041.000.patch > > > These tests fails in trunk now. MAPREDUCE-6984 may be related. > {noformat} > hadoop.mapreduce.v2.TestMROldApiJobs.testJobSucceed > hadoop.mapred.TestJobCleanup.testCustomAbort > hadoop.mapreduce.lib.output.TestJobOutputCommitter.testCustomAbort > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7041) Several unit tests in hadoop-mapreduce-client-jobclient are failed
[ https://issues.apache.org/jira/browse/MAPREDUCE-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340329#comment-16340329 ] Haibo Chen commented on MAPREDUCE-7041: --- +1. Check this in shortly. > Several unit tests in hadoop-mapreduce-client-jobclient are failed > -- > > Key: MAPREDUCE-7041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7041 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Gergo Repas >Priority: Major > Attachments: MAPREDUCE-7041.000.patch > > > These tests fails in trunk now. MAPREDUCE-6984 may be related. > {noformat} > hadoop.mapreduce.v2.TestMROldApiJobs.testJobSucceed > hadoop.mapred.TestJobCleanup.testCustomAbort > hadoop.mapreduce.lib.output.TestJobOutputCommitter.testCustomAbort > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7033) Map outputs implicitly rely on permissive umask for shuffle
[ https://issues.apache.org/jira/browse/MAPREDUCE-7033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340328#comment-16340328 ] Hadoop QA commented on MAPREDUCE-7033: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 9m 53s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 33s{color} | {color:green} trunk passed {color} | | {color:red}-1{color} | {color:red} shadedclient {color} | {color:red} 2m 45s{color} | {color:red} branch has errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 30s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 16s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 27s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 10s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 3s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 6s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 17s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 47m 10s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | MAPREDUCE-7033 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907784/MAPREDUCE-7033.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux b2e43405c79f 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 16be42d | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7320/testReport/ | | Max. process+thread count | 407 (vs. ulimit of 5000) | | modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core | | Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7320/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT
[jira] [Commented] (MAPREDUCE-7027) HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340278#comment-16340278 ] Hadoop QA commented on MAPREDUCE-7027: -- | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 20s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m 0s{color} | {color:green} The patch appears to include 1 new or modified test files. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 21s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 14s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 24s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 6s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 26s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 16s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 21s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 10s{color} | {color:green} hadoop-tools/hadoop-archive-logs: The patch generated 0 new + 3 unchanged - 3 fixed = 3 total (was 6) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 34s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 32s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m 41s{color} | {color:green} hadoop-archive-logs in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 51m 23s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | MAPREDUCE-7027 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12902861/MAPREDUCE-7027.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 4eb1abef6329 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 16be42d | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7319/testReport/ | | Max. process+thread count | 507 (vs. ulimit of 5000) | | modules | C: hadoop-tools/hadoop-archive-logs U: hadoop-tools/hadoop-archive-logs | | Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7319/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > HadoopArchiveLogs shouldn't delete the original logs if
[jira] [Updated] (MAPREDUCE-7033) Map outputs implicitly rely on permissive umask for shuffle
[ https://issues.apache.org/jira/browse/MAPREDUCE-7033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-7033: -- Status: Patch Available (was: Open) Attaching a patch that updates the permissions of the output files, if necessary, to give the shuffle handler sufficient access. Still needs a unit test. > Map outputs implicitly rely on permissive umask for shuffle > --- > > Key: MAPREDUCE-7033 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7033 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: MAPREDUCE-7033.001.patch > > > Map tasks do not explicitly set the permissions of their output files for > shuffle. In a secure cluster the shuffle service is running as a different > user than the map task, so the output files require group readability in > order to serve up the data during the shuffle phase. If the user's UNIX > umask is too restrictive (e.g.: 077) then the map task's file.out and > file.out.index permissions can be too restrictive to allow the shuffle > handler to access them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7033) Map outputs implicitly rely on permissive umask for shuffle
[ https://issues.apache.org/jira/browse/MAPREDUCE-7033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe updated MAPREDUCE-7033: -- Attachment: MAPREDUCE-7033.001.patch > Map outputs implicitly rely on permissive umask for shuffle > --- > > Key: MAPREDUCE-7033 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7033 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > Attachments: MAPREDUCE-7033.001.patch > > > Map tasks do not explicitly set the permissions of their output files for > shuffle. In a secure cluster the shuffle service is running as a different > user than the map task, so the output files require group readability in > order to serve up the data during the shuffle phase. If the user's UNIX > umask is too restrictive (e.g.: 077) then the map task's file.out and > file.out.index permissions can be too restrictive to allow the shuffle > handler to access them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7027) HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-7027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Kanter updated MAPREDUCE-7027: - Status: Patch Available (was: Open) > HadoopArchiveLogs shouldn't delete the original logs if the HAR creation fails > -- > > Key: MAPREDUCE-7027 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7027 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Reporter: Gergely Novák >Assignee: Gergely Novák >Priority: Critical > Attachments: MAPREDUCE-7027.001.patch > > > If the hadoop archive command fails for any reason (for example because of an > OutOfMemoryError) the HadoopArchiveLogs tool will still delete the original > log files, so all the logs will be lost. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Assigned] (MAPREDUCE-7033) Map outputs implicitly rely on permissive umask for shuffle
[ https://issues.apache.org/jira/browse/MAPREDUCE-7033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Lowe reassigned MAPREDUCE-7033: - Assignee: Jason Lowe Priority: Critical (was: Major) Target Version/s: 3.1.0, 3.0.1 Raising the priority of this since I recently discovered this long-lingering bug can actually prevent proper shuffling on secure clusters in Hadoop 3.x. HADOOP-11347 fixed a bug in the local filesystem so it started honoring the configured fs.permissions.umask-mode for local files. That fix now exposes the lack of explicit permissions in the MapReduce framework for any case where the configured umask is at least 0044. > Map outputs implicitly rely on permissive umask for shuffle > --- > > Key: MAPREDUCE-7033 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7033 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Reporter: Jason Lowe >Assignee: Jason Lowe >Priority: Critical > > Map tasks do not explicitly set the permissions of their output files for > shuffle. In a secure cluster the shuffle service is running as a different > user than the map task, so the output files require group readability in > order to serve up the data during the shuffle phase. If the user's UNIX > umask is too restrictive (e.g.: 077) then the map task's file.out and > file.out.index permissions can be too restrictive to allow the shuffle > handler to access them. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7042) Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled
[ https://issues.apache.org/jira/browse/MAPREDUCE-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yesha Vora updated MAPREDUCE-7042: -- Summary: Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir when ATS v2 is enabled (was: Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir) > Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir when > ATS v2 is enabled > --- > > Key: MAPREDUCE-7042 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7042 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Yesha Vora >Priority: Major > > Steps: > 1) Start a mapreduce job > {code} > hadoop jar > /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient-3.0.0.3.0.0.0-751-tests.jar > sleep "-Dmapreduce.job.user.name=hrt_qa" -m 10 -r 1 -mt 1000 -rt > 1000{code} > 2) kill job > 3) Validate job is present at mapreduce.jobhistory.done-dir > mapreduce.jobhistory.done-dir dir does not have job_1516776025831_0048 entry. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7042) Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-7042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340174#comment-16340174 ] Xuan Gong commented on MAPREDUCE-7042: -- The symptom is that we can not find this application in JHS. A little bit more context: 1) we have ATSv2 enabled. 2) we get ATSv2 socketTimeout Exception while putting the timeline entity {code} [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Exception while publishing configs on JOB_SUBMITTED Event for the job : job_1516776025831_0063 org.apache.hadoop.yarn.exceptions.YarnException: Failed while publishing entity Caused by: java.net.SocketTimeoutException: Read timed out {code} 3) It happens that we run an MR application and kill it immediately. But if we let the application run for a while(for a couple of minutes) and kill it, JHS will have the application. > Killed mapreduce job data does not move to mapreduce.jobhistory.done-dir > > > Key: MAPREDUCE-7042 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7042 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 3.0.0 >Reporter: Yesha Vora >Priority: Major > > Steps: > 1) Start a mapreduce job > {code} > hadoop jar > /usr/hdp/current/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient-3.0.0.3.0.0.0-751-tests.jar > sleep "-Dmapreduce.job.user.name=hrt_qa" -m 10 -r 1 -mt 1000 -rt > 1000{code} > 2) kill job > 3) Validate job is present at mapreduce.jobhistory.done-dir > mapreduce.jobhistory.done-dir dir does not have job_1516776025831_0048 entry. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Resolved] (MAPREDUCE-7043) Make EventHubs system configs compatible with Samza standalone
[ https://issues.apache.org/jira/browse/MAPREDUCE-7043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Boris Shkolnik resolved MAPREDUCE-7043. --- Resolution: Invalid > Make EventHubs system configs compatible with Samza standalone > -- > > Key: MAPREDUCE-7043 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7043 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Boris Shkolnik >Assignee: Boris Shkolnik >Priority: Major > > EventHubs configs are using the old style configs (system.*.streams.*.config). > It needs to be moved to the new stream-id configs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Created] (MAPREDUCE-7043) Make EventHubs system configs compatible with Samza standalone
Boris Shkolnik created MAPREDUCE-7043: - Summary: Make EventHubs system configs compatible with Samza standalone Key: MAPREDUCE-7043 URL: https://issues.apache.org/jira/browse/MAPREDUCE-7043 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Boris Shkolnik Assignee: Boris Shkolnik EventHubs configs are using the old style configs (system.*.streams.*.config). It needs to be moved to the new stream-id configs. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7041) Several unit tests in hadoop-mapreduce-client-jobclient are failed
[ https://issues.apache.org/jira/browse/MAPREDUCE-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339545#comment-16339545 ] Hadoop QA commented on MAPREDUCE-7041: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 10m 17s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 23s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 28s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 19s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 6s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 37s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 20s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 28s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 23s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 15s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 26s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 30s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 17s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:green}+1{color} | {color:green} unit {color} | {color:green} 9m 20s{color} | {color:green} hadoop-mapreduce-client-app in the patch passed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 21s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black} 60m 52s{color} | {color:black} {color} | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:5b98639 | | JIRA Issue | MAPREDUCE-7041 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12907723/MAPREDUCE-7041.000.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 38b28f532f84 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 82cc6f6 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | Test Results | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7318/testReport/ | | Max. process+thread count | 593 (vs. ulimit of 5000) | | modules | C: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app U: hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app | | Console output | https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7318/console | | Powered by | Apache Yetus
[jira] [Updated] (MAPREDUCE-7041) Several unit tests in hadoop-mapreduce-client-jobclient are failed
[ https://issues.apache.org/jira/browse/MAPREDUCE-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergo Repas updated MAPREDUCE-7041: --- Status: Patch Available (was: Open) I have run all /hadoop-mapreduce-project/hadoop-mapreduce-client/... unit tests with the attached fix. All but one passed, that one is an unrelated error (TestMRJobs.testThreadDumpOnTaskTimeout:1220 No thread dump), it failed even if the MAPREDUCE-6984 changes are reverted. > Several unit tests in hadoop-mapreduce-client-jobclient are failed > -- > > Key: MAPREDUCE-7041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7041 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Gergo Repas >Priority: Major > Attachments: MAPREDUCE-7041.000.patch > > > These tests fails in trunk now. MAPREDUCE-6984 may be related. > {noformat} > hadoop.mapreduce.v2.TestMROldApiJobs.testJobSucceed > hadoop.mapred.TestJobCleanup.testCustomAbort > hadoop.mapreduce.lib.output.TestJobOutputCommitter.testCustomAbort > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Updated] (MAPREDUCE-7041) Several unit tests in hadoop-mapreduce-client-jobclient are failed
[ https://issues.apache.org/jira/browse/MAPREDUCE-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gergo Repas updated MAPREDUCE-7041: --- Attachment: MAPREDUCE-7041.000.patch > Several unit tests in hadoop-mapreduce-client-jobclient are failed > -- > > Key: MAPREDUCE-7041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7041 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Gergo Repas >Priority: Major > Attachments: MAPREDUCE-7041.000.patch > > > These tests fails in trunk now. MAPREDUCE-6984 may be related. > {noformat} > hadoop.mapreduce.v2.TestMROldApiJobs.testJobSucceed > hadoop.mapred.TestJobCleanup.testCustomAbort > hadoop.mapreduce.lib.output.TestJobOutputCommitter.testCustomAbort > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7041) Several unit tests in hadoop-mapreduce-client-jobclient are failed
[ https://issues.apache.org/jira/browse/MAPREDUCE-7041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16339123#comment-16339123 ] Gergo Repas commented on MAPREDUCE-7041: [~tasanuma0829] Yes, sure, thanks for assigning it to me. > Several unit tests in hadoop-mapreduce-client-jobclient are failed > -- > > Key: MAPREDUCE-7041 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-7041 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Takanobu Asanuma >Assignee: Gergo Repas >Priority: Major > > These tests fails in trunk now. MAPREDUCE-6984 may be related. > {noformat} > hadoop.mapreduce.v2.TestMROldApiJobs.testJobSucceed > hadoop.mapred.TestJobCleanup.testCustomAbort > hadoop.mapreduce.lib.output.TestJobOutputCommitter.testCustomAbort > {noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org