[
https://issues.apache.org/jira/browse/MAPREDUCE-6551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15011091#comment-15011091
]
Hadoop QA commented on MAPREDUCE-6551:
--------------------------------------
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} docker + precommit patch detected. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
21s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 43s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 51s
{color} | {color:green} trunk passed with JDK v1.7.0_85 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 19s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
37s {color} | {color:green} trunk passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 30s
{color} | {color:red}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
in trunk has 2 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 54s
{color} | {color:green} trunk passed with JDK v1.7.0_85 {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m 23s
{color} | {color:red} hadoop-mapreduce-client-app in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 53s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 8m 18s {color}
| {color:red} hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66 with
JDK v1.8.0_66 generated 2 new issues (was 360, now 360). {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 53s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 3m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_85 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 12m 10s
{color} | {color:red}
hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85 with JDK v1.7.0_85
generated 3 new issues (was 365, now 365). {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 3m 51s
{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m 22s
{color} | {color:red} Patch generated 6 new checkstyle issues in
hadoop-mapreduce-project/hadoop-mapreduce-client (total was 770, now 775).
{color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 9s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
32s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
36s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 48s
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 51s
{color} | {color:green} the patch passed with JDK v1.7.0_85 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 9m 57s {color}
| {color:red} hadoop-mapreduce-client-app in the patch failed with JDK
v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 8s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.8.0_66. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 10m 16s {color}
| {color:red} hadoop-mapreduce-client-app in the patch failed with JDK
v1.7.0_85. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 22s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.7.0_85. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 24s
{color} | {color:red} Patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 65m 25s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_66 Failed junit tests | hadoop.mapreduce.v2.app.TestMRAppMaster |
| | hadoop.mapreduce.v2.app.rm.TestRMContainerAllocator |
| | hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt |
| JDK v1.7.0_85 Failed junit tests | hadoop.mapreduce.v2.app.TestMRAppMaster |
| | hadoop.mapreduce.v2.app.rm.TestRMContainerAllocator |
| | hadoop.mapreduce.v2.app.job.impl.TestTaskAttempt |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:date2015-11-18 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12772996/MAPREDUCE-6551.001.patch
|
| JIRA Issue | MAPREDUCE-6551 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs checkstyle |
| uname | Linux da698100aa33 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/patchprocess/apache-yetus-3f4279a/precommit/personality/hadoop.sh
|
| git revision | trunk / 4a3b8f1 |
| findbugs | v3.0.0 |
| findbugs |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html
|
| mvninstall |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/patch-mvninstall-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app.txt
|
| javac | hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.8.0_66.txt
|
| javac | hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/diff-compile-javac-hadoop-mapreduce-project_hadoop-mapreduce-client-jdk1.7.0_85.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/diff-checkstyle-hadoop-mapreduce-project_hadoop-mapreduce-client.txt
|
| unit |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
|
| unit |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt
|
| unit test logs |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.8.0_66.txt
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-app-jdk1.7.0_85.txt
|
| JDK v1.7.0_85 Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/testReport/ |
| asflicense |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/artifact/patchprocess/patch-asflicense-problems.txt
|
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U: hadoop-mapreduce-project/hadoop-mapreduce-client |
| Max memory used | 77MB |
| Powered by | Apache Yetus http://yetus.apache.org |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6172/console |
This message was automatically generated.
> Dynamic adjust mapTaskAttempt memory size
> -----------------------------------------
>
> Key: MAPREDUCE-6551
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6551
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: task
> Affects Versions: 2.7.1
> Reporter: Lin Yiqun
> Assignee: Lin Yiqun
> Attachments: MAPREDUCE-6551.001.patch
>
>
> I found a scenario that the map tasks cost so much resource of cluster.This
> scenario will be happened that if there are many small file blokcs (even some
> are not reach 1M),and this will lead to many map task to read.And in
> gengeral,a map task attempt will use the default config
> {{MRJobConfig#MAP_MEMORY_MB}} to set its resourceCapcity's memory to deal
> with their datas.And this will cause a problem that map tasks cost so much
> memory resource and target data is small.So I have a idea that wherther we
> can dynamic set mapTaskAttempt memory size by its inputDataLength.And this
> value can be provided by {{TaskSplitMetaInfo#getInputDataLength}}
> methods.Besides that,we should provided a standard unit dataLength for a
> standard memory size.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
