[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455990#comment-16455990
]
Hadoop QA commented on MAPREDUCE-7072:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
18s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
31s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
34s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
4s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
21s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 54s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:f667ef1 |
| JIRA Issue | MAPREDUCE-7072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920941/MAPREDUCE-7072-branch-2.03.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux bdb6feac05e5 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2 / a91d5c7 |
| maven | version: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_171 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7404/testReport/ |
| Max. process+thread count | 159 (vs. ulimit of 1) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7404/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16455808#comment-16455808
]
Wilfred Spiegelenburg commented on MAPREDUCE-7072:
--
No idea what I did yesterday but the diff was not correct and did not
correspond to what I had locally.
Fixed it up: compile pass, tests pass
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072-branch-2.02.patch,
> MAPREDUCE-7072-branch-2.03.patch, MAPREDUCE-7072.01.patch,
> MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454622#comment-16454622
]
Hadoop QA commented on MAPREDUCE-7072:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 21m
29s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
1s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} branch-2 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 13m
53s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
36s{color} | {color:green} branch-2 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
29s{color} | {color:green} branch-2 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 3m 7s{color}
| {color:red} hadoop-mapreduce-client-core in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
23s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 44m 5s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.mapreduce.jobhistory.TestHistoryViewerPrinter |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:f667ef1 |
| JIRA Issue | MAPREDUCE-7072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920719/MAPREDUCE-7072-branch-2.02.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 843559b5a3bf 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-2 / a5fc638 |
| maven | version: Apache Maven 3.3.9
(bb52d8502b132ec0a5a3f4c09453c07478323dc5; 2015-11-10T16:41:47+00:00) |
| Default Java | 1.7.0_171 |
| unit |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7403/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7403/testReport/ |
| Max. process+thread count | 159 (vs. ulimit of 1) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7403/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
>
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16454557#comment-16454557
]
Robert Kanter commented on MAPREDUCE-7072:
--
There was some kind of problem with the Jenkins node
([https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7402/).] I've
retriggered it: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7403/
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072-branch-2.02.patch,
> MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16453412#comment-16453412
]
Wilfred Spiegelenburg commented on MAPREDUCE-7072:
--
Branch 2 patch attached, compiles with java 7 after removing two fields in the
job info creation
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072-branch-2.02.patch,
> MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451329#comment-16451329
]
Hadoop QA commented on MAPREDUCE-7072:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} MAPREDUCE-7072 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | MAPREDUCE-7072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920459/MAPREDUCE-7072.02.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7398/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451328#comment-16451328
]
Robert Kanter commented on MAPREDUCE-7072:
--
Retriggered Jenkins at
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7398/
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16451267#comment-16451267
]
Hadoop QA commented on MAPREDUCE-7072:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 6s{color}
| {color:red} MAPREDUCE-7072 does not apply to trunk. Rebase required? Wrong
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | MAPREDUCE-7072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920459/MAPREDUCE-7072.02.patch
|
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7397/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450386#comment-16450386
]
Hudson commented on MAPREDUCE-7072:
---
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #14056 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/14056/])
MAPREDUCE-7072. mapred job -history prints duplicate counter in human (rkanter:
rev 1b9ecc264a6abe9d9d5412318c67d3d2936bd9ac)
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestHistoryViewerPrinter.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/JSONHistoryViewerPrinter.java
* (edit)
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/jobhistory/HumanReadableHistoryViewerPrinter.java
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450360#comment-16450360
]
Robert Kanter commented on MAPREDUCE-7072:
--
I committed this to trunk, but it fails to compile on branch-2. [~wilfreds],
can you create a branch-2 version of the patch?
{noformat}
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile
(default-testCompile) on project hadoop-mapreduce-client-core: Compilation
failure: Compilation failure:
[ERROR]
/Users/rkanter/dev/hadoop-git/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestHistoryViewerPrinter.java:[958,8]
cannot find symbol
[ERROR] symbol: variable succeededMaps
[ERROR] location: variable job of type
org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.JobInfo
[ERROR]
/Users/rkanter/dev/hadoop-git/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/jobhistory/TestHistoryViewerPrinter.java:[959,8]
cannot find symbol
[ERROR] symbol: variable succeededReduces
[ERROR] location: variable job of type
org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.JobInfo
[ERROR] -> [Help 1]
{noformat}
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16450347#comment-16450347
]
Robert Kanter commented on MAPREDUCE-7072:
--
+1 LGTM
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16449915#comment-16449915
]
Wilfred Spiegelenburg commented on MAPREDUCE-7072:
--
updated the patch with the review comments, I completely overlooked this far
more elegant solution.
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072.01.patch, MAPREDUCE-7072.02.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16448958#comment-16448958
]
Robert Kanter commented on MAPREDUCE-7072:
--
Great investigation [~wilfreds]! The patch looks good overall, just one thing:
# Instead of using the iterator:
{code:java}
Iterator groupNames = totalCounters.iterator();
while (groupNames.hasNext()) {
String groupName = groupNames.next().getName();
{code}
I think we can still do a for-each loop like this:
{code:java}
for (CounterGroup counterGroup : totalCounters) {
String groupName = counterGroup.getName();
{code}
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
> Attachments: MAPREDUCE-7072.01.patch
>
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16447715#comment-16447715
]
Hadoop QA commented on MAPREDUCE-7072:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 22m
58s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
24s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
32s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
10m 0s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
50s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
9m 56s{color} | {color:green} patch has no errors when building and testing our
client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
42s{color} | {color:green} hadoop-mapreduce-client-core in the patch passed.
{color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
20s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 52m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:8620d2b |
| JIRA Issue | MAPREDUCE-7072 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12920239/MAPREDUCE-7072.01.patch
|
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit shadedclient findbugs checkstyle |
| uname | Linux 6e80395b4c1c 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12
13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 63803e7 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_162 |
| findbugs | v3.1.0-RC1 |
| Test Results |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7394/testReport/ |
| Max. process+thread count | 411 (vs. ulimit of 1) |
| modules | C:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
U:
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core |
| Console output |
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/7394/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org |
This message was automatically generated.
> mapred job -history prints duplicate
[jira] [Commented] (MAPREDUCE-7072) mapred job -history prints duplicate counter in human output
[
https://issues.apache.org/jira/browse/MAPREDUCE-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16425564#comment-16425564
]
Wilfred Spiegelenburg commented on MAPREDUCE-7072:
--
The root cause of the issue is located in the {{AbstractCounters}} code
{{getGroupNames()}}
When you track through the code in the debugger the number of counter groups
returned is higher than expected. This is due to the fact that we add the
deprecated counters names to the list of counter group names before we return.
The display name of the counters that are tracked in the deprecated list,
stored in the legacyMap, are the same as the display names in the
non-deprecated counters. The deprecated counters added are already in the non
deprecated list which causes the duplication.
It works in the JSON format because it internally uses a HashMap. The HashMap
uses the name of the counter groups as the key. The keys clash and we thus
overwrite the existing value with the value from the deprecated value.
To track where this issue is coming from: MAPREDUCE-4053 changed the iteration
to work for oozie and seems related to OOZIE-777 and the HadoopELFunctions
which still seems to use the deprecated counter name.
Changing what the method returns is thus not possible without breaking oozie.
We can use the iterator that can be returned by the abstract counters as it
does not include the deprecated names.
> mapred job -history prints duplicate counter in human output
>
>
> Key: MAPREDUCE-7072
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-7072
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 3.0.0
>Reporter: Wilfred Spiegelenburg
>Assignee: Wilfred Spiegelenburg
>Priority: Major
>
> 'mapred job -history' command prints duplicate entries for counters only for
> the human output format. It does not do this for the JSON format.
> mapred job -history /user/history/somefile.jhist -format human
> {code}
>
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
> ...
> |Job Counters |Total megabyte-seconds taken by all map tasks|0 |0 |268,288,000
>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: mapreduce-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-h...@hadoop.apache.org
