[jira] [Commented] (MAPREDUCE-6656) [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363
[ https://issues.apache.org/jira/browse/MAPREDUCE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207851#comment-15207851 ] Akira AJISAKA commented on MAPREDUCE-6656: -- +1 pending Jenkins. Thank you for the quick fix, J.Andreina. > [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363 > > > Key: MAPREDUCE-6656 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6656 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.4 >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Blocker > Attachments: MAPREDUCE-6656-trunk-01.patch > > > After the fix of MAPREDUCE-6363, in NNBench OP_DELETE Operation isn't working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA updated MAPREDUCE-6580:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.7.3
2.8.0
Status: Resolved (was: Patch Available)
Committed this to branch-2.7 and above. Thanks [~ebadger] for the contribution!
> Test failure : TestMRJobsWithProfiler
> -
>
> Key: MAPREDUCE-6580
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6580
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Eric Badger
> Fix For: 2.8.0, 2.7.3
>
> Attachments: MAPREDUCE-6580.001.patch
>
>
> From
> [https://builds.apache.org/job/PreCommit-YARN-Build/9976/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt]
> TestMRJobsWithProfiler fails intermittently
> {code}
> Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 212.973 sec
> <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> testDifferentProfilers(org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler)
> Time elapsed: 133.116 sec <<< FAILURE!
> java.lang.AssertionError: expected:<4> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testProfilerInternal(TestMRJobsWithProfiler.java:212)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testDifferentProfilers(TestMRJobsWithProfiler.java:117)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6656) [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363
[ https://issues.apache.org/jira/browse/MAPREDUCE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated MAPREDUCE-6656: -- Status: Patch Available (was: Open) > [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363 > > > Key: MAPREDUCE-6656 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6656 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.4 >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Blocker > Attachments: MAPREDUCE-6656-trunk-01.patch > > > After the fix of MAPREDUCE-6363, in NNBench OP_DELETE Operation isn't working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6656) [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363
[ https://issues.apache.org/jira/browse/MAPREDUCE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] J.Andreina updated MAPREDUCE-6656: -- Attachment: MAPREDUCE-6656-trunk-01.patch Thanks [~ajisakaa] for checking this issue . Have uploaded the patch. Please review. > [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363 > > > Key: MAPREDUCE-6656 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6656 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.4 >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Blocker > Attachments: MAPREDUCE-6656-trunk-01.patch > > > After the fix of MAPREDUCE-6363, in NNBench OP_DELETE Operation isn't working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6656) [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363
[ https://issues.apache.org/jira/browse/MAPREDUCE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207831#comment-15207831 ] J.Andreina commented on MAPREDUCE-6656: --- Ok I'll provide the patch soon > [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363 > > > Key: MAPREDUCE-6656 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6656 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.4 >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Blocker > > After the fix of MAPREDUCE-6363, in NNBench OP_DELETE Operation isn't working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6315) Implement retrieval of logs for crashed MR-AM via jhist in the staging directory
[ https://issues.apache.org/jira/browse/MAPREDUCE-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated MAPREDUCE-6315: - Attachment: MAPREDUCE-6315.003.patch not sure yet if the test failures are related, but fixing the checkstyle error already. > Implement retrieval of logs for crashed MR-AM via jhist in the staging > directory > > > Key: MAPREDUCE-6315 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6315 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: client, mr-am >Affects Versions: 2.7.0 >Reporter: Gera Shegalov >Assignee: Gera Shegalov >Priority: Critical > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-6315.001.patch, MAPREDUCE-6315.002.patch, > MAPREDUCE-6315.003.patch > > > When all AM attempts crash, there is no record of them in JHS. Thus no easy > way to get the logs. This JIRA automates the procedure by utilizing the jhist > file in the staging directory. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-5937) hadoop/mapred job -history shows the counters twice in the output.
[ https://issues.apache.org/jira/browse/MAPREDUCE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weiwei Yang updated MAPREDUCE-5937: --- Assignee: (was: Weiwei Yang) > hadoop/mapred job -history shows the counters twice in the > output. > - > > Key: MAPREDUCE-5937 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5937 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.2.0 >Reporter: Jinghui Wang > Labels: BB2015-05-TBR > Attachments: MAPREDUCE-5937.patch, job_history_cli_sample.out > > > HiistoryView#printCounters method uses AbstractCounter#getGroupNames, which > includes legacy groups can cause duplicates on CLI output. > See attached example output. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6110) JobHistoryServer CLI throws NullPointerException with job ids that do not exist
[
https://issues.apache.org/jira/browse/MAPREDUCE-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207445#comment-15207445
]
Hudson commented on MAPREDUCE-6110:
---
FAILURE: Integrated in Hadoop-trunk-Commit #9485 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/9485/])
MAPREDUCE-6110. JobHistoryServer CLI throws NullPointerException with
(gtcarrera9: rev 57ead18a85e15aef1993f49157cf05aed38f1c87)
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/tools/CLI.java
*
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/tools/TestCLI.java
> JobHistoryServer CLI throws NullPointerException with job ids that do not
> exist
> ---
>
> Key: MAPREDUCE-6110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Reporter: Li Lu
>Assignee: Kai Sasaki
>Priority: Minor
> Fix For: 2.9.0
>
> Attachments: MAPREDUCE-6110.01.patch, MAPREDUCE-6110.02.patch,
> MAPREDUCE-6110.03.patch
>
>
> When using JobHistoryServer CLI to query a job id that does not exist on the
> server, it may throw NullPointerException sometimes.
> I tried "mapred job -events 0 100", and the result was:
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.tools.CLI.listEvents(CLI.java:487)
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:316)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
> Similar symptoms also appear with -list-attempt-ids, but were fine with
> -status and -set-priority.
> I traced back to CLI.listEvents, and line 487 is:
> {code}
> TaskCompletionEvent[] events = job.
> getTaskCompletionEvents(fromEventId, numEvents);
> {code}
> The job object is obtained from JobID.forName(jobid)) (line 316), which will
> return null if the job does not exist on server.
> Maybe we want to have some behaviors consistent with -status here, by simply
> reporting jobId does not exist?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6110) JobHistoryServer CLI throws NullPointerException with job ids that do not exist
[
https://issues.apache.org/jira/browse/MAPREDUCE-6110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Lu updated MAPREDUCE-6110:
-
Resolution: Fixed
Hadoop Flags: Reviewed
Fix Version/s: 2.9.0
Status: Resolved (was: Patch Available)
I have committed the patch into trunk and branch-2. Thanks [~lewuathe] for the
contribution!
> JobHistoryServer CLI throws NullPointerException with job ids that do not
> exist
> ---
>
> Key: MAPREDUCE-6110
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6110
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: jobhistoryserver
>Reporter: Li Lu
>Assignee: Kai Sasaki
>Priority: Minor
> Fix For: 2.9.0
>
> Attachments: MAPREDUCE-6110.01.patch, MAPREDUCE-6110.02.patch,
> MAPREDUCE-6110.03.patch
>
>
> When using JobHistoryServer CLI to query a job id that does not exist on the
> server, it may throw NullPointerException sometimes.
> I tried "mapred job -events 0 100", and the result was:
> Exception in thread "main" java.lang.NullPointerException
> at org.apache.hadoop.mapreduce.tools.CLI.listEvents(CLI.java:487)
> at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:316)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1237)
> Similar symptoms also appear with -list-attempt-ids, but were fine with
> -status and -set-priority.
> I traced back to CLI.listEvents, and line 487 is:
> {code}
> TaskCompletionEvent[] events = job.
> getTaskCompletionEvents(fromEventId, numEvents);
> {code}
> The job object is obtained from JobID.forName(jobid)) (line 316), which will
> return null if the job does not exist on server.
> Maybe we want to have some behaviors consistent with -status here, by simply
> reporting jobId does not exist?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (MAPREDUCE-6657) job history server can fail on startup when NameNode is in start phase
Haibo Chen created MAPREDUCE-6657: - Summary: job history server can fail on startup when NameNode is in start phase Key: MAPREDUCE-6657 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6657 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver Reporter: Haibo Chen Assignee: Haibo Chen Job history server will try to create a history directory in HDFS on startup. When NameNode is in safe mode, it will keep retrying for a configurable time period. However, it should also keeps retrying if the name node is in start state. Safe mode does not happen until the NN is out of the startup phase. A RetriableException with the text "NameNode still not started" is thrown when the NN is in its internal service startup phase. We should add the check for this specific exception in isBecauseSafeMode() to account for that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6535) TaskID default constructor results in NPE on toString()
[
https://issues.apache.org/jira/browse/MAPREDUCE-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207209#comment-15207209
]
Hadoop QA commented on MAPREDUCE-6535:
--
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 11s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
54s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 24s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
17s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 30s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 4s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 25s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
24s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 2m 46s {color}
| {color:red}
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.8.0_74
with JDK v1.8.0_74 generated 3 new + 136 unchanged - 0 fixed = 139 total (was
136) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 3m 8s {color}
| {color:red}
hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-jdk1.7.0_95
with JDK v1.7.0_95 generated 3 new + 139 unchanged - 0 fixed = 142 total (was
139) {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
15s {color} | {color:green}
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core:
patch generated 0 new + 13 unchanged - 6 fixed = 13 total (was 19) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 29s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
12s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 22s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m 52s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.8.0_74. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 16s
{color} | {color:green} hadoop-mapreduce-client-core in the patch passed with
JDK v1.7.0_95. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s {color} | {color:green} Patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} |
[jira] [Updated] (MAPREDUCE-6535) TaskID default constructor results in NPE on toString()
[
https://issues.apache.org/jira/browse/MAPREDUCE-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Templeton updated MAPREDUCE-6535:
Attachment: MAPREDUCE-6535.001.patch
This patch sets the default task type to REDUCE. It also fixes all the
javadocs and add a full set of unit tests.
> TaskID default constructor results in NPE on toString()
> ---
>
> Key: MAPREDUCE-6535
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6535
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 2.6.0
>Reporter: Daniel Templeton
>Assignee: Daniel Templeton
> Attachments: MAPREDUCE-6535.001.patch
>
>
> This code will reproduce the issue:
> {code}
> new TaskAttemptID().toString();
> {code}
> The issue is that the default constructor leaves the type {{null}}. The
> {{get()}} in {{CharTaskTypesMaps.getRepresentingCharacter()}} then throws an
> NPE on the null type key.
> The simplest solution would be to only call the {{get()}} on line 288 of
> {{TaskID.java}} if {{type}} is not {{null}} and return some other literal
> otherwise. Since no part of the code is tripping on the NPE, what we choose
> for the literal shouldn't matter. How about "x"?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206814#comment-15206814
]
Akira AJISAKA commented on MAPREDUCE-6580:
--
Nice catch, Eric! LGTM, +1.
> Test failure : TestMRJobsWithProfiler
> -
>
> Key: MAPREDUCE-6580
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6580
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Eric Badger
> Attachments: MAPREDUCE-6580.001.patch
>
>
> From
> [https://builds.apache.org/job/PreCommit-YARN-Build/9976/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt]
> TestMRJobsWithProfiler fails intermittently
> {code}
> Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 212.973 sec
> <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> testDifferentProfilers(org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler)
> Time elapsed: 133.116 sec <<< FAILURE!
> java.lang.AssertionError: expected:<4> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testProfilerInternal(TestMRJobsWithProfiler.java:212)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testDifferentProfilers(TestMRJobsWithProfiler.java:117)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6656) [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363
[ https://issues.apache.org/jira/browse/MAPREDUCE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206781#comment-15206781 ] Akira AJISAKA commented on MAPREDUCE-6656: -- Rethinking this, I decided to fix this issue here. [comment|https://issues.apache.org/jira/browse/MAPREDUCE-6363?focusedCommentId=15206759=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15206759] Hi [~andreina], now you are the assignee, so would you provide a patch? > [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363 > > > Key: MAPREDUCE-6656 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6656 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.4 >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Blocker > > After the fix of MAPREDUCE-6363, in NNBench OP_DELETE Operation isn't working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (MAPREDUCE-6656) [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363
[ https://issues.apache.org/jira/browse/MAPREDUCE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA updated MAPREDUCE-6656: - Affects Version/s: 2.6.4 Target Version/s: 2.8.0, 2.7.3, 2.6.5 Priority: Blocker (was: Major) Description: After the fix of MAPREDUCE-6363, in NNBench OP_DELETE Operation isn't working. (was: After the fix of MAPREDUCE-6363 , in NNBench OP_DELETE Operation is'nt working) Summary: [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363 (was: [NNBench] OP_DELETE Operation is'nt working) > [NNBench] OP_DELETE operation isn't working after MAPREDUCE-6363 > > > Key: MAPREDUCE-6656 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6656 > Project: Hadoop Map/Reduce > Issue Type: Bug >Affects Versions: 2.6.4 >Reporter: J.Andreina >Assignee: J.Andreina >Priority: Blocker > > After the fix of MAPREDUCE-6363, in NNBench OP_DELETE Operation isn't working. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Moved] (MAPREDUCE-6656) [NNBench] OP_DELETE Operation is'nt working
[ https://issues.apache.org/jira/browse/MAPREDUCE-6656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA moved HDFS-10191 to MAPREDUCE-6656: - Key: MAPREDUCE-6656 (was: HDFS-10191) Project: Hadoop Map/Reduce (was: Hadoop HDFS) > [NNBench] OP_DELETE Operation is'nt working > --- > > Key: MAPREDUCE-6656 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6656 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: J.Andreina >Assignee: J.Andreina > > After the fix of MAPREDUCE-6363 , in NNBench OP_DELETE Operation is'nt working -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[
https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA resolved MAPREDUCE-6363.
--
Resolution: Fixed
Target Version/s: 2.6.4, 2.8.0 (was: 2.8.0, 2.6.4)
Rethinking this, it would be better to add {{doDeleteOp}} in a separate jira
because 2.6.4, the v8 patch was committed to, has been released.
> [NNBench] Lease mismatch error when running with multiple mappers
> -
>
> Key: MAPREDUCE-6363
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Reporter: Brahma Reddy Battula
>Assignee: Bibin A Chundatt
>Priority: Critical
> Fix For: 2.8.0, 2.7.3, 2.6.4
>
> Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch,
> MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch,
> MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch,
> MAPREDUCE-6363-008.patch, nnbench.log
>
>
> Command :
> ./yarn jar
> ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar
> nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456
> -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s`
> -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from
> 192.168.105.214:36320: error:
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Reopened] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers
[
https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akira AJISAKA reopened MAPREDUCE-6363:
--
Reopen this because of missing {{doDeleteOp(fileName, reporter)}} in the v8
patch. Thanks [~andreina] for reporting this issue in HDFS-10191.
> [NNBench] Lease mismatch error when running with multiple mappers
> -
>
> Key: MAPREDUCE-6363
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Reporter: Brahma Reddy Battula
>Assignee: Bibin A Chundatt
>Priority: Critical
> Fix For: 2.8.0, 2.7.3, 2.6.4
>
> Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch,
> MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch,
> MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch,
> MAPREDUCE-6363-008.patch, nnbench.log
>
>
> Command :
> ./yarn jar
> ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar
> nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456
> -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s`
> -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler
> 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from
> 192.168.105.214:36320: error:
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
> at
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
> at
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
> at
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
> at
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
> at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
> at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15206503#comment-15206503
]
Eric Badger commented on MAPREDUCE-6580:
TestUberAM and TestMRJobs both have failed recent builds in [MAPREDUCE-6315]
and are unrelated to this patch.
TestMRIntermediateDataEncryption fails intermittently with a timeout in recent
builds [MAPREDUCE-6579], [YARN-4340].
> Test failure : TestMRJobsWithProfiler
> -
>
> Key: MAPREDUCE-6580
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6580
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Reporter: Rohith Sharma K S
>Assignee: Eric Badger
> Attachments: MAPREDUCE-6580.001.patch
>
>
> From
> [https://builds.apache.org/job/PreCommit-YARN-Build/9976/artifact/patchprocess/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient-jdk1.8.0_66.txt]
> TestMRJobsWithProfiler fails intermittently
> {code}
> Running org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 212.973 sec
> <<< FAILURE! - in org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler
> testDifferentProfilers(org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler)
> Time elapsed: 133.116 sec <<< FAILURE!
> java.lang.AssertionError: expected:<4> but was:<1>
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.failNotEquals(Assert.java:743)
> at org.junit.Assert.assertEquals(Assert.java:118)
> at org.junit.Assert.assertEquals(Assert.java:555)
> at org.junit.Assert.assertEquals(Assert.java:542)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testProfilerInternal(TestMRJobsWithProfiler.java:212)
> at
> org.apache.hadoop.mapreduce.v2.TestMRJobsWithProfiler.testDifferentProfilers(TestMRJobsWithProfiler.java:117)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (MAPREDUCE-6580) Test failure : TestMRJobsWithProfiler
[
https://issues.apache.org/jira/browse/MAPREDUCE-6580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205933#comment-15205933
]
Hadoop QA commented on MAPREDUCE-6580:
--
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 12s
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s {color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 22s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 28s
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 11s
{color} | {color:green} trunk passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 14s
{color} | {color:green} trunk passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
23s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 17s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 19s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 25s
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
11s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
35s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 9s
{color} | {color:green} the patch passed with JDK v1.8.0_74 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 12s
{color} | {color:green} the patch passed with JDK v1.7.0_95 {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 94m 51s {color}
| {color:red} hadoop-mapreduce-client-jobclient in the patch failed with JDK
v1.8.0_74. {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 110m 49s
{color} | {color:red} hadoop-mapreduce-client-jobclient in the patch failed
with JDK v1.7.0_95. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m 28s
{color} | {color:red} Patch generated 19 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 219m 22s {color}
| {color:black} {color} |
\\
\\
|| Reason || Tests ||
| JDK v1.8.0_74 Failed junit tests | hadoop.mapreduce.v2.TestUberAM |
| | hadoop.mapreduce.v2.TestMRJobs |
| JDK v1.7.0_95 Failed junit tests | hadoop.mapreduce.v2.TestUberAM |
| | hadoop.mapreduce.v2.TestMRJobs |
| JDK v1.7.0_95 Timed out junit tests |
org.apache.hadoop.mapred.TestMRIntermediateDataEncryption |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Image:yetus/hadoop:fbe3e86 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12794629/MAPREDUCE-6580.001.patch
|
| JIRA Issue | MAPREDUCE-6580 |
| Optional Tests | asflicense compile javac javadoc mvninstall mvnsite
unit findbugs
