[jira] [Created] (MAPREDUCE-6605) Configuration options mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc-count.auto-incr NOT WORKING

2016-01-13 Thread Dong Zhen (JIRA)
Dong Zhen created MAPREDUCE-6605:


 Summary: Configuration options 
mapreduce.map.skip.proc.count.autoincr and 
mapreduce.reduce.skip.proc-count.auto-incr NOT WORKING
 Key: MAPREDUCE-6605
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6605
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Dong Zhen


As the default configuration file shows,
https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml

 we can configure options
mapreduce.map.skip.proc.count.autoincr
mapreduce.reduce.skip.proc.count.autoincr
in the mapred-default.xml. But they do not work because the expected keys in 
org.apache.hadoop.mapreduce.MRJobConfig.java

public static final String MAP_SKIP_INCR_PROC_COUNT = 
"mapreduce.map.skip.proc-count.auto-incr"
public static final String REDUCE_SKIP_INCR_PROC_COUNT = 
"mapreduce.reduce.skip.proc-count.auto-incr";

Or
in org.apache.hadoop.mapreduce.util.ConfigUtil.java

new DeprecationDelta("mapred.skip.map.auto.incr.proc.count",
MRJobConfig.MAP_SKIP_INCR_PROC_COUNT),
new DeprecationDelta("mapred.skip.reduce.auto.incr.proc.count",
MRJobConfig.REDUCE_SKIP_INCR_PROC_COUNT),

we can change them to mapreduce.map.skip.proc-count.auto-incr and 
mapreduce.reduce.skip.proc-count.auto-incr in the default configuration file.
 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6605) Configuration options mapreduce.map.skip.proc.count.autoincr and mapreduce.reduce.skip.proc-count.auto-incr NOT WORKING

2016-01-13 Thread Dong Zhen (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dong Zhen updated MAPREDUCE-6605:
-
Affects Version/s: 2.7.1

> Configuration options mapreduce.map.skip.proc.count.autoincr and 
> mapreduce.reduce.skip.proc-count.auto-incr NOT WORKING
> ---
>
> Key: MAPREDUCE-6605
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6605
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.7.1
>Reporter: Dong Zhen
>
> As the default configuration file shows,
> https://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
>  we can configure options
> mapreduce.map.skip.proc.count.autoincr
> mapreduce.reduce.skip.proc.count.autoincr
> in the mapred-default.xml. But they do not work because the expected keys in 
> org.apache.hadoop.mapreduce.MRJobConfig.java
> public static final String MAP_SKIP_INCR_PROC_COUNT = 
> "mapreduce.map.skip.proc-count.auto-incr"
> public static final String REDUCE_SKIP_INCR_PROC_COUNT = 
> "mapreduce.reduce.skip.proc-count.auto-incr";
> Or
> in org.apache.hadoop.mapreduce.util.ConfigUtil.java
> new DeprecationDelta("mapred.skip.map.auto.incr.proc.count",
> MRJobConfig.MAP_SKIP_INCR_PROC_COUNT),
> new DeprecationDelta("mapred.skip.reduce.auto.incr.proc.count",
> MRJobConfig.REDUCE_SKIP_INCR_PROC_COUNT),
> we can change them to mapreduce.map.skip.proc-count.auto-incr and 
> mapreduce.reduce.skip.proc-count.auto-incr in the default configuration file.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory

2016-01-13 Thread Jason Lowe (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15096769#comment-15096769
 ] 

Jason Lowe commented on MAPREDUCE-6554:
---

+1 latest patch lgtm.  Could you also provide a patch for branch-2.7/branch-2.6?

> MRAppMaster servicestart failing  with NPE in 
> MRAppMaster#parsePreviousJobHistory
> -
>
> Key: MAPREDUCE-6554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6554
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-MAPREDUCE-6554.patch, 0002-MAPREDUCE-6554.patch, 
> 0003-MAPREDUCE-6554.patch
>
>
> Create scenario so that MR app master gets preempted.
> On next MRAppMaster launch tried to recover previous job history file 
> {{MRAppMaster#parsePreviousJobHistory}}
> {noformat}
> 2015-11-21 13:52:27,722 INFO [main] 
> org.apache.hadoop.service.AbstractService: Service 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED; 
> cause: java.lang.NullPointerException
> java.lang.NullPointerException
> at java.io.StringReader.(StringReader.java:50)
> at org.apache.avro.Schema$Parser.parse(Schema.java:917)
> at org.apache.avro.Schema.parse(Schema.java:966)
> at 
> org.apache.hadoop.mapreduce.jobhistory.EventReader.(EventReader.java:75)
> at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:139)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.parsePreviousJobHistory(MRAppMaster.java:1256)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.processRecovery(MRAppMaster.java:1225)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1087)
> at 
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1673)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1566)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1499)
> 2015-11-21 13:52:27,725 INFO [main] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping 
> JobHistoryEventHandler. Size of the outstanding queue size is 0
> {noformat}
> EventReader(EventReader stream)
> {noformat}
>  this.version = in.readLine();
> ...
> Schema myschema = new 
> SpecificData(Event.class.getClassLoader()).getSchema(Event.class);
> this.schema = Schema.parse(in.readLine());
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6595) Fix findbugs warnings in OutputCommitter and FileOutputCommitter

2016-01-13 Thread Akira AJISAKA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097681#comment-15097681
 ] 

Akira AJISAKA commented on MAPREDUCE-6595:
--

Ran test-patch.sh for MAPREDUCE-6595.testing.patch with YETUS-271.03.patch.
{noformat}
| Vote |  Subsystem |  Runtime   | Comment

|  +1  |   @author  |  0m 00s| The patch does not contain any @author 
|  ||| tags.
|  -1  |test4tests  |  0m 00s| The patch doesn't appear to include any 
|  ||| new or modified tests. Please justify why
|  ||| no new tests are needed for this patch.
|  ||| Also please list what manual steps were
|  ||| performed to verify this patch.
|   0  |mvndep  |  1m 41s| Maven dependency ordering for branch 
|  +1  |mvninstall  |  7m 38s| trunk passed 
|  -1  |   compile  |  2m 17s| hadoop-mapreduce-project in trunk 
|  ||| failed.
|  +1  |checkstyle  |  0m 22s| trunk passed 
|  +1  |   mvnsite  |  3m 31s| trunk passed 
|  +1  |mvneclipse  |  1m 32s| trunk passed 
|   0  |  findbugs  |  0m 00s| Skipped branch modules with no Java 
|  ||| source: hadoop-mapreduce-project
|   0  |  findbugs  |  1m 38s| 
|  ||| 
hadoop-mapreduce-project/hadoop-mapreduce-
|  ||| client/hadoop-mapreduce-client-core in
|  ||| trunk has 2 extant Findbugs warnings.
|  +1  |   javadoc  |  2m 31s| trunk passed 
|   0  |mvndep  |  0m 41s| Maven dependency ordering for patch 
|  +1  |mvninstall  |  2m 25s| the patch passed 
|  -1  |   compile  |  1m 37s| hadoop-mapreduce-project in the patch 
|  ||| failed.
|  -1  | javac  |  1m 37s| hadoop-mapreduce-project in the patch 
|  ||| failed.
|  +1  |checkstyle  |  0m 25s| the patch passed 
|  +1  |   mvnsite  |  2m 54s| the patch passed 
|  +1  |mvneclipse  |  1m 04s| the patch passed 
|  +1  |whitespace  |  0m 00s| Patch has no whitespace issues. 
|  +1  |   xml  |  0m 01s| The patch has no ill-formed XML file. 
|   0  |  findbugs  |  0m 00s| Skipped patch modules with no Java 
|  ||| source: hadoop-mapreduce-project
|  +1  |  findbugs  |  1m 32s| the patch passed 
|  +1  |   javadoc  |  2m 05s| the patch passed 
|  -1  |asflicense  |  0m 38s| Patch generated 9 ASF License warnings. 
|  ||  36m 13s   | 
{noformat}
The output shows that the 2 findbugs warnings are fixed. (actually ignored)

> Fix findbugs warnings in OutputCommitter and FileOutputCommitter
> 
>
> Key: MAPREDUCE-6595
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6595
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Attachments: MAPREDUCE-6595.01.patch, MAPREDUCE-6595.testing.patch, 
> findbugsHtml.html
>
>
> There are 2 findbugs warnings in hadoop-mapreduce-client-core module. 
> https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6237/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6595) Fix findbugs warnings in OutputCommitter and FileOutputCommitter

2016-01-13 Thread Akira AJISAKA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097683#comment-15097683
 ] 

Akira AJISAKA commented on MAPREDUCE-6595:
--

Filed MAPREDUCE-6600.

> Fix findbugs warnings in OutputCommitter and FileOutputCommitter
> 
>
> Key: MAPREDUCE-6595
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6595
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Akira AJISAKA
>Assignee: Akira AJISAKA
> Attachments: MAPREDUCE-6595.01.patch, MAPREDUCE-6595.testing.patch, 
> findbugsHtml.html
>
>
> There are 2 findbugs warnings in hadoop-mapreduce-client-core module. 
> https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6237/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6540) TestMRTimelineEventHandling fails

2016-01-13 Thread Vinod Kumar Vavilapalli (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated MAPREDUCE-6540:
---
Fix Version/s: (was: 2.7.3)
   (was: 2.8.0)
   2.7.2

> TestMRTimelineEventHandling fails
> -
>
> Key: MAPREDUCE-6540
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6540
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: test
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
> Fix For: 2.7.2, 2.6.3
>
> Attachments: MAPREDUCE-6540.001.patch
>
>
> TestMRTimelineEventHandling fails after YARN-2859 is merged because it 
> changed the port the AHS binds to in a mini cluster.
> {noformat}
> Running org.apache.hadoop.mapred.TestMRTimelineEventHandling
> Tests run: 3, Failures: 0, Errors: 2, Skipped: 0, Time elapsed: 184.38 sec 
> <<< FAILURE! - in org.apache.hadoop.mapred.TestMRTimelineEventHandling
> testMRTimelineEventHandling(org.apache.hadoop.mapred.TestMRTimelineEventHandling)
>   Time elapsed: 70.528 sec  <<< ERROR!
> java.io.IOException: Job didn't finish in 30 seconds
>   at 
> org.apache.hadoop.mapred.UtilsForTests.runJobSucceed(UtilsForTests.java:622)
>   at 
> org.apache.hadoop.mapred.TestMRTimelineEventHandling.testMRTimelineEventHandling(TestMRTimelineEventHandling.java:99)
> testMapreduceJobTimelineServiceEnabled(org.apache.hadoop.mapred.TestMRTimelineEventHandling)
>   Time elapsed: 84.312 sec  <<< ERROR!
> java.io.IOException: Job didn't finish in 30 seconds
>   at 
> org.apache.hadoop.mapred.UtilsForTests.runJobSucceed(UtilsForTests.java:622)
>   at 
> org.apache.hadoop.mapred.TestMRTimelineEventHandling.testMapreduceJobTimelineServiceEnabled(TestMRTimelineEventHandling.java:162)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6549) multibyte delimiters with LineRecordReader cause duplicate records

2016-01-13 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097241#comment-15097241
 ] 

Hudson commented on MAPREDUCE-6549:
---

FAILURE: Integrated in Hadoop-trunk-Commit #9102 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9102/])
MAPREDUCE-6549. multibyte delimiters with LineRecordReader cause (vinodkv: rev 
9b33a03900c9ffdb9f4b4c59161fdce39ca8c73d)
* hadoop-mapreduce-project/CHANGES.txt


> multibyte delimiters with LineRecordReader cause duplicate records
> --
>
> Key: MAPREDUCE-6549
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6549
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv1, mrv2
>Affects Versions: 2.7.2
>Reporter: Dustin Cote
>Assignee: Wilfred Spiegelenburg
> Fix For: 2.7.2, 2.6.3
>
> Attachments: MAPREDUCE-6549-1.patch, MAPREDUCE-6549-2.patch, 
> MAPREDUCE-6549.3.patch
>
>
> LineRecorderReader currently produces duplicate records under certain 
> scenarios such as:
> 1) input string: "abc+++def++ghi++" 
> delimiter string: "+++" 
> test passes with all sizes of the split 
> 2) input string: "abc++def+++ghi++" 
> delimiter string: "+++" 
> test fails with a split size of 4 
> 2) input string: "abc+++def++ghi++" 
> delimiter string: "++" 
> test fails with a split size of 5 
> 3) input string "abc+++defg++hij++" 
> delimiter string: "++" 
> test fails with a split size of 4 
> 4) input string "abc++def+++ghi++" 
> delimiter string: "++" 
> test fails with a split size of 9 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6377) JHS sorting on state column not working in webUi

2016-01-13 Thread Vinod Kumar Vavilapalli (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6377?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated MAPREDUCE-6377:
---
Fix Version/s: (was: 2.7.3)
   2.7.2

Pulled this into 2.7.2 to keep the release up-to-date with 2.6.3. Changing 
fix-versions to reflect the same.

> JHS sorting on state column not working in webUi
> 
>
> Key: MAPREDUCE-6377
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6377
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: jobhistoryserver
>Affects Versions: 2.7.0
> Environment: 2 NM, JHS
>Reporter: Bibin A Chundatt
>Assignee: zhihai xu
>Priority: Minor
> Fix For: 2.7.2, 2.6.3
>
> Attachments: MAPREDUCE-6377.000.patch, Sorting Issue.png, 
> state_sorted1.pdf, state_sorted2.pdf
>
>
> Steps to reproduce
> 
> 1. Install and setup HA cluster with JHS
> 2.Create state in in JHS where few jobs are killed and Success
> Check sorting State in JHS WebUI
> Actual
> =
> Sorting on state column  not working in JHS
> Expected
> ==
> Sorting on state column should be working



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-5883) "Total megabyte-seconds" in job counters is slightly misleading

2016-01-13 Thread Vinod Kumar Vavilapalli (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated MAPREDUCE-5883:
---
Fix Version/s: (was: 2.7.3)
   2.7.2

Pulled this into 2.7.2 to keep the release up-to-date with 2.6.3. Changing 
fix-versions to reflect the same.

> "Total megabyte-seconds" in job counters is slightly misleading
> ---
>
> Key: MAPREDUCE-5883
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5883
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 3.0.0, 2.4.0
>Reporter: Nathan Roberts
>Assignee: Nathan Roberts
>Priority: Minor
>  Labels: BB2015-05-TBR
> Fix For: 2.7.2, 2.6.3
>
> Attachments: MAPREDUCE-5883.patch
>
>
> The following counters are in milliseconds so "megabyte-seconds" might be 
> better stated as "megabyte-milliseconds"
> MB_MILLIS_MAPS.name=   Total megabyte-seconds taken by all map 
> tasks
> MB_MILLIS_REDUCES.name=Total megabyte-seconds taken by all reduce 
> tasks
> VCORES_MILLIS_MAPS.name=   Total vcore-seconds taken by all map tasks
> VCORES_MILLIS_REDUCES.name=Total vcore-seconds taken by all reduce 
> tasks



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6549) multibyte delimiters with LineRecordReader cause duplicate records

2016-01-13 Thread Vinod Kumar Vavilapalli (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vinod Kumar Vavilapalli updated MAPREDUCE-6549:
---
Fix Version/s: (was: 2.7.3)
   (was: 2.8.0)
   2.7.2

Pulled this into 2.7.2 to keep the release up-to-date with 2.6.3. Changing 
fix-versions to reflect the same.

> multibyte delimiters with LineRecordReader cause duplicate records
> --
>
> Key: MAPREDUCE-6549
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6549
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv1, mrv2
>Affects Versions: 2.7.2
>Reporter: Dustin Cote
>Assignee: Wilfred Spiegelenburg
> Fix For: 2.7.2, 2.6.3
>
> Attachments: MAPREDUCE-6549-1.patch, MAPREDUCE-6549-2.patch, 
> MAPREDUCE-6549.3.patch
>
>
> LineRecorderReader currently produces duplicate records under certain 
> scenarios such as:
> 1) input string: "abc+++def++ghi++" 
> delimiter string: "+++" 
> test passes with all sizes of the split 
> 2) input string: "abc++def+++ghi++" 
> delimiter string: "+++" 
> test fails with a split size of 4 
> 2) input string: "abc+++def++ghi++" 
> delimiter string: "++" 
> test fails with a split size of 5 
> 3) input string "abc+++defg++hij++" 
> delimiter string: "++" 
> test fails with a split size of 4 
> 4) input string "abc++def+++ghi++" 
> delimiter string: "++" 
> test fails with a split size of 9 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers

2016-01-13 Thread Akira AJISAKA (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097401#comment-15097401
 ] 

Akira AJISAKA commented on MAPREDUCE-6363:
--

+1, committing this.

> [NNBench] Lease mismatch error when running with multiple mappers
> -
>
> Key: MAPREDUCE-6363
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks
>Reporter: Brahma Reddy Battula
>Assignee: Vlad Sharanhovich
>Priority: Critical
> Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, 
> MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch, 
> MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch, 
> MAPREDUCE-6363-008.patch, nnbench.log
>
>
> Command :
> ./yarn jar 
> ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar 
> nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 
> -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` 
> -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 
> 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 
> 192.168.105.214:36320: error: 
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:396)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers

2016-01-13 Thread Akira AJISAKA (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6363:
-
   Resolution: Fixed
 Hadoop Flags: Reviewed
Fix Version/s: 2.6.4
   2.7.3
   2.8.0
   Status: Resolved  (was: Patch Available)

Committed this to trunk, branch-2, branch-2.8, branch-2.7, and branch-2.6. 
Thanks [~bibinchundatt] for the contribution!

> [NNBench] Lease mismatch error when running with multiple mappers
> -
>
> Key: MAPREDUCE-6363
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks
>Reporter: Brahma Reddy Battula
>Assignee: Bibin A Chundatt
>Priority: Critical
> Fix For: 2.8.0, 2.7.3, 2.6.4
>
> Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, 
> MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch, 
> MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch, 
> MAPREDUCE-6363-008.patch, nnbench.log
>
>
> Command :
> ./yarn jar 
> ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar 
> nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 
> -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` 
> -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 
> 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 
> 192.168.105.214:36320: error: 
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:396)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Resolved] (MAPREDUCE-6465) NNBench result wrong when number of reducers greater than 1

2016-01-13 Thread Akira AJISAKA (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA resolved MAPREDUCE-6465.
--
Resolution: Duplicate

This issue is fixed by MAPREDUCE-6363. Closing this issue.
Thanks Bibin for reporting this and thanks Brahma for the comment.

> NNBench result wrong when number of reducers greater than 1
> ---
>
> Key: MAPREDUCE-6465
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6465
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
> Attachments: 0001-HDFS-8977.patch, 0002-MAPREDUCE-6465.patch
>
>
> Currently NNBench#analyzeResults consider only the part- for analysis
> {code}
> TPS: Create/Write/Close: 0
>  Avg exec time (ms): Create/Write/Close: Infinity
>  Avg Lat (ms): Create/Write: Infinity
> Avg Lat (ms): Close: NaN
> {code}
> Should consider all part files for output.
> or disable reduces option



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers

2016-01-13 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15097594#comment-15097594
 ] 

Hudson commented on MAPREDUCE-6363:
---

FAILURE: Integrated in Hadoop-trunk-Commit #9105 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/9105/])
MAPREDUCE-6363. [NNBench] Lease mismatch error when running with (aajisaka: rev 
7b0964f354e90968c2dac2f7acc17214732aed64)
* 
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/hdfs/NNBench.java
* hadoop-mapreduce-project/CHANGES.txt


> [NNBench] Lease mismatch error when running with multiple mappers
> -
>
> Key: MAPREDUCE-6363
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks
>Reporter: Brahma Reddy Battula
>Assignee: Bibin A Chundatt
>Priority: Critical
> Fix For: 2.8.0, 2.7.3, 2.6.4
>
> Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, 
> MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch, 
> MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch, 
> MAPREDUCE-6363-008.patch, nnbench.log
>
>
> Command :
> ./yarn jar 
> ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar 
> nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 
> -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` 
> -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 
> 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 
> 192.168.105.214:36320: error: 
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:396)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Updated] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers

2016-01-13 Thread Akira AJISAKA (JIRA)

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Akira AJISAKA updated MAPREDUCE-6363:
-
Assignee: Bibin A Chundatt  (was: Vlad Sharanhovich)

> [NNBench] Lease mismatch error when running with multiple mappers
> -
>
> Key: MAPREDUCE-6363
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks
>Reporter: Brahma Reddy Battula
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, 
> MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch, 
> MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch, 
> MAPREDUCE-6363-008.patch, nnbench.log
>
>
> Command :
> ./yarn jar 
> ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar 
> nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 
> -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` 
> -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 
> 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 
> 192.168.105.214:36320: error: 
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:396)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6554) MRAppMaster servicestart failing with NPE in MRAppMaster#parsePreviousJobHistory

2016-01-13 Thread Bibin A Chundatt (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15096245#comment-15096245
 ] 

Bibin A Chundatt commented on MAPREDUCE-6554:
-

[~jlowe]
Testcase/Findbug failures are not related to patch attached. Could you please 
review patch attached.

> MRAppMaster servicestart failing  with NPE in 
> MRAppMaster#parsePreviousJobHistory
> -
>
> Key: MAPREDUCE-6554
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6554
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Bibin A Chundatt
>Assignee: Bibin A Chundatt
>Priority: Critical
> Attachments: 0001-MAPREDUCE-6554.patch, 0002-MAPREDUCE-6554.patch, 
> 0003-MAPREDUCE-6554.patch
>
>
> Create scenario so that MR app master gets preempted.
> On next MRAppMaster launch tried to recover previous job history file 
> {{MRAppMaster#parsePreviousJobHistory}}
> {noformat}
> 2015-11-21 13:52:27,722 INFO [main] 
> org.apache.hadoop.service.AbstractService: Service 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED; 
> cause: java.lang.NullPointerException
> java.lang.NullPointerException
> at java.io.StringReader.(StringReader.java:50)
> at org.apache.avro.Schema$Parser.parse(Schema.java:917)
> at org.apache.avro.Schema.parse(Schema.java:966)
> at 
> org.apache.hadoop.mapreduce.jobhistory.EventReader.(EventReader.java:75)
> at 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryParser.parse(JobHistoryParser.java:139)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.parsePreviousJobHistory(MRAppMaster.java:1256)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.processRecovery(MRAppMaster.java:1225)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1087)
> at 
> org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster$4.run(MRAppMaster.java:1570)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1673)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1566)
> at 
> org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1499)
> 2015-11-21 13:52:27,725 INFO [main] 
> org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping 
> JobHistoryEventHandler. Size of the outstanding queue size is 0
> {noformat}
> EventReader(EventReader stream)
> {noformat}
>  this.version = in.readLine();
> ...
> Schema myschema = new 
> SpecificData(Event.class.getClassLoader()).getSchema(Event.class);
> this.schema = Schema.parse(in.readLine());
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Commented] (MAPREDUCE-6363) [NNBench] Lease mismatch error when running with multiple mappers

2016-01-13 Thread Bibin A Chundatt (JIRA)

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-6363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15096564#comment-15096564
 ] 

Bibin A Chundatt commented on MAPREDUCE-6363:
-

[~ajisakaa]
Testcase failures are not related to patch attached. Could you please help in 
reviewing patch.

> [NNBench] Lease mismatch error when running with multiple mappers
> -
>
> Key: MAPREDUCE-6363
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6363
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: benchmarks
>Reporter: Brahma Reddy Battula
>Assignee: Vlad Sharanhovich
>Priority: Critical
> Attachments: HDFS4929.patch, MAPREDUCE-6363-001.patch, 
> MAPREDUCE-6363-002.patch, MAPREDUCE-6363-003.patch, MAPREDUCE-6363-004.patch, 
> MAPREDUCE-6363-005.patch, MAPREDUCE-6363-006.patch, MAPREDUCE-6363-007.patch, 
> MAPREDUCE-6363-008.patch, nnbench.log
>
>
> Command :
> ./yarn jar 
> ../share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.0.1-tests.jar 
> nnbench -operation create_write -numberOfFiles 1000 -blockSize 268435456 
> -bytesToWrite 102400 -baseDir /benchmarks/NNBench`hostname -s` 
> -replicationFactorPerFile 3 -maps 100 -reduces 10
> Trace :
> 013-06-21 10:44:53,763 INFO org.apache.hadoop.ipc.Server: IPC Server handler 
> 7 on 9005, call org.apache.hadoop.hdfs.protocol.ClientProtocol.addBlock from 
> 192.168.105.214:36320: error: 
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
> org.apache.hadoop.hdfs.server.namenode.LeaseExpiredException: Lease mismatch 
> on /benchmarks/NNBenchlinux-185/data/file_linux-214__0 owned by 
> DFSClient_attempt_1371782327901_0001_m_48_0_1383437860_1 but is accessed 
> by DFSClient_attempt_1371782327901_0001_m_84_0_1880545303_1
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2351)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2098)
>   at 
> org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2019)
>   at 
> org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:501)
>   at 
> org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:213)
>   at 
> org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:52012)
>   at 
> org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:435)
>   at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:925)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1710)
>   at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1706)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:396)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1232)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)


[jira] [Created] (MAPREDUCE-6606) Findbug issue in org.apache.hadoop.mapred.OutputCommitter

2016-01-13 Thread Bibin A Chundatt (JIRA)
Bibin A Chundatt created MAPREDUCE-6606:
---

 Summary: Findbug issue in org.apache.hadoop.mapred.OutputCommitter
 Key: MAPREDUCE-6606
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6606
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Bibin A Chundatt


{noformat}
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext) 
doesn't override method in superclass because parameter type 
org.apache.hadoop.mapred.JobContext doesn't match superclass parameter type 
org.apache.hadoop.mapreduce.JobContext


Bug type NM_WRONG_PACKAGE_INTENTIONAL (click for details)
In class org.apache.hadoop.mapred.OutputCommitter
In method 
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext)
superclass is org.apache.hadoop.mapreduce.OutputCommitter
Did you intend to override 
org.apache.hadoop.mapreduce.OutputCommitter.isCommitJobRepeatable(JobContext)
Actual type org.apache.hadoop.mapred.JobContext
Expected org.apache.hadoop.mapreduce.JobContext
Overrides 
org.apache.hadoop.mapred.OutputCommitter.isCommitJobRepeatable(JobContext)
At OutputCommitter.java:[line 230]
{noformat}

https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/6268/artifact/patchprocess/branch-findbugs-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-core-warnings.html#Warnings_BAD_PRACTICE




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)