[
https://issues.apache.org/jira/browse/MAPREDUCE-5931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14035711#comment-14035711
]
Jason Lowe commented on MAPREDUCE-5931:
---------------------------------------
Thanks for the patch, Gera. SleepJob is currently a "test" job, but I'd rather
not cement it further into the test-only area by having it explicitly pull in
junit.Assert for this. SleepJob is so useful outside of unit tests that
arguably it should be in mapreduce examples or something rather than isolated
to jobclient tests, but that's for another JIRA.
As such I'd rather see this error handled more like a "regular" job handles it,
like the usage error earlier in the code where it prints to stderr and returns
a non-zero exit code. This is another instance of a usage error, and it would
be nice to be consistent in how they are handled.
> Validate SleepJob command line parameters
> -----------------------------------------
>
> Key: MAPREDUCE-5931
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5931
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
> Affects Versions: 1.2.1, 2.4.0
> Reporter: Gera Shegalov
> Assignee: Gera Shegalov
> Priority: Minor
> Attachments: MAPREDUCE-5931.v01.patch
>
>
> This is a minor issue per se. I had a typo in my script specifying a negative
> number of reducers for the SleepJob. It results in the exception that is far
> from the root cause, and appeared as a serious issue with the map-side sort.
> {noformat}
> 2014-06-17 21:42:48,072 INFO [main] org.apache.hadoop.mapred.MapTask:
> Ignoring exception during close for
> org.apache.hadoop.mapred.MapTask$NewOutputCollector@972141f
> java.lang.NullPointerException
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1447)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:700)
> at org.apache.hadoop.mapred.MapTask.closeQuietly(MapTask.java:1990)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:774)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:173)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> 2014-06-17 21:42:48,075 WARN [main] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.IllegalArgumentException
> at java.nio.ByteBuffer.allocate(ByteBuffer.java:330)
> at org.apache.hadoop.mapred.SpillRecord.<init>(SpillRecord.java:51)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1824)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1484)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:700)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:770)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:340)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:173)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1626)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
