[
https://issues.apache.org/jira/browse/MAPREDUCE-5031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karthik Kambatla updated MAPREDUCE-5031:
----------------------------------------
Description:
While trying to reproduce MAPREDUCE-5028 on trunk, ran into what seems to be a
different issue. To reproduce:
Psuedo-dist mode: mapreduce.{map,reduce}.memory.mb=2048,
mapreduce.{map,reduce}.java.opts=-Xmx2048m, mapreduce.task.io.sort.mb=1280
The map tasks fail with the following error:
{noformat}
Error: java.lang.IndexOutOfBoundsException at
java.nio.Buffer.checkIndex(Buffer.java:512) at
java.nio.ByteBufferAsIntBufferL.put(ByteBufferAsIntBufferL.java:113) at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1141) at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:686) at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at
org.apache.hadoop.examples.WordCount$TokenizerMapper.map(WordCount.java:47) at
org.apache.hadoop.examples.WordCount$TokenizerMapper.map(WordCount.java:36) at
org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:757) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:396) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1488)
at
org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
{noformat}
was:
While trying to reproduce MAPREDUCE-5028 on trunk, ran into what seems to be a
different issue. To reproduce:
Psuedo-dist mode: mapreduce.{map,reduce}.memory.mb=2048,
mapreduce.{map,reduce}.java.opts=-Xmx2048m, mapreduce.task.io.sort.mb=1280
The map tasks fail with the following error:
{noformat}
Error: java.lang.IndexOutOfBoundsException at
java.nio.Buffer.checkIndex(Buffer.java:512) at
java.nio.ByteBufferAsIntBufferL.put(ByteBufferAsIntBufferL.java:113) at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1141) at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:686) at
org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
at
org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
at org.apache.hadoop.examples.WordCount$TokenizerMapper.map(WordCount.java:47)
at org.apache.hadoop.examples.WordCount$TokenizerMapper.map(WordCount.java:36)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at
org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:757) at
org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) at
org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) at
java.security.AccessController.doPrivileged(Native Method) at
javax.security.auth.Subject.doAs(Subject.java:396) at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1488)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
{noformat}
> Maps hitting IndexOutOfBoundsException for higher values of
> mapreduce.task.io.sort.mb
> -------------------------------------------------------------------------------------
>
> Key: MAPREDUCE-5031
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5031
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
> Affects Versions: 2.0.3-alpha, 0.23.5
> Reporter: Karthik Kambatla
> Assignee: Karthik Kambatla
>
> While trying to reproduce MAPREDUCE-5028 on trunk, ran into what seems to be
> a different issue. To reproduce:
> Psuedo-dist mode: mapreduce.{map,reduce}.memory.mb=2048,
> mapreduce.{map,reduce}.java.opts=-Xmx2048m, mapreduce.task.io.sort.mb=1280
> The map tasks fail with the following error:
> {noformat}
> Error: java.lang.IndexOutOfBoundsException at
> java.nio.Buffer.checkIndex(Buffer.java:512) at
> java.nio.ByteBufferAsIntBufferL.put(ByteBufferAsIntBufferL.java:113) at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1141)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:686)
> at
> org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
> at
> org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
> at
> org.apache.hadoop.examples.WordCount$TokenizerMapper.map(WordCount.java:47)
> at
> org.apache.hadoop.examples.WordCount$TokenizerMapper.map(WordCount.java:36)
> at
> org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at
> org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:757) at
> org.apache.hadoop.mapred.MapTask.run(MapTask.java:339) at
> org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:158) at
> java.security.AccessController.doPrivileged(Native Method) at
> javax.security.auth.Subject.doAs(Subject.java:396) at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1488)
> at
> org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:153)
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
