[ https://issues.apache.org/jira/browse/MAPREDUCE-5506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Andrew Wang resolved MAPREDUCE-5506. ------------------------------------ Resolution: Won't Fix Resolving as WONTFIX since mr1 has been removed. > Hadoop-1.1.1 occurs ArrayIndexOutOfBoundsException with MultithreadedMapRunner > ------------------------------------------------------------------------------ > > Key: MAPREDUCE-5506 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5506 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv1 > Affects Versions: 1.1.1 > Environment: RHEL 6.3 x86_64 > Reporter: sam liu > Priority: Blocker > > After I set: > - 'jobConf.setMapRunnerClass(MultithreadedMapRunner.class);' in MR app > - 'mapred.map.multithreadedrunner.threads = 2' in mapred-site.xml > A simple MR app failed as its Map task encountered > ArrayIndexOutOfBoundsException as below(please ignore the line numbers in the > exception as I added some log print codes): > java.lang.ArrayIndexOutOfBoundsException > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer$Buffer.write(MapTask.java:1331) > at java.io.DataOutputStream.write(DataOutputStream.java:101) > at org.apache.hadoop.io.Text.write(Text.java:282) > at > org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:90) > at > org.apache.hadoop.io.serializer.WritableSerialization$WritableSerializer.serialize(WritableSerialization.java:77) > at > org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1060) > at > org.apache.hadoop.mapred.MapTask$OldOutputCollector.collect(MapTask.java:591) > at study.hadoop.mapreduce.sample.WordCount$Map.map(WordCount.java:41) > at study.hadoop.mapreduce.sample.WordCount$Map.map(WordCount.java:1) > at > org.apache.hadoop.mapred.lib.MultithreadedMapRunner$MapperInvokeRunable.run(MultithreadedMapRunner.java:231) > at > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:897) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:919) > at java.lang.Thread.run(Thread.java:738) > And the exception happens on line 'System.arraycopy(b, off, kvbuffer, > bufindex, len)' in MapTask.java#MapOutputBuffer#Buffer#write(). When the > exception occurs, 'b.length=4' but 'len=9'. > Btw, if I set 'mapred.map.multithreadedrunner.threads = 1', no exception > happened. So it should be an issue caused by multiple threads. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org