We are noticing shuffle BAD_ID errors using hadoop trunk and Yarn. The stack
traces are as follows and any ideas why these exceptions are happening and how
to fix them.
java.lang.IllegalArgumentException: Encoded byte size for String was -42, which
is outside of 0..1000 range.
at
org.apache.hadoop.io.WritableUtils.readStringSafely(WritableUtils.java:477)
at
org.apache.hadoop.mapreduce.task.reduce.ShuffleHeader.readFields(ShuffleHeader.java:60)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:363)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)
java.lang.IllegalArgumentException: TaskAttemptId string :
��^Z��m%attempt_1346265984657_000 is not properly formed
at
org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:187)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:364)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)
java.lang.IllegalArgumentException: TaskAttemptId string : @ is not properly
formed
at
org.apache.hadoop.mapreduce.TaskAttemptID.forName(TaskAttemptID.java:187)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyMapOutput(Fetcher.java:364)
at
org.apache.hadoop.mapreduce.task.reduce.Fetcher.copyFromHost(Fetcher.java:323)
at org.apache.hadoop.mapreduce.task.reduce.Fetcher.run(Fetcher.java:179)
Thanks
Bhaskar Devireddy
