Hey,
I need help debugging the following exception, I tried googling but all the
questions are still open. The code had been running in prod since one
month, and this happened.
2016-08-25 09:21:20,130 WARN [main]
org.apache.hadoop.mapred.YarnChild: Exception running child :
java.lang.RuntimeException: next value iterator failed
at
org.apache.hadoop.mapreduce.task.ReduceContextImpl$ValueIterator.next(ReduceContextImpl.java:242)
at
com.flipkart.dpg.unified.session.reducer.SessionReducer.reduce(SessionReducer.java:54)
at
com.flipkart.dpg.unified.session.reducer.SessionReducer.reduce(SessionReducer.java:24)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:171)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:162)
Caused by: java.io.IOException: Invalid int encoding
at
org.apache.avro.io.DirectBinaryDecoder.readInt(DirectBinaryDecoder.java:113)
at org.apache.avro.io.BinaryDecoder.readString(BinaryDecoder.java:259)
at
org.apache.avro.io.ResolvingDecoder.readString(ResolvingDecoder.java:201)
at
org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:430)
at
org.apache.avro.reflect.ReflectDatumReader.readString(ReflectDatumReader.java:229)
at
org.apache.avro.generic.GenericDatumReader.readString(GenericDatumReader.java:422)
at
org.apache.avro.generic.GenericDatumReader.readWithoutConversion(GenericDatumReader.java:180)
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:152)
...
...
at
org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:144)
at
org.apache.avro.hadoop.io.AvroDeserializer.deserialize(AvroDeserializer.java:122)
at
org.apache.avro.hadoop.io.AvroDeserializer.deserialize(AvroDeserializer.java:48)
at
org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:146)
at
org.apache.hadoop.mapreduce.task.ReduceContextImpl$ValueIterator.next(ReduceContextImpl.java:239)
... 10 more
2016-08-25 09:21:20,132 INFO [main] org.apache.hadoop.mapred.Task:
Runnning cleanup for the task
Any pointers on how I can debug / fix this would be much appreciated.
Thanks,
Akshay Aggarwal
