I am not sure, what can be the issue...I had it long back and got no
response. I tried these things:
1. Increased the Child JVM heap size.
2. Reduced the number of reducers for the job.
3. Check whether your disks are not getting full while running the query.
3. Checked my data again. I think many times the error comes because of
dirty data. 1 easy way to check whether data is clean or not is to count
the number of delimiters/row. Sometimes,there are some other control
characters instead of space that we can't see in normal text editors, use
vi to check those also. Simple python hadoop streaming or pig scripts can
help you to do that.
Probably someone in community can give better answer to the exact problem.
I hope it would help.
Regards,
Praveenesh
On Fri, Sep 7, 2012 at 8:13 AM, 曹坤 <caofang...@gmail.com> wrote:
> Hi praveenesh kumar :
> I am getting the same error today.
> Do you have any solution ?
>
>
> 2012/3/23 praveenesh kumar <praveen...@gmail.com>
>
>> Hi all,
>>
>> I am getting this following error when I am trying to do select ...with
>> group by operation.I am grouping on around 25 columns
>>
>> java.lang.RuntimeException:
>> org.apache.hadoop.hive.ql.metadata.HiveException:
>> Hive Runtime Error: Unable to deseralize reduce input key from
>> x1x128x0x0x0x0x1x254x174x1x49x55x52x46x50x53x52x46x49x46x48x0x1x142x145x93x11x1x128x87x4x73x1x128x32x107x137x1x130x165x214x131x1x49x0x1x51x48x48x120x53x48x0x1x132x11x106x192x1x128x13x178x250x1x128x0x1x0x1x78x86x0x1x55x48x50x0x1x56x57x48x53x52x0x1x50x48x54x0x1x49x51x51x55x51x0x1x48x0x1x48x46x48x0x1x48x0x1x49x55x53x55x52x54x56x55x0x1x48x0x1x0x1x0x1x0x1x0x255
>> ...
>>
>>
>> Detailed logs...
>>
>> 2012-03-23 06:31:42,187 FATAL ExecReducer:
>> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error:
>> Unable to deserialize reduce input key from
>> x1x128x0x0x0x0x0x87x66x1x54x54x46x56x55x46x48x46x48x0x1x142x124x217x207x1x128x86x17x13x1x128x29x65x57x1x130x141x82x245x1x49x0x1x49x56x48x120x49x53x48x0x1x131x235x47x199x1x128x10x161x93x1x128x0x1x0x1x73x76x0x1x55x55x51x0x1x54x48x54x50x57x0x1x56x55x0x1x49x51x51x49x48x0x1x53x0x1x56x46x48x0x1x50x50x0x1x50x52x51x57x49x52x55x56x0x1x48x0x1x0x1x0x1x0x1x0x255
>> with properties
>> {columns=_col0,_col1,_col2,_col3,_col4,_col5,_col6,_col7,_col8,_col9,_col10,_col11,_col12,_col13,_col14,_col15,_col16,_col17,_col18,_col19,_col20,_col21,_col22,_col23,_col24,
>> serialization.sort.order=+++++++++++++++++++++++++,
>> columns.types=bigint,string,int,int,int,int,string,string,int,int,int,string,string,string,string,string,string,string,string,string,string,double,string,string,double}
>> at
>> org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:204)
>> at
>> org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:519)
>> at
>> org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:420)
>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>> at java.security.AccessController.doPrivileged(Native
>> Method)
>> at javax.security.auth.Subject.doAs(Subject.java:396)
>> at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1059)
>> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>> Caused by: org.apache.hadoop.hive.serde2.SerDeException:
>> java.io.EOFException
>> at
>> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:173)
>> at
>> org.apache.hadoop.hive.ql.exec.ExecReducer.reduce(ExecReducer.java:202)
>> ... 7 more
>> Caused by: java.io.EOFException
>> at
>> org.apache.hadoop.hive.serde2.binarysortable.InputByteBuffer.read(InputByteBuffer.java:54)
>> at
>> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:183)
>> at
>> org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe.deserialize(BinarySortableSerDe.java:169)
>> ... 8 more
>>
>>
>> Any help would be highly appreciated as I am kind of stuck here.
>>
>> Thanks.
>>
>
>
>
> --
> Best wishs！
> 曹 坤
>
>
