Tao Jie created MAPREDUCE-6459:
----------------------------------
Summary: native task crashes when merging spilled file on ppc64
Key: MAPREDUCE-6459
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6459
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.6.0
Environment: Linux version 2.6.32-431.el6.ppc64
Reporter: Tao Jie
Attachments: ppc64_error.txt
when running native task on ppc64，merging spilled files fails since we could
not deserialize local spill file correctly.
Function readVLong in WritableUtils.h and Buffers.h, we try to compare a char
with a number and convert a char to int64_t. It does not work correctly on
ppc64 since char definition is different between ppc64 and x86 platform. On x86
platform char is defined as signed number while on ppc64 char is unsigned. As a
result, we write EOF marker [-1, -1] at the end of spill partition, but
deserialize chars as [255, 255].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
