[
https://issues.apache.org/jira/browse/MAPREDUCE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14708648#comment-14708648
]
Tao Jie commented on MAPREDUCE-6459:
------------------------------------
Char represents [-127, 128] in x86 but [0, 255] in ppc64.
In atteached patch，when we read or write a char as a integer, we cast the char
byte as int8_t which represents [-127, 128] both in x86 and ppc64.
> native task crashes when merging spilled file on ppc64
> ------------------------------------------------------
>
> Key: MAPREDUCE-6459
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6459
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.6.0
> Environment: Linux version 2.6.32-431.el6.ppc64
> Reporter: Tao Jie
> Attachments: MAPREDUCE-6459.001.patch, ppc64_error.txt
>
>
> when running native task on ppc64，merging spilled files fails since we could
> not deserialize local spill file correctly.
> Function readVLong in WritableUtils.h and Buffers.h, we try to compare a char
> with a number and convert a char to int64_t. It does not work correctly on
> ppc64 since char definition is different between ppc64 and x86 platform. On
> x86 platform char is defined as signed number while on ppc64 char is
> unsigned. As a result, we write EOF marker [-1, -1] at the end of spill
> partition, but deserialize chars as [255, 255].
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
