[
https://issues.apache.org/jira/browse/MAPREDUCE-6459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tao Jie updated MAPREDUCE-6459:
-------------------------------
Attachment: MAPREDUCE-6459.002.patch
> native task crashes when merging spilled file on ppc64
> ------------------------------------------------------
>
> Key: MAPREDUCE-6459
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6459
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Affects Versions: 2.6.0
> Environment: Linux version 2.6.32-431.el6.ppc64
> Reporter: Tao Jie
> Assignee: Tao Jie
> Attachments: MAPREDUCE-6459.001.patch, MAPREDUCE-6459.002.patch,
> ppc64_error.txt
>
>
> when running native task on ppc64，merging spilled files fails since we could
> not deserialize local spill file correctly.
> Function readVLong in WritableUtils.h and Buffers.h, we try to compare a char
> with a number and convert a char to int64_t. It does not work correctly on
> ppc64 since char definition is different between ppc64 and x86 platform. On
> x86 platform char is defined as signed number while on ppc64 char is
> unsigned. As a result, we write EOF marker [-1, -1] at the end of spill
> partition, but deserialize chars as [255, 255].
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
