Jun He created MAPREDUCE-6980:
---------------------------------
Summary: Add support of ARM64 hardware crc instructions for
nativetask
Key: MAPREDUCE-6980
URL: https://issues.apache.org/jira/browse/MAPREDUCE-6980
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: nativetask, performance
Environment: ARM64 platform with CRC extension
Reporter: Jun He
Priority: Minor
CRC instructions are introduced since ARM64V8 ISA. With this extension Arm
supports CRC32(0x04C11DB7) and CRC32C(0x1EDC6F41) checksum hardware computation.
This patch enables support on such arm64 platform, similiar as
[HADOOP-11660|https://issues.apache.org/jira/browse/HADOOP-11660].
Benchmark is done on a A57 platform using nttest
(hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/target/native/test/nttest),
and the result is as following:
Before (sw crc):
[ RUN ] Perf.CRC
17/10/10 03:05:18 INFO CRC time: 0.10112s size: 100.000M speed: 988.92M/s
17/10/10 03:05:18 INFO CRC32C time: 0.10043s size: 100.000M speed: 995.74M/s
[ OK ] Perf.CRC (216 ms)
After (hw crc):
[ RUN ] Perf.CRC
17/10/11 05:06:27 INFO CRC time: 0.03173s size: 100.000M speed: 3151.14M/s
17/10/11 05:06:27 INFO CRC32C time: 0.03146s size: 100.000M speed: 3178.97M/s
[ OK ] Perf.CRC (77 ms)
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
