[
https://issues.apache.org/jira/browse/HADOOP-11996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kai Zheng updated HADOOP-11996:
-------------------------------
Attachment: HADOOP-11996-v7.patch
Rebased the patch:
1. Building with {{mvn package -Pdist -Pnative -Drequire.isal -Dbundle.isal
-Disal.lib=/usr/lib -DskipTests}}
{noformat}
...
[INFO] Apache Hadoop Mini-Cluster ........................ SUCCESS [ 0.050 s]
[INFO] Apache Hadoop Scheduler Load Simulator ............ SUCCESS [ 3.277 s]
[INFO] Apache Hadoop Tools Dist .......................... SUCCESS [ 2.738 s]
[INFO] Apache Hadoop Kafka Library support ............... SUCCESS [ 1.460 s]
[INFO] Apache Hadoop Tools ............................... SUCCESS [ 0.020 s]
[INFO] Apache Hadoop Distribution ........................ SUCCESS [ 16.572 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 11:38 min
[INFO] Finished at: 2016-02-03T16:25:44+08:00
[INFO] Final Memory: 164M/896M
[INFO] ------------------------------------------------------------------------
{noformat}
2. Checknative
{noformat}
[root@zkdesk hadoop]# cd hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/
[root@zkdesk hadoop-3.0.0-SNAPSHOT]# bin/hadoop checknative
16/02/03 16:26:23 INFO bzip2.Bzip2Factory: Successfully loaded & initialized
native-bzip2 library system-native
16/02/03 16:26:23 INFO zlib.ZlibFactory: Successfully loaded & initialized
native-zlib library
Native library checking:
hadoop: true
/home/workspace/hadoop/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/lib/native/libhadoop.so.1.0.0
zlib: true /lib64/libz.so.1
snappy: true /lib64/libsnappy.so.1
lz4: true revision:99
bzip2: true /lib64/libbz2.so.1
openssl: true /lib64/libcrypto.so
ISA-L: true
/home/workspace/hadoop/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/lib/native/libisal.so.2
{noformat}
3. Run the test
{noformat}
[root@zkdesk hadoop]#
./hadoop-common-project/hadoop-common/target/native/erasure_code_test
Performing erasure code test
Recovering (numAllUnits = 9, numDataUnits = 6, numErased = 2)
- ErasedIndexes = 1 7 - DecodeIndex = 2 3 4 5 6 8
EncodeMatrix:
1 0 0 0 0 0 0 1 0
0 0 0 0 0 1 0 0 0
0 0 0 1 0 0 0 0 0
0 1 0 0 0 0 0 0 1
122 186 71 167 142 244 186 122 167
71 244 142 173 157 221 152 61 170
InvertMatrix:
2 173 10 213 24 40
62 183 132 127 27 36
1 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0 0 0 1 0 0
DecodeMatrix:
62 183 132 127 27 36 48 175 26
24 92 105 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0
Successfully done, passed!
{noformat}
4. Building on Windows passed.
> Native erasure coder facilities based on ISA-L
> ----------------------------------------------
>
> Key: HADOOP-11996
> URL: https://issues.apache.org/jira/browse/HADOOP-11996
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HADOOP-11996-initial.patch, HADOOP-11996-v2.patch,
> HADOOP-11996-v3.patch, HADOOP-11996-v4.patch, HADOOP-11996-v5.patch,
> HADOOP-11996-v6.patch, HADOOP-11996-v7.patch
>
>
> While working on HADOOP-11540 and etc., it was found useful to write the
> basic facilities based on Intel ISA-L library separately from JNI stuff. It's
> also easy to debug and troubleshooting, as no JNI or Java stuffs are involved.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
