[
https://issues.apache.org/jira/browse/HADOOP-11887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14591085#comment-14591085
]
Colin Patrick McCabe commented on HADOOP-11887:
-----------------------------------------------
Thanks for waiting.
I can see you're using {{-Drequire.erasurecode}}, {{-Derasurecode.prefix}},
etc. for the build parameters. This might create confusion, though. Users
might ask if erasure encoding is disabled if they build without the
{{-Derasurecode...}} options. Given that the name of the library is ISA-L,
maybe we can have {{-Drequire.ISAL}}, {{-DISAL.prefix}}, etc.? Seems more
straightforward.
{code}
+add_executable(check_native_lib
+ ${T}/util/check_native_lib.c
+)
{code}
I can see why you wanted this new program, to tell you what was enabled and
what wasn't. But in Hadoop, this functionality is provided by {{hadoop
checknatve}}. Using {{checknative}} is a much better way of doing it since it
means that cluster administrators can figure out whether they have deployed a
version of {{libhadoop.so}} with or without any given feature.
I think we should avoid adding a new test program for this, and instead add
this functionality to {{checknative}}. Here is the current output of {{hadoop
checknative}}:
{code}
cmccabe@keter:~/> hadoop checknative
15/06/17 18:12:34 INFO bzip2.Bzip2Factory: Successfully loaded & initialized
native-bzip2 library system-native
15/06/17 18:12:34 INFO zlib.ZlibFactory: Successfully loaded & initialized
native-zlib library
Native library checking:
hadoop: true
/home/cmccabe/hadoop4/hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/lib/native/libhadoop.so.1.0.0
zlib: true /lib64/libz.so.1
snappy: true /usr/local/lib64/libsnappy.so.1
lz4: true revision:99
bzip2: true /usr/lib64/libbz2.so.1
openssl: true /usr/lib64/libcrypto.so
{code}
It would be very helpful to see an ISAL line in there, I think.
{code}
49 typedef unsigned char u8;
{code}
Should use the standard {{uint8_t}} type here from {{stdint.h}}
{code}
278 memset(src_in_err, 0, TEST_SOURCES);
{code}
Would be nice to use {{sizeof(src_in_err)}} here in case that size ever changes.
thanks, [~drankye].
> Introduce Intel ISA-L erasure coding library for the native support
> -------------------------------------------------------------------
>
> Key: HADOOP-11887
> URL: https://issues.apache.org/jira/browse/HADOOP-11887
> Project: Hadoop Common
> Issue Type: Sub-task
> Components: io
> Reporter: Kai Zheng
> Assignee: Kai Zheng
> Attachments: HADOOP-11887-v1.patch, HADOOP-11887-v2.patch,
> HADOOP-11887-v3.patch, HADOOP-11887-v4.patch
>
>
> This is to introduce Intel ISA-L erasure coding library for the native
> support, via dynamic loading mechanism (dynamic module, like *.so in *nix and
> *.dll on Windows).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
