[
https://issues.apache.org/jira/browse/HADOOP-17924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17418001#comment-17418001
]
Masatake Iwasaki edited comment on HADOOP-17924 at 9/21/21, 9:24 AM:
---------------------------------------------------------------------
I tried openssl-1.1.1l on CentOS 8. The native code of hadoop-common did not
have compilation issues.
I got following error on hadoop-pipe. The cause seems to be that
/lib64/libk5crypto.so (of krb5-libs) built against OS provided openssl does not
work with locally built openssl-1.1.1l. This looks like kind of build
environment issue mentioned by [~aajisaka] .
{noformat}
$ wget https://www.openssl.org/source/openssl-1.1.1l.tar.gz
$ tar zxf openssl-1.1.1l.tar.gz
$ cd openssl-1.1.1l
$ ./config --prefix=/usr/local -Wl,-rpath=/usr/local/lib64
$ make
$ sudo make install
$ openssl version
OpenSSL 1.1.1l 24 Aug 2021
$ cd path/to/hadoop
$ mvn clean install -Pdist -Pnative -Dopenssl.prefix=/usr/local -DskipTests
-DskipShade
...
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/c++ -g -O2 -Wall -pthread
-D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -rdynamic
CMakeFiles/wordcount-simple.dir/main/native/examples/impl/wordcount-simple.cc.o
-o examples/wordcount-simple -Wl,-rpath,/usr/local/lib64 libhadooppipes.a
libhadooputils.a /usr/local/lib64/libssl.so /usr/local/lib64/libcrypto.so -ldl
-ltirpc
[WARNING] make[2]: Leaving directory
'/home/centos/srcs/hadoop/hadoop-tools/hadoop-pipes/target/native'
[WARNING] make[2]: Leaving directory
'/home/centos/srcs/hadoop/hadoop-tools/hadoop-pipes/target/native'
[WARNING] make[1]: Leaving directory
'/home/centos/srcs/hadoop/hadoop-tools/hadoop-pipes/target/native'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_derive@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_ctrl@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_CTX_new_id@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_CTX_free@OPENSSL_1_1_1b'
[WARNING] collect2: error: ld returned 1 exit status
[WARNING] make[2]: *** [CMakeFiles/pipes-sort.dir/build.make:107:
examples/pipes-sort] Error 1
[WARNING] make[1]: *** [CMakeFiles/Makefile2:134:
CMakeFiles/pipes-sort.dir/all] Error 2
[WARNING] make[1]: *** Waiting for unfinished jobs....
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_derive@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_ctrl@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_CTX_new_id@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_CTX_free@OPENSSL_1_1_1b'
[WARNING] collect2: error: ld returned 1 exit status
[WARNING] make[2]: *** [CMakeFiles/wordcount-simple.dir/build.make:107:
examples/wordcount-simple] Error 1
[WARNING] make[1]: *** [CMakeFiles/Makefile2:106:
CMakeFiles/wordcount-simple.dir/all] Error 2
[WARNING] make: *** [Makefile:103: all] Error 2
{noformat}
was (Author: iwasakims):
I tried openssl-1.1.1l on CentOS 8. The native code of hadoop-common did not
have compilation issues.
I got following error on hadoop-pipe. The cause seems to be that
/lib64/libk5crypto.so built against OS provided krb5-libs does not work with
locally built openssl-1.1.1l. This looks like kind of build environment issue
mentioned by [~aajisaka] .
{noformat}
$ wget https://www.openssl.org/source/openssl-1.1.1l.tar.gz
$ tar zxf openssl-1.1.1l.tar.gz
$ cd openssl-1.1.1l
$ ./config --prefix=/usr/local -Wl,-rpath=/usr/local/lib64
$ make
$ sudo make install
$ openssl version
OpenSSL 1.1.1l 24 Aug 2021
$ cd path/to/hadoop
$ mvn clean install -Pdist -Pnative -Dopenssl.prefix=/usr/local -DskipTests
-DskipShade
...
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/c++ -g -O2 -Wall -pthread
-D_FILE_OFFSET_BITS=64 -D_GNU_SOURCE -rdynamic
CMakeFiles/wordcount-simple.dir/main/native/examples/impl/wordcount-simple.cc.o
-o examples/wordcount-simple -Wl,-rpath,/usr/local/lib64 libhadooppipes.a
libhadooputils.a /usr/local/lib64/libssl.so /usr/local/lib64/libcrypto.so -ldl
-ltirpc
[WARNING] make[2]: Leaving directory
'/home/centos/srcs/hadoop/hadoop-tools/hadoop-pipes/target/native'
[WARNING] make[2]: Leaving directory
'/home/centos/srcs/hadoop/hadoop-tools/hadoop-pipes/target/native'
[WARNING] make[1]: Leaving directory
'/home/centos/srcs/hadoop/hadoop-tools/hadoop-pipes/target/native'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_derive@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_ctrl@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_CTX_new_id@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_CTX_free@OPENSSL_1_1_1b'
[WARNING] collect2: error: ld returned 1 exit status
[WARNING] make[2]: *** [CMakeFiles/pipes-sort.dir/build.make:107:
examples/pipes-sort] Error 1
[WARNING] make[1]: *** [CMakeFiles/Makefile2:134:
CMakeFiles/pipes-sort.dir/all] Error 2
[WARNING] make[1]: *** Waiting for unfinished jobs....
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_derive@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_ctrl@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_CTX_new_id@OPENSSL_1_1_1b'
[WARNING] /opt/rh/gcc-toolset-9/root/usr/bin/ld: /lib64/libk5crypto.so.3:
undefined reference to `EVP_KDF_CTX_free@OPENSSL_1_1_1b'
[WARNING] collect2: error: ld returned 1 exit status
[WARNING] make[2]: *** [CMakeFiles/wordcount-simple.dir/build.make:107:
examples/wordcount-simple] Error 1
[WARNING] make[1]: *** [CMakeFiles/Makefile2:106:
CMakeFiles/wordcount-simple.dir/all] Error 2
[WARNING] make: *** [Makefile:103: all] Error 2
{noformat}
> Upgrade OpenSSL to 1.1.1l for vulnerability fix
> -----------------------------------------------
>
> Key: HADOOP-17924
> URL: https://issues.apache.org/jira/browse/HADOOP-17924
> Project: Hadoop Common
> Issue Type: Bug
> Reporter: Rajshree Mishra
> Priority: Major
>
> A vulnerability scan reported the following CVEs in openSSL 1.1.1k in Hadoop:
> CVE-2021-3711
> CVE-2021-3712
> Affects jars :
> libssl.so.1.1
> libcrypto.so.1.1
> libcrypto.a
> libssl.a
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
