[jira] [Updated] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Masatake Iwasaki updated HADOOP-16441:
--
Fix Version/s: 3.3.0
Hadoop Flags: Reviewed
Resolution: Fixed
Status: Resolved (was: Patch Available)
Committed. Thanks, [~magnum].
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Assignee: KWON BYUNGCHANG
>Priority: Major
> Fix For: 3.3.0
>
> Attachments: HADOOP-16441.001.patch, HADOOP-16441.002.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
> lrwxrwxrwx 1 magnum magnum 18 Jul 22 04:13 libhadoop.so ->
> libhadoop.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 1026006 Jul 22 04:13 libhadoop.so.1.0.0
> -rw-rw-r-- 1 magnum magnum 475720 Jul 22 04:22 libhadooputils.a
> -rw-rw-r-- 1 magnum magnum 458600 Jul 22 04:16 libhdfs.a
> lrwxrwxrwx 1 magnum magnum 16 Jul 22 04:16 libhdfs.so -> libhdfs.so.0.0.0
> -rwxrwxr-x 1 magnum magnum 286052 Jul 22 04:16 libhdfs.so.0.0.0
> -rw-r--r-- 1 magnum magnum 1393974 Jul 9 04:47 libisal.a
> -rwxr-xr-x 1 magnum magnum 915 Jul 9 04:47 libisal.la
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so ->
> libisal.so.2.0.27
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so.2 ->
> libisal.so.2.0.27
> -rwxr-xr-x 1 magnum magnum 767778 Jul 9 04:47 libisal.so.2.0.27
> ==
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so ->
> libk5crypto.so.3.1
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so.3 ->
> libk5crypto.so.3.1
> -rwxr-xr-x 1 magnum magnum 210840 May 9 2018 libk5crypto.so.3.1
> ==
> -rw-rw-r-- 1 magnum magnum 8584562 Jul 22 04:21 libnativetask.a
> lrwxrwxrwx 1 magnum magnum 22 Jul 22 04:21 libnativetask.so ->
> libnativetask.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 3393065 Jul 22 04:21 libnativetask.so.1.0.0
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so ->
> libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so.1 ->
> libsnappy.so.1.1.4
> -rwxr-xr-x 1 magnum magnum 23800 Jun 10 2014 libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so -> libzstd.so.1.4.0
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so.1 ->
> libzstd.so.1.4.0
> -rwxr-xr-x 1 magnum magnum 649784 Apr 29 16:58 libzstd.so.1.4.0
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KWON BYUNGCHANG updated HADOOP-16441:
-
Attachment: HADOOP-16441.002.patch
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Assignee: KWON BYUNGCHANG
>Priority: Major
> Attachments: HADOOP-16441.001.patch, HADOOP-16441.002.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
> lrwxrwxrwx 1 magnum magnum 18 Jul 22 04:13 libhadoop.so ->
> libhadoop.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 1026006 Jul 22 04:13 libhadoop.so.1.0.0
> -rw-rw-r-- 1 magnum magnum 475720 Jul 22 04:22 libhadooputils.a
> -rw-rw-r-- 1 magnum magnum 458600 Jul 22 04:16 libhdfs.a
> lrwxrwxrwx 1 magnum magnum 16 Jul 22 04:16 libhdfs.so -> libhdfs.so.0.0.0
> -rwxrwxr-x 1 magnum magnum 286052 Jul 22 04:16 libhdfs.so.0.0.0
> -rw-r--r-- 1 magnum magnum 1393974 Jul 9 04:47 libisal.a
> -rwxr-xr-x 1 magnum magnum 915 Jul 9 04:47 libisal.la
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so ->
> libisal.so.2.0.27
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so.2 ->
> libisal.so.2.0.27
> -rwxr-xr-x 1 magnum magnum 767778 Jul 9 04:47 libisal.so.2.0.27
> ==
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so ->
> libk5crypto.so.3.1
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so.3 ->
> libk5crypto.so.3.1
> -rwxr-xr-x 1 magnum magnum 210840 May 9 2018 libk5crypto.so.3.1
> ==
> -rw-rw-r-- 1 magnum magnum 8584562 Jul 22 04:21 libnativetask.a
> lrwxrwxrwx 1 magnum magnum 22 Jul 22 04:21 libnativetask.so ->
> libnativetask.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 3393065 Jul 22 04:21 libnativetask.so.1.0.0
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so ->
> libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so.1 ->
> libsnappy.so.1.1.4
> -rwxr-xr-x 1 magnum magnum 23800 Jun 10 2014 libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so -> libzstd.so.1.4.0
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so.1 ->
> libzstd.so.1.4.0
> -rwxr-xr-x 1 magnum magnum 649784 Apr 29 16:58 libzstd.so.1.4.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Updated] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
KWON BYUNGCHANG updated HADOOP-16441:
-
Attachment: HADOOP-16441.001.patch
Status: Patch Available (was: Open)
I have attched patch. I tested only on centos7 and it works well.
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Priority: Major
> Attachments: HADOOP-16441.001.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
> lrwxrwxrwx 1 magnum magnum 18 Jul 22 04:13 libhadoop.so ->
> libhadoop.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 1026006 Jul 22 04:13 libhadoop.so.1.0.0
> -rw-rw-r-- 1 magnum magnum 475720 Jul 22 04:22 libhadooputils.a
> -rw-rw-r-- 1 magnum magnum 458600 Jul 22 04:16 libhdfs.a
> lrwxrwxrwx 1 magnum magnum 16 Jul 22 04:16 libhdfs.so -> libhdfs.so.0.0.0
> -rwxrwxr-x 1 magnum magnum 286052 Jul 22 04:16 libhdfs.so.0.0.0
> -rw-r--r-- 1 magnum magnum 1393974 Jul 9 04:47 libisal.a
> -rwxr-xr-x 1 magnum magnum 915 Jul 9 04:47 libisal.la
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so ->
> libisal.so.2.0.27
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so.2 ->
> libisal.so.2.0.27
> -rwxr-xr-x 1 magnum magnum 767778 Jul 9 04:47 libisal.so.2.0.27
> ==
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so ->
> libk5crypto.so.3.1
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so.3 ->
> libk5crypto.so.3.1
> -rwxr-xr-x 1 magnum magnum 210840 May 9 2018 libk5crypto.so.3.1
> ==
> -rw-rw-r-- 1 magnum magnum 8584562 Jul 22 04:21 libnativetask.a
> lrwxrwxrwx 1 magnum magnum 22 Jul 22 04:21 libnativetask.so ->
> libnativetask.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 3393065 Jul 22 04:21 libnativetask.so.1.0.0
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so ->
> libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so.1 ->
> libsnappy.so.1.1.4
> -rwxr-xr-x 1 magnum magnum 23800 Jun 10 2014 libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so -> libzstd.so.1.4.0
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so.1 ->
> libzstd.so.1.4.0
> -rwxr-xr-x 1 magnum magnum 649784 Apr 29 16:58 libzstd.so.1.4.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
