[jira] [Commented] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987552#comment-16987552
]
Hudson commented on HADOOP-16441:
-
SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #17717 (See
[https://builds.apache.org/job/Hadoop-trunk-Commit/17717/])
HADOOP-16441. if use -Dbundle.openssl=true, bundled with unnecessary
(iwasakims: rev ccca5f4a404c02f27997b19bbc679b1d7d2c127f)
* (edit) dev-support/bin/dist-copynativelibs
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Assignee: KWON BYUNGCHANG
>Priority: Major
> Attachments: HADOOP-16441.001.patch, HADOOP-16441.002.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
> lrwxrwxrwx 1 magnum magnum 18 Jul 22 04:13 libhadoop.so ->
> libhadoop.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 1026006 Jul 22 04:13 libhadoop.so.1.0.0
> -rw-rw-r-- 1 magnum magnum 475720 Jul 22 04:22 libhadooputils.a
> -rw-rw-r-- 1 magnum magnum 458600 Jul 22 04:16 libhdfs.a
> lrwxrwxrwx 1 magnum magnum 16 Jul 22 04:16 libhdfs.so -> libhdfs.so.0.0.0
> -rwxrwxr-x 1 magnum magnum 286052 Jul 22 04:16 libhdfs.so.0.0.0
> -rw-r--r-- 1 magnum magnum 1393974 Jul 9 04:47 libisal.a
> -rwxr-xr-x 1 magnum magnum 915 Jul 9 04:47 libisal.la
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so ->
> libisal.so.2.0.27
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so.2 ->
> libisal.so.2.0.27
> -rwxr-xr-x 1 magnum magnum 767778 Jul 9 04:47 libisal.so.2.0.27
> ==
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so ->
> libk5crypto.so.3.1
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so.3 ->
> libk5crypto.so.3.1
> -rwxr-xr-x 1 magnum magnum 210840 May 9 2018 libk5crypto.so.3.1
> ==
> -rw-rw-r-- 1 magnum magnum 8584562 Jul 22 04:21 libnativetask.a
> lrwxrwxrwx 1 magnum magnum 22 Jul 22 04:21 libnativetask.so ->
> libnativetask.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 3393065 Jul 22 04:21 libnativetask.so.1.0.0
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so ->
> libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so.1 ->
> libsnappy.so.1.1.4
> -rwxr-xr-x 1 magnum magnum 23800 Jun 10 2014 libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so -> libzstd.so.1.4.0
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so.1 ->
> libzstd.so.1.4.0
> -rwxr-xr-x 1 magnum magnum 649784 Apr 29 16:58 libzstd.so.1.4.0
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16987543#comment-16987543
]
Masatake Iwasaki commented on HADOOP-16441:
---
+1. I'm committing this.
The 002 worked on Ubuntu 18.04 as expected. Unexpectedly bundled files such as
libhcrypto* and libk5crypto* was removed by applying the patch.
{noformat}
before:
$ ls hadoop-dist/target/hadoop-3.3.0-SNAPSHOT/lib/native
examples libcrypto.solibcrypto.so.1.1 libhadoop.so
libhadooppipes.a libhcrypto.so.4 libhdfs.a libhdfs.so.0.0.0
libk5crypto.so.3.1 libnativetask.so
libcrypto.a libcrypto.so.1.0.0 libhadoop.a libhadoop.so.1.0.0
libhadooputils.a libhcrypto.so.4.1.0 libhdfs.so libk5crypto.so.3
libnativetask.a libnativetask.so.1.0.0
after:
$ ls hadoop-dist/target/hadoop-3.3.0-SNAPSHOT/lib/native/
examples libcrypto.a libcrypto.so libcrypto.so.1.0.0 libcrypto.so.1.1
libhadoop.a libhadoop.so libhadoop.so.1.0.0 libhadooppipes.a
libhadooputils.a libhdfs.a libhdfs.so libhdfs.so.0.0.0 libnativetask.a
libnativetask.so libnativetask.so.1.0.0
{noformat}
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Assignee: KWON BYUNGCHANG
>Priority: Major
> Attachments: HADOOP-16441.001.patch, HADOOP-16441.002.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
> lrwxrwxrwx 1 magnum magnum 18 Jul 22 04:13 libhadoop.so ->
> libhadoop.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 1026006 Jul 22 04:13 libhadoop.so.1.0.0
> -rw-rw-r-- 1 magnum magnum 475720 Jul 22 04:22 libhadooputils.a
> -rw-rw-r-- 1 magnum magnum 458600 Jul 22 04:16 libhdfs.a
> lrwxrwxrwx 1 magnum magnum 16 Jul 22 04:16 libhdfs.so -> libhdfs.so.0.0.0
> -rwxrwxr-x 1 magnum magnum 286052 Jul 22 04:16 libhdfs.so.0.0.0
> -rw-r--r-- 1 magnum magnum 1393974 Jul 9 04:47 libisal.a
> -rwxr-xr-x 1 magnum magnum 915 Jul 9 04:47 libisal.la
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so ->
> libisal.so.2.0.27
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so.2 ->
> libisal.so.2.0.27
> -rwxr-xr-x 1 magnum magnum 767778 Jul 9 04:47 libisal.so.2.0.27
> ==
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so ->
> libk5crypto.so.3.1
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so.3 ->
> libk5crypto.so.3.1
> -rwxr-xr-x 1 magnum magnum 210840 May 9 2018 libk5crypto.so.3.1
> ==
> -rw-rw-r-- 1 magnum magnum 8584562 Jul 22 04:21 libnativetask.a
> lrwxrwxrwx 1 magnum magnum 22 Jul 22 04:21 libnativetask.so ->
> libnativetask.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 3393065 Jul 22 04:21 libnativetask.so.1.0.0
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so ->
> libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so.1 ->
> libsnappy.so.1.1.4
> -rwxr-xr-x 1 magnum magnum 23800 Jun 10 2014
[jira] [Commented] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890608#comment-16890608
]
Hadoop QA commented on HADOOP-16441:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 1m
5s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
17m 1s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
0s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
18s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 53s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 1m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 35m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=19.03.0 Server=19.03.0 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16441 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12975447/HADOOP-16441.002.patch
|
| Optional Tests | dupname asflicense shellcheck shelldocs |
| uname | Linux 04a7cbc02e11 4.15.0-48-generic #51-Ubuntu SMP Wed Apr 3
08:28:49 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / ee87e9a |
| maven | version: Apache Maven 3.3.9 |
| shellcheck | v0.4.6 |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16407/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Assignee: KWON BYUNGCHANG
>Priority: Major
> Attachments: HADOOP-16441.001.patch, HADOOP-16441.002.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
>
[jira] [Commented] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890575#comment-16890575
]
KWON BYUNGCHANG commented on HADOOP-16441:
--
[~iwasakims] Thank you for your feedback.
{{set -x}} is for debugging. I attached patch removed debugging code.
I agree with your opinion.
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Assignee: KWON BYUNGCHANG
>Priority: Major
> Attachments: HADOOP-16441.001.patch, HADOOP-16441.002.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
> lrwxrwxrwx 1 magnum magnum 18 Jul 22 04:13 libhadoop.so ->
> libhadoop.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 1026006 Jul 22 04:13 libhadoop.so.1.0.0
> -rw-rw-r-- 1 magnum magnum 475720 Jul 22 04:22 libhadooputils.a
> -rw-rw-r-- 1 magnum magnum 458600 Jul 22 04:16 libhdfs.a
> lrwxrwxrwx 1 magnum magnum 16 Jul 22 04:16 libhdfs.so -> libhdfs.so.0.0.0
> -rwxrwxr-x 1 magnum magnum 286052 Jul 22 04:16 libhdfs.so.0.0.0
> -rw-r--r-- 1 magnum magnum 1393974 Jul 9 04:47 libisal.a
> -rwxr-xr-x 1 magnum magnum 915 Jul 9 04:47 libisal.la
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so ->
> libisal.so.2.0.27
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so.2 ->
> libisal.so.2.0.27
> -rwxr-xr-x 1 magnum magnum 767778 Jul 9 04:47 libisal.so.2.0.27
> ==
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so ->
> libk5crypto.so.3.1
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so.3 ->
> libk5crypto.so.3.1
> -rwxr-xr-x 1 magnum magnum 210840 May 9 2018 libk5crypto.so.3.1
> ==
> -rw-rw-r-- 1 magnum magnum 8584562 Jul 22 04:21 libnativetask.a
> lrwxrwxrwx 1 magnum magnum 22 Jul 22 04:21 libnativetask.so ->
> libnativetask.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 3393065 Jul 22 04:21 libnativetask.so.1.0.0
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so ->
> libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so.1 ->
> libsnappy.so.1.1.4
> -rwxr-xr-x 1 magnum magnum 23800 Jun 10 2014 libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so -> libzstd.so.1.4.0
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so.1 ->
> libzstd.so.1.4.0
> -rwxr-xr-x 1 magnum magnum 649784 Apr 29 16:58 libzstd.so.1.4.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890296#comment-16890296
]
Masatake Iwasaki commented on HADOOP-16441:
---
Thanks for working on this, [~magnum].
{noformat}
+set -x
{noformat}
Is this intentional change? This looks like temporary one for your debugging.
The patch looks good to me overall. Since I'm using CentOS for development too,
it would be nice to get comments from someone using another Linux distro like
Ubuntu, while I think they have same naming convention for lib files.
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Priority: Major
> Attachments: HADOOP-16441.001.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
> lrwxrwxrwx 1 magnum magnum 18 Jul 22 04:13 libhadoop.so ->
> libhadoop.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 1026006 Jul 22 04:13 libhadoop.so.1.0.0
> -rw-rw-r-- 1 magnum magnum 475720 Jul 22 04:22 libhadooputils.a
> -rw-rw-r-- 1 magnum magnum 458600 Jul 22 04:16 libhdfs.a
> lrwxrwxrwx 1 magnum magnum 16 Jul 22 04:16 libhdfs.so -> libhdfs.so.0.0.0
> -rwxrwxr-x 1 magnum magnum 286052 Jul 22 04:16 libhdfs.so.0.0.0
> -rw-r--r-- 1 magnum magnum 1393974 Jul 9 04:47 libisal.a
> -rwxr-xr-x 1 magnum magnum 915 Jul 9 04:47 libisal.la
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so ->
> libisal.so.2.0.27
> lrwxrwxrwx 1 magnum magnum 17 Jul 9 04:47 libisal.so.2 ->
> libisal.so.2.0.27
> -rwxr-xr-x 1 magnum magnum 767778 Jul 9 04:47 libisal.so.2.0.27
> ==
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so ->
> libk5crypto.so.3.1
> lrwxrwxrwx 1 magnum magnum 18 Aug 3 2018 libk5crypto.so.3 ->
> libk5crypto.so.3.1
> -rwxr-xr-x 1 magnum magnum 210840 May 9 2018 libk5crypto.so.3.1
> ==
> -rw-rw-r-- 1 magnum magnum 8584562 Jul 22 04:21 libnativetask.a
> lrwxrwxrwx 1 magnum magnum 22 Jul 22 04:21 libnativetask.so ->
> libnativetask.so.1.0.0
> -rwxrwxr-x 1 magnum magnum 3393065 Jul 22 04:21 libnativetask.so.1.0.0
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so ->
> libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 18 Jul 9 04:45 libsnappy.so.1 ->
> libsnappy.so.1.1.4
> -rwxr-xr-x 1 magnum magnum 23800 Jun 10 2014 libsnappy.so.1.1.4
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so -> libzstd.so.1.4.0
> lrwxrwxrwx 1 magnum magnum 16 Jul 9 04:45 libzstd.so.1 ->
> libzstd.so.1.4.0
> -rwxr-xr-x 1 magnum magnum 649784 Apr 29 16:58 libzstd.so.1.4.0
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
-
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
[jira] [Commented] (HADOOP-16441) if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
[
https://issues.apache.org/jira/browse/HADOOP-16441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16889933#comment-16889933
]
Hadoop QA commented on HADOOP-16441:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
43s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
14m 48s{color} | {color:green} branch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} shellcheck {color} | {color:green} 0m
1s{color} | {color:green} There were no new shellcheck issues. {color} |
| {color:green}+1{color} | {color:green} shelldocs {color} | {color:green} 0m
16s{color} | {color:green} There were no new shelldocs issues. {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}
13m 7s{color} | {color:green} patch has no errors when building and testing
our client artifacts. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
36s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=18.09.7 Server=18.09.7 Image:yetus/hadoop:bdbca0e53b4 |
| JIRA Issue | HADOOP-16441 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12975368/HADOOP-16441.001.patch
|
| Optional Tests | dupname asflicense shellcheck shelldocs |
| uname | Linux 4ef5ed21554c 4.15.0-52-generic #56-Ubuntu SMP Tue Jun 4
22:49:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / acdb0a1 |
| maven | version: Apache Maven 3.3.9 |
| shellcheck | v0.4.6 |
| Max. process+thread count | 308 (vs. ulimit of 5500) |
| modules | C: . U: . |
| Console output |
https://builds.apache.org/job/PreCommit-HADOOP-Build/16401/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> if use -Dbundle.openssl=true, bundled with unnecessary libk5crypto.*
> -
>
> Key: HADOOP-16441
> URL: https://issues.apache.org/jira/browse/HADOOP-16441
> Project: Hadoop Common
> Issue Type: Bug
> Components: build
>Affects Versions: 3.1.2
>Reporter: KWON BYUNGCHANG
>Priority: Major
> Attachments: HADOOP-16441.001.patch
>
>
> If use -Dbundle.openssl to copy the contents of the openssl.lib directory
> into the final tar file,
> finally bundled with unnecessary libk5crypto.*
> build log
> {noformat}
> + bundle_native_lib true openssl.lib crypto /usr/lib64
> + declare bundleoption=true
> + declare liboption=openssl.lib
> + declare libpattern=crypto
> + declare libdir=/usr/lib64
> + echo 'Checking to bundle with:'
> + echo 'bundleoption=true, liboption=openssl.lib, pattern=crypto
> libdir=/usr/lib64'
> + [[ true != \t\r\u\e ]]
> + [[ -z /usr/lib64 ]]
> + [[ ! -d /usr/lib64 ]]
> + cd /usr/lib64
> + cd
> /home/magnum/hadoop-hdp/hadoop-common-project/hadoop-common/target/hadoop-common-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> + tar xfBp -
> + tar cf - ./libcrypto.so ./libcrypto.so.10 ./libcrypto.so.1.0.2k
> ./libk5crypto.so ./libk5crypto.so.3 ./libk5crypto.so.3.1
> {noformat}
>
> bundled native library list
> {noformat}
> [magnum@0dabe9f5564d hadoop-hdp]$ ls -al
> hadoop-dist/target/hadoop-3.1.1.3.1.2.3.1.0.0-78/lib/native/
> total 22704
> drwxrwxr-x 3 magnum magnum4096 Jul 22 04:22 .
> drwxrwxr-x 3 magnum magnum 20 Jul 22 04:30 ..
> drwxrwxr-x 2 magnum magnum 94 Jul 22 04:22 examples
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so ->
> libcrypto.so.1.0.2k
> lrwxrwxrwx 1 magnum magnum 19 Jul 9 03:20 libcrypto.so.10 ->
> libcrypto.so.1.0.2k
> -rwxr-xr-x 1 magnum magnum 2516624 Mar 12 10:12 libcrypto.so.1.0.2k
> -rw-rw-r-- 1 magnum magnum 1820202 Jul 22 04:13 libhadoop.a
> -rw-rw-r-- 1 magnum magnum 1607168 Jul 22 04:22 libhadooppipes.a
> lrwxrwxrwx 1 magnum magnum 18 Jul 22 04:13 libhadoop.so ->
>
