xyu opened a new pull request #767: URL: https://github.com/apache/bigtop/pull/767
Current Bigtop 1.5.0 build env does not have [zstd compression](https://github.com/facebook/zstd) headers so the Hadoop Native Library that's built does not support `zstd`: ``` $ hadoop checknative -a 21/04/13 23:18:20 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native 21/04/13 23:18:20 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library Native library checking: hadoop: true /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0 zlib: true /lib/x86_64-linux-gnu/libz.so.1 snappy: true /usr/lib/hadoop/lib/native/libsnappy.so.1 zstd : false lz4: true revision:10301 bzip2: true /lib/x86_64-linux-gnu/libbz2.so.1 openssl: true /usr/lib/x86_64-linux-gnu/libcrypto.so 21/04/13 23:18:20 INFO util.ExitUtil: Exiting with status 1: ExitException ``` Update the toolchain to install headers for zstd which causes this feature to get turned on: ``` $ hadoop checknative -a 21/04/19 20:41:00 INFO bzip2.Bzip2Factory: Successfully loaded & initialized native-bzip2 library system-native 21/04/19 20:41:00 INFO zlib.ZlibFactory: Successfully loaded & initialized native-zlib library Native library checking: hadoop: true /usr/lib/hadoop/lib/native/libhadoop.so.1.0.0 zlib: true /lib/x86_64-linux-gnu/libz.so.1 snappy: true /usr/lib/hadoop/lib/native/libsnappy.so.1 zstd : true /usr/lib/x86_64-linux-gnu/libzstd.so.1 lz4: true revision:10301 bzip2: true /lib/x86_64-linux-gnu/libbz2.so.1 openssl: true /usr/lib/x86_64-linux-gnu/libcrypto.so ``` I have verified that these headers exist for all the supported distributions: ``` $ docker run -it --rm centos:7 sh -c 'rpm -ivh https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm >/dev/null 2>&1 ; yum search libzstd-devel 2>/dev/null' Loaded plugins: fastestmirror, ovl Determining fastest mirrors epel/x86_64/metalink | 7.7 kB 00:00 * base: linux-mirrors.fnal.gov * epel: mirror.team-cymru.com * extras: repo.ialab.dsu.edu * updates: mirrors.unifiedlayer.com base | 3.6 kB 00:00 epel | 4.7 kB 00:00 extras | 2.9 kB 00:00 updates | 2.9 kB 00:00 (1/7): base/7/x86_64/group_gz | 153 kB 00:00 (2/7): extras/7/x86_64/primary_db | 232 kB 00:00 (3/7): epel/x86_64/group_gz | 96 kB 00:01 (4/7): epel/x86_64/updateinfo | 1.0 MB 00:01 (5/7): base/7/x86_64/primary_db | 6.1 MB 00:01 (6/7): epel/x86_64/primary_db | 6.9 MB 00:01 (7/7): updates/7/x86_64/primary_db | 7.1 MB 00:01 ========================== N/S matched: libzstd-devel ========================== libzstd-devel.x86_64 : Header files for Zstd library Name and summary matches only, use "search all" for everything. ``` ``` $ docker run -it --rm centos:8 sh -c 'yum search libzstd-devel 2>/dev/null' ===================== Name Exactly Matched: libzstd-devel ====================== libzstd-devel.i686 : Header files for Zstd library libzstd-devel.x86_64 : Header files for Zstd library ``` ``` $ docker run -it --rm debian:10-slim sh -c 'apt-get update > /dev/null ; apt-cache search libzstd-dev' libzstd-dev - fast lossless compression algorithm -- development files ``` ``` $ docker run -it --rm fedora:33 sh -c 'yum search libzstd-devel 2>/dev/null' ===================== Name Exactly Matched: libzstd-devel ====================== libzstd-devel.i686 : Header files for Zstd library libzstd-devel.x86_64 : Header files for Zstd library ``` ``` $ docker run -it --rm ubuntu:18.04 sh -c 'apt-get update > /dev/null ; apt-cache search libzstd-dev' libzstd-dev - fast lossless compression algorithm -- development files libzstd1-dev - transitional package for libzstd-dev ``` ``` $ docker run -it --rm ubuntu:20.04 sh -c 'apt-get update > /dev/null ; apt-cache search libzstd-dev' libzstd-dev - fast lossless compression algorithm -- development files ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
