[ https://issues.apache.org/jira/browse/AMBARI-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16255983#comment-16255983 ]
Hudson commented on AMBARI-22444: --------------------------------- SUCCESS: Integrated in Jenkins build Ambari-trunk-Commit #8404 (See [https://builds.apache.org/job/Ambari-trunk-Commit/8404/]) AMBARI-22444 - Add Native Libraries To Tez Tarball (part4) (jhurley: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=365c91e92698f6a566b8503486f80c736af79e70]) * (edit) ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py > Add Native Libraries To Tez Tarball > ----------------------------------- > > Key: AMBARI-22444 > URL: https://issues.apache.org/jira/browse/AMBARI-22444 > Project: Ambari > Issue Type: Bug > Affects Versions: 2.6.1 > Reporter: Jonathan Hurley > Assignee: Jonathan Hurley > Priority: Blocker > Fix For: 2.6.1 > > Attachments: AMBARI-22444-2.patch, AMBARI-22444.patch > > > As part of the investigation for getting patch upgrades working in Ambari > 2.6.1, it was determined that the Tez tarball will need to have the hadoop > native libraries added to it so that they can be detected from the tarball. > STR: > - Install ZK, MapR, Tez, Yarn, Hive > - Enable a non-LZO codec, like Snappy > - Patch Hive to a new version > - Change the following properties in {{tez-site}}: > -- tez.am.launch.env = LD_LIBRARY=./tezlib/lib/native > -- tez.task.launch.env = LD_LIBRARY=./tezlib/lib/native > When Hive commands run, they will attempt to load the native snappy libraries > from the Tez tarball and will fail with: > {code} > Caused by: java.io.IOException: Unable to get CompressorType for codec > (org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to > missing native libraries for the codec. > at > org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.<init>(ExternalSorter.java:217) > {code} > In order to fix this, the Tez tarball should include the native hadoop > libraries as well: > {code} > ├── tez > │ ├── lib > │ │ ├── native > │ │ │ ├── libhadoop.a > │ │ │ ├── libhadoop.so -> libhadoop.so.1.0.0 > │ │ │ ├── libhadoop.so.1.0.0 > │ │ │ ├── libhadooppipes.a > │ │ │ ├── libhadooputils.a > │ │ │ ├── libhdfs.a > │ │ │ ├── libhdfs.so -> libhdfs.so.0.0.0 > │ │ │ ├── libhdfs.so.0.0.0 > │ │ │ ├── libsnappy.so -> libsnappy.so.1.1.4 > │ │ │ ├── libsnappy.so.1 -> libsnappy.so.1.1.4 > │ │ │ └── libsnappy.so.1.1.4 > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)