[ https://issues.apache.org/jira/browse/AMBARI-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jonathan Hurley updated AMBARI-22444: ------------------------------------- Attachment: AMBARI-22444.patch > Add Native Libraries To Tez Tarball > ----------------------------------- > > Key: AMBARI-22444 > URL: https://issues.apache.org/jira/browse/AMBARI-22444 > Project: Ambari > Issue Type: Bug > Affects Versions: 2.6.1 > Reporter: Jonathan Hurley > Assignee: Jonathan Hurley > Priority: Blocker > Fix For: 2.6.1 > > Attachments: AMBARI-22444.patch > > > As part of the investigation for getting patch upgrades working in Ambari > 2.6.1, it was determined that the Tez tarball will need to have the hadoop > native libraries added to it so that they can be detected from the tarball. > STR: > - Install ZK, MapR, Tez, Yarn, Hive > - Enable a non-LZO codec, like Snappy > - Patch Hive to a new version > - Change the following properties in {{tez-site}}: > -- tez.am.launch.env = LD_LIBRARY=./tezlib/lib/native > -- tez.task.launch.env = LD_LIBRARY=./tezlib/lib/native > When Hive commands run, they will attempt to load the native snappy libraries > from the Tez tarball and will fail with: > {code} > Caused by: java.io.IOException: Unable to get CompressorType for codec > (org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to > missing native libraries for the codec. > at > org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.<init>(ExternalSorter.java:217) > {code} > In order to fix this, the Tez tarball should include the native hadoop > libraries as well: > {code} > ├── tez > │ ├── lib > │ │ ├── native > │ │ │ ├── libhadoop.a > │ │ │ ├── libhadoop.so -> libhadoop.so.1.0.0 > │ │ │ ├── libhadoop.so.1.0.0 > │ │ │ ├── libhadooppipes.a > │ │ │ ├── libhadooputils.a > │ │ │ ├── libhdfs.a > │ │ │ ├── libhdfs.so -> libhdfs.so.0.0.0 > │ │ │ ├── libhdfs.so.0.0.0 > │ │ │ ├── libsnappy.so -> libsnappy.so.1.1.4 > │ │ │ ├── libsnappy.so.1 -> libsnappy.so.1.1.4 > │ │ │ └── libsnappy.so.1.1.4 > {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)