[
https://issues.apache.org/jira/browse/AMBARI-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16254441#comment-16254441
]
Hudson commented on AMBARI-22444:
---------------------------------
SUCCESS: Integrated in Jenkins build Ambari-branch-2.6 #459 (See
[https://builds.apache.org/job/Ambari-branch-2.6/459/])
AMBARI-22444 - Add Native Libraries To Tez Tarball (part2) (jhurley:
[http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=a8496eedbf55310bc37e44d0012520ed0e014737])
* (edit)
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
> Add Native Libraries To Tez Tarball
> -----------------------------------
>
> Key: AMBARI-22444
> URL: https://issues.apache.org/jira/browse/AMBARI-22444
> Project: Ambari
> Issue Type: Bug
> Affects Versions: 2.6.1
> Reporter: Jonathan Hurley
> Assignee: Jonathan Hurley
> Priority: Blocker
> Fix For: 2.6.1
>
> Attachments: AMBARI-22444.patch
>
>
> As part of the investigation for getting patch upgrades working in Ambari
> 2.6.1, it was determined that the Tez tarball will need to have the hadoop
> native libraries added to it so that they can be detected from the tarball.
> STR:
> - Install ZK, MapR, Tez, Yarn, Hive
> - Enable a non-LZO codec, like Snappy
> - Patch Hive to a new version
> - Change the following properties in {{tez-site}}:
> -- tez.am.launch.env = LD_LIBRARY=./tezlib/lib/native
> -- tez.task.launch.env = LD_LIBRARY=./tezlib/lib/native
> When Hive commands run, they will attempt to load the native snappy libraries
> from the Tez tarball and will fail with:
> {code}
> Caused by: java.io.IOException: Unable to get CompressorType for codec
> (org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to
> missing native libraries for the codec.
> at
> org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.<init>(ExternalSorter.java:217)
> {code}
> In order to fix this, the Tez tarball should include the native hadoop
> libraries as well:
> {code}
> ├── tez
> │ ├── lib
> │ │ ├── native
> │ │ │ ├── libhadoop.a
> │ │ │ ├── libhadoop.so -> libhadoop.so.1.0.0
> │ │ │ ├── libhadoop.so.1.0.0
> │ │ │ ├── libhadooppipes.a
> │ │ │ ├── libhadooputils.a
> │ │ │ ├── libhdfs.a
> │ │ │ ├── libhdfs.so -> libhdfs.so.0.0.0
> │ │ │ ├── libhdfs.so.0.0.0
> │ │ │ ├── libsnappy.so -> libsnappy.so.1.1.4
> │ │ │ ├── libsnappy.so.1 -> libsnappy.so.1.1.4
> │ │ │ └── libsnappy.so.1.1.4
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
