[
https://issues.apache.org/jira/browse/AMBARI-22444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16255431#comment-16255431
]
Jonathan Hurley commented on AMBARI-22444:
------------------------------------------
Seems like this patch has issues with non-root and {{tarfile}}:
{code}
File
"/usr/lib/python2.6/site-packages/resource_management/libraries/functions/copy_tarball.py",
line 92, in _prepare_tez_tarball
with closing(tarfile.open(tez_tarball_with_native_lib, "w:gz")) as
new_tez_tarball:
File "/usr/lib64/python2.6/tarfile.py", line 1670, in open
return func(name, filemode, fileobj, **kwargs)
File "/usr/lib64/python2.6/tarfile.py", line 1714, in gzopen
fileobj = bltn_open(name, mode + "b")
IOError: [Errno 13] Permission denied:
'/var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz'
{code}
Attached [^AMBARI-22444-2.patch] to correct it
> Add Native Libraries To Tez Tarball
> -----------------------------------
>
> Key: AMBARI-22444
> URL: https://issues.apache.org/jira/browse/AMBARI-22444
> Project: Ambari
> Issue Type: Bug
> Affects Versions: 2.6.1
> Reporter: Jonathan Hurley
> Assignee: Jonathan Hurley
> Priority: Blocker
> Fix For: 2.6.1
>
> Attachments: AMBARI-22444-2.patch, AMBARI-22444.patch
>
>
> As part of the investigation for getting patch upgrades working in Ambari
> 2.6.1, it was determined that the Tez tarball will need to have the hadoop
> native libraries added to it so that they can be detected from the tarball.
> STR:
> - Install ZK, MapR, Tez, Yarn, Hive
> - Enable a non-LZO codec, like Snappy
> - Patch Hive to a new version
> - Change the following properties in {{tez-site}}:
> -- tez.am.launch.env = LD_LIBRARY=./tezlib/lib/native
> -- tez.task.launch.env = LD_LIBRARY=./tezlib/lib/native
> When Hive commands run, they will attempt to load the native snappy libraries
> from the Tez tarball and will fail with:
> {code}
> Caused by: java.io.IOException: Unable to get CompressorType for codec
> (org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to
> missing native libraries for the codec.
> at
> org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.<init>(ExternalSorter.java:217)
> {code}
> In order to fix this, the Tez tarball should include the native hadoop
> libraries as well:
> {code}
> ├── tez
> │ ├── lib
> │ │ ├── native
> │ │ │ ├── libhadoop.a
> │ │ │ ├── libhadoop.so -> libhadoop.so.1.0.0
> │ │ │ ├── libhadoop.so.1.0.0
> │ │ │ ├── libhadooppipes.a
> │ │ │ ├── libhadooputils.a
> │ │ │ ├── libhdfs.a
> │ │ │ ├── libhdfs.so -> libhdfs.so.0.0.0
> │ │ │ ├── libhdfs.so.0.0.0
> │ │ │ ├── libsnappy.so -> libsnappy.so.1.1.4
> │ │ │ ├── libsnappy.so.1 -> libsnappy.so.1.1.4
> │ │ │ └── libsnappy.so.1.1.4
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
