-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63807/
-----------------------------------------------------------
Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
Bugs: AMBARI-22444
https://issues.apache.org/jira/browse/AMBARI-22444
Repository: ambari
Description
-------
As part of the investigation for getting patch upgrades working in Ambari
2.6.1, it was determined that the Tez tarball will need to have the hadoop
native libraries added to it so that they can be detected from the tarball.
STR:
- Install ZK, MapR, Tez, Yarn, Hive
- Enable a non-LZO codec, like Snappy
- Patch Hive to a new version
- Change the following properties in {{tez-site}}:
-- tez.am.launch.env = LD_LIBRARY=./tezlib/lib/native
-- tez.task.launch.env = LD_LIBRARY=./tezlib/lib/native
When Hive commands run, they will attempt to load the native snappy libraries
from the Tez tarball and will fail with:
{code}
Caused by: java.io.IOException: Unable to get CompressorType for codec
(org.apache.hadoop.io.compress.SnappyCodec). This is most likely due to missing
native libraries for the codec.
at
org.apache.tez.runtime.library.common.sort.impl.ExternalSorter.<init>(ExternalSorter.java:217)
{code}
In order to fix this, the Tez tarball should include the native hadoop
libraries as well:
{code}
??? tez
? ??? lib
? ? ??? native
? ? ? ??? libhadoop.a
? ? ? ??? libhadoop.so -> libhadoop.so.1.0.0
? ? ? ??? libhadoop.so.1.0.0
? ? ? ??? libhadooppipes.a
? ? ? ??? libhadooputils.a
? ? ? ??? libhdfs.a
? ? ? ??? libhdfs.so -> libhdfs.so.0.0.0
? ? ? ??? libhdfs.so.0.0.0
? ? ? ??? libsnappy.so -> libsnappy.so.1.1.4
? ? ? ??? libsnappy.so.1 -> libsnappy.so.1.1.4
? ? ? ??? libsnappy.so.1.1.4
{code}
Diffs
-----
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
03b6213ac4
ambari-common/src/main/python/resource_management/libraries/functions/tar_archive.py
c682c3e24a
Diff: https://reviews.apache.org/r/63807/diff/1/
Testing
-------
Manual testing to ensure the file is created and uploaded.
UNIT TESTS PENDING...
Thanks,
Jonathan Hurley
