-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/51863/
-----------------------------------------------------------
Review request for Ambari, Andrew Onischuk, Laszlo Puskas, Sumit Mohanty, and
Venkat Ranganathan.
Bugs: AMBARI-18383
https://issues.apache.org/jira/browse/AMBARI-18383
Repository: ambari
Description
-------
Failed exception:
"Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.SparkMain],
main()
threw exception, Missing py4j and/or pyspark zip files. Please add them to the
lib folder
or to the Spark sharelib.
org.apache.oozie.action.hadoop.OozieActionConfiguratorException: Missing py4j
and/or
pyspark zip files. Please add them to the lib folder or to the Spark sharelib."
Those two files were not present in the share lib
hrt_qa@hn0-3ad6fe:~$ hdfs dfs -ls /user/oozie/share/lib/spark/*.zip
ls: `/user/oozie/share/lib/spark/*.zip': No such file or directory
Diffs
-----
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_service.py
8d80638
ambari-server/src/main/resources/scripts/Ambaripreupload.py 7c8c601
Diff: https://reviews.apache.org/r/51863/diff/
Testing
-------
Manually tested on sys-prepped cluster.
Execution log
2016-09-13 21:17:32,590 - Execute[('mv',
'/usr/hdp/2.5.1.0-16/oozie/share/lib/spark',
'/usr/hdp/2.5.1.0-16/oozie/share/lib/spark-orig')] {'sudo': True}
2016-09-13 21:17:32,613 -
Directory['/usr/hdp/2.5.1.0-16/oozie/share/lib/spark'] {'owner': 'oozie',
'create_parents': True}
2016-09-13 21:17:32,635 - Creating directory
Directory['/usr/hdp/2.5.1.0-16/oozie/share/lib/spark'] since it doesn't exist.
2016-09-13 21:17:32,746 - Changing owner for
/usr/hdp/2.5.1.0-16/oozie/share/lib/spark from 0 to oozie
2016-09-13 21:17:32,771 - Execute['cp -f
/usr/hdp/2.5.1.0-16/oozie/share/lib/spark-orig/oozie-sharelib-spark*.jar
/usr/hdp/2.5.1.0-16/oozie/share/lib/spark'] {'user': 'oozie'}
2016-09-13 21:17:32,804 - Execute['cp -P /usr/hdp/2.5.1.0-16/spark/lib/*.jar
/usr/hdp/2.5.1.0-16/oozie/share/lib/spark'] {'user': 'oozie'}
2016-09-13 21:17:33,125 - Execute['find
/usr/hdp/2.5.1.0-16/oozie/share/lib/spark/ -type l -delete'] {'user': 'oozie'}
2016-09-13 21:17:33,163 - Execute['rm -f
/usr/hdp/2.5.1.0-16/oozie/share/lib/spark/spark-examples*.jar'] {'user':
'oozie'}
2016-09-13 21:17:33,220 - Execute['cp -f
/usr/hdp/2.5.1.0-16/spark/python/lib/*.zip
/usr/hdp/2.5.1.0-16/oozie/share/lib/spark'] {'user': 'oozie'}
2016-09-13 21:17:33,258 - Execute['cp -f
/usr/hdp/2.5.1.0-16/spark/python/lib/*.jar
/usr/hdp/2.5.1.0-16/oozie/share/lib/spark'] {'user': 'oozie'}
No jar files found in Spark client python lib.
2016-09-13 21:17:33,293 - HdfsResource['/tmp/oozie/user/oozie/share']
{'security_enabled': False, 'hadoop_conf_dir': '/etc/hadoop/conf', 'keytab':
None, 'source': '/usr/hdp/2.5.1.0-16/oozie/share', 'default_fs':
'adl://oozie3hadl25.azuredatalakestore.net:443', 'user': 'hdfs',
'hdfs_resource_ignore_file':
'/var/lib/ambari-agent/data/.hdfs_resource_ignore', 'hdfs_site':
{'dfs.webhdfs.enabled': False}, 'kinit_path_local': None, 'principal_name':
None, 'recursive_chmod': True, 'owner': 'oozie', 'hadoop_bin_dir':
'/usr/hdp/2.5.1.0-16/hadoop/bin', 'type': 'directory', 'action':
['create_on_execute'], 'mode': 0755}
Thanks,
Aravindan Vijayan
