-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/44835/
-----------------------------------------------------------

(Updated March 15, 2016, 6:32 a.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Juanjo  
Marron, Nate Cole, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-15420
    https://issues.apache.org/jira/browse/AMBARI-15420


Repository: ambari


Description (updated)
-------

Refactor resource_management library, to generalize the library and remove 
HDP-specific hardcodings

Specifically,
1. Make stack-selector, conf-selector tools stack-driven instead of hardcoding 
hdp-select and conf-select
2. Make stack-root stack-driven instead of hardcoding /usr/hdp
3. Make copy_tarball mappings stack-driven 
4. Make PACKAGE_DIRS mapping in conf_select use stack-root instead of 
hardcoding the "/usr/hdp"

In addition, also added a feature in the stack processing engine to support 
properties values to be defined a external property file (See tarball_map.json, 
stack_tools.json in patch)

Three config properties are added
1. cluster-env/stack_root
2. cluster-env/stack_tools
3. cluster-env/tarball_map
Corresponding helper functions get_stack_root(), get_stack_tool(), 
get_tarball_map() are added in script.py, which will set the defaults if these 
config properties are not defined (ambari-server upgrade scenario needs to be 
addressed to add these config properties on upgrade). These helper functions 
are used to remove hardcodings in resource_management library.


Remaining HDP-specific logic in resource_management library

1. conf_select::_valid()
2. conf_select::get_hadoop_conf_dir()
3. list_ambari_managed_repos::repository_names
4. version_select_util::get_component_version()
5. script::get_stack_version()
6. script::should_expose_component_version()
7. get_lzo_packages::get_lzo_packages()

Refactoring this HDP-specific logic would require "Stack Featurization" 
(AMBARI-13364) to be in place.


Diffs
-----

  ambari-agent/src/main/python/ambari_agent/HostCheckReportFileHandler.py 
ee7db0a 
  
ambari-common/src/main/python/resource_management/libraries/functions/conf_select.py
 59c717b 
  
ambari-common/src/main/python/resource_management/libraries/functions/copy_tarball.py
 647b8b6 
  
ambari-common/src/main/python/resource_management/libraries/functions/default.py
 23383dc 
  
ambari-common/src/main/python/resource_management/libraries/functions/dynamic_variable_interpretation.py
 a20b03c 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_stack_version.py
 f2e6567 
  
ambari-common/src/main/python/resource_management/libraries/functions/install_windows_msi.py
 f1cd9cb 
  
ambari-common/src/main/python/resource_management/libraries/functions/repo_version_history.py
 d585dea 
  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin.py
 4d9d8a4 
  
ambari-common/src/main/python/resource_management/libraries/functions/setup_ranger_plugin_xml.py
 2ccc0c6 
  
ambari-common/src/main/python/resource_management/libraries/functions/stack_select.py
 c94d956 
  
ambari-common/src/main/python/resource_management/libraries/functions/version_select_util.py
 95c5cba 
  ambari-common/src/main/python/resource_management/libraries/script/script.py 
a8098a0 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/AmbariMetaInfo.java
 4be4049 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/ConfigurationDirectory.java
 7f21aaa 
  ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
0c7faea 
  
ambari-server/src/main/java/org/apache/ambari/server/stack/StackDefinitionDirectory.java
 c739211 
  ambari-server/src/main/java/org/apache/ambari/server/stack/StackModule.java 
7d934bb 
  ambari-server/src/main/java/org/apache/ambari/server/state/PropertyInfo.java 
e7c9c27 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ValueAttributesInfo.java
 3f7f756 
  ambari-server/src/main/java/org/apache/ambari/server/utils/JsonUtils.java 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/accumulo_script.py
 12ca388 
  
ambari-server/src/main/resources/common-services/ACCUMULO/1.6.1.2.2.0/package/scripts/status_params.py
 45dbb24 
  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/atlas_client.py
 d000846 
  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/status_params.py
 2c06c40 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/status_params.py
 535c821 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat_client.py
 85e7012 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_upgrade.py
 c3d15e5 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
 c5e61e6 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/status_params.py
 d0924b9 
  
ambari-server/src/main/resources/common-services/KNOX/0.5.0.2.2/package/scripts/knox_gateway.py
 30b9a41 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
 81a227e 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server.py
 030fb2d 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie_server_upgrade.py
 27e2766 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/status_params.py
 954bb80 
  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
 e5b54cd 
  
ambari-server/src/main/resources/common-services/SPARK/1.2.0.2.2/package/scripts/params.py
 906b198 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/params_linux.py
 ec37243 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1.2.1/package/scripts/status_params.py
 984a4ba 
  
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_linux.py
 eb80ad6 
  
ambari-server/src/main/resources/common-services/TEZ/0.4.0.2.1/package/scripts/params_windows.py
 dd732f5 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 8b2aec5 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
 e05ed60 
  
ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5.2.0/package/scripts/status_params.py
 d18e4d7 
  ambari-server/src/main/resources/custom_actions/scripts/install_packages.py 
08bdcc3 
  ambari-server/src/main/resources/custom_actions/scripts/remove_bits.py 
e69a2e4 
  ambari-server/src/main/resources/custom_actions/scripts/ru_set_all.py b8bf176 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
3fb82e9 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/after-INSTALL/scripts/shared_initialization.py
 96dc104 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/hooks/before-INSTALL/scripts/shared_initialization.py
 07faae4 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_tools.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/tarball_map.json 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/stack/ServiceModuleTest.java
 92c1200 
  ambari-server/src/test/java/org/apache/ambari/server/utils/TestJsonUtils.java 
PRE-CREATION 
  ambari-server/src/test/python/custom_actions/test_ru_set_all.py 3090f6b 
  ambari-server/src/test/python/host_scripts/TestAlertDiskSpace.py 2608050 
  ambari-server/src/test/python/stacks/2.0.6/HDFS/test_hdfs_client.py b5b43d6 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_mapreduce2_client.py 
efe6038 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_yarn_client.py 4601092 
  
ambari-server/src/test/python/stacks/2.0.6/hooks/after-INSTALL/test_after_install.py
 daee726 
  ambari-server/src/test/python/stacks/2.1/TEZ/test_tez_client.py ab08776 

Diff: https://reviews.apache.org/r/44835/diff/


Testing
-------


Thanks,

Jayush Luniya

Reply via email to