Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-20 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191492
---



Thanks for reviews

- Andrew Onischuk


On Nov. 20, 2017, 10:35 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 20, 2017, 10:35 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32266782437d6dad3f6c76de7e8106cd 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e82a08acf187999b95b02bca8b8fb54a 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60b40513e8d107d45ed2fbf4620ce9db 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f6476c52deab49e24028d10817c83af8ef 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616f0991862cd7e020e7e18f3b389883b 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004f934e00030e1b9d8952fd42d71f8e3 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b7966c044c4e5971bb400517f82aaa95c6 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f9ff4922da78a31661308804b517893 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d2260318420e869f4011d54117e8233a6cd9928 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36fbda06d86effb7c1d04edbc20a1a876 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54a03db3e4209edf5d2089eb86c14fd3 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e943dc47428c07c60c872580c141c5dbc 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f295627bba4e3b30328bb7f521cb9b3 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3101e646f265b5dca6e32bb179c9856 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-20 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/
---

(Updated Nov. 20, 2017, 10:35 a.m.)


Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
Papirkovskyy, Nate Cole, and Sid Wagle.


Bugs: AMBARI-22467
https://issues.apache.org/jira/browse/AMBARI-22467


Repository: ambari


Description
---

Changes Summary:
- Do not install lzo if GPL license if not accepted, instead give out 
logger.warning and skip
- Remove installation of lzo from metainfo.xml. Since we need to install lzo on 
restart if lzo is enabled/license accepted, we still need to do this in Python 
code. No need to duplicate.
Clients will also use that code during install.
- Move function get_lzo_packages to lzo_utils.py
- Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
lzo for HDFS,Oozie to use lzo_utils.py 

Notes:
- Impl is Patch Upgrade friendly as uses the same code to detect 
{stack_version} as all the other packages. That code (not implemented here) 
gets version depending component specific repo info not stack specific.


Diffs (updated)
-

  
ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
 f144b2da32266782437d6dad3f6c76de7e8106cd 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
 cfbb7d81e82a08acf187999b95b02bca8b8fb54a 
  
ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
 31e78b9e60b40513e8d107d45ed2fbf4620ce9db 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
2df1b9f6476c52deab49e24028d10817c83af8ef 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
 6f702d3616f0991862cd7e020e7e18f3b389883b 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
 fe488c3004f934e00030e1b9d8952fd42d71f8e3 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 5ca012b7966c044c4e5971bb400517f82aaa95c6 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 abbe59ed9f9ff4922da78a31661308804b517893 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
 7d2260318420e869f4011d54117e8233a6cd9928 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
 cd1f7ad36fbda06d86effb7c1d04edbc20a1a876 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
 f6e8391a54a03db3e4209edf5d2089eb86c14fd3 
  ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
07a9ef8e943dc47428c07c60c872580c141c5dbc 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
63e85e363f295627bba4e3b30328bb7f521cb9b3 
  ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
86ca03a4a3101e646f265b5dca6e32bb179c9856 


Diff: https://reviews.apache.org/r/63916/diff/2/

Changes: https://reviews.apache.org/r/63916/diff/1-2/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-20 Thread Andrew Onischuk 


> On Nov. 17, 2017, 4:25 p.m., Nate Cole wrote:
> > Please also verify if BigInsight stacks are affected by this change.  If 
> > that is the case, then a separate JIRA should be opened to address that.
> 
> Jonathan Hurley wrote:
> Great catch! Looks like it does:
> 
> https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HDFS/package/scripts/hdfs.py#L118-L119
> 
> So BI 4.0 and 4.2 will need it ... 4.2.5 extends common-services, so it 
> doesn't need it.
> 
> Also - don't forget to merge into 3.0 stack in trunk as well ... anything 
> Python changes in branch-2.6 need extra-work done when going into trunk.
> 
> Andrew Onischuk wrote:
> Sure @Jonathan.
> Do you guys think a separate jira should be fixVersion 3.0 or 2.6?

Created it for 3.0


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191346
---


On Nov. 17, 2017, 2:40 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 2:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-18 Thread Andrew Onischuk 


> On Nov. 17, 2017, 4:25 p.m., Nate Cole wrote:
> > Please also verify if BigInsight stacks are affected by this change.  If 
> > that is the case, then a separate JIRA should be opened to address that.
> 
> Jonathan Hurley wrote:
> Great catch! Looks like it does:
> 
> https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HDFS/package/scripts/hdfs.py#L118-L119
> 
> So BI 4.0 and 4.2 will need it ... 4.2.5 extends common-services, so it 
> doesn't need it.
> 
> Also - don't forget to merge into 3.0 stack in trunk as well ... anything 
> Python changes in branch-2.6 need extra-work done when going into trunk.

Sure @Jonathan.
Do you guys think a separate jira should be fixVersion 3.0 or 2.6?


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191346
---


On Nov. 17, 2017, 2:40 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 2:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-18 Thread Andrew Onischuk 


> On Nov. 17, 2017, 4:15 p.m., Nate Cole wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
> > Lines 82-83 (patched)
> > 
> >
> > Any key checks we need here?  Use default(...)?

These values are always present because they are generated by ambari-server. So 
seeing KeyError would be a good indicator that something went wrong. Instead of 
masking the problem with default values. Please let me know if you think 
differently


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191345
---


On Nov. 17, 2017, 2:40 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 2:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-18 Thread Andrew Onischuk 


> On Nov. 17, 2017, 4:15 p.m., Nate Cole wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
> > Lines 77-79 (patched)
> > 
> >
> > Is a zero-length possible?  Should log it if that's the case.

No it is not possible. I copied it from old code.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191345
---


On Nov. 17, 2017, 2:40 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 2:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-18 Thread Andrew Onischuk 


> On Nov. 17, 2017, 4:02 p.m., Jonathan Hurley wrote:
> > ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
> > Lines 66-69 (patched)
> > 
> >
> > Out of curiousity, do we think this should be a Fail?

As far as I know. Paul and Sid discussed this and decided that user should be 
able to manage lzo himself if he doesn't accept GPL.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191344
---


On Nov. 17, 2017, 2:40 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 2:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-17 Thread Jonathan Hurley 


> On Nov. 17, 2017, 11:25 a.m., Nate Cole wrote:
> > Please also verify if BigInsight stacks are affected by this change.  If 
> > that is the case, then a separate JIRA should be opened to address that.

Great catch! Looks like it does:
https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/resources/stacks/BigInsights/4.2/services/HDFS/package/scripts/hdfs.py#L118-L119

So BI 4.0 and 4.2 will need it ... 4.2.5 extends common-services, so it doesn't 
need it.

Also - don't forget to merge into 3.0 stack in trunk as well ... anything 
Python changes in branch-2.6 need extra-work done when going into trunk.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191346
---


On Nov. 17, 2017, 9:40 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 9:40 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-17 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191346
---



Please also verify if BigInsight stacks are affected by this change.  If that 
is the case, then a separate JIRA should be opened to address that.

- Nate Cole


On Nov. 17, 2017, 9:40 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 9:40 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-17 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191345
---



We should add python tests for enabled and disabled LZO for coverage.  If not 
here, then please make sure there's a JIRA for it.


ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 67-68 (patched)


"Cannot install LZO.  The GPL license must be explicitly enabled using 
'ambari-server setup' on the Ambari host, then restart the server and try 
again."



ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 77-79 (patched)


Is a zero-length possible?  Should log it if that's the case.



ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 82-83 (patched)


Any key checks we need here?  Use default(...)?



ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
Lines 308-311 (original), 309-310 (patched)


This will happen only for Oozie servers, not the client


- Nate Cole


On Nov. 17, 2017, 9:40 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 9:40 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-17 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191344
---


Fix it, then Ship it!





ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 44 (patched)


Use stack_features.get_stack_feature_version() instead of trying to get it 
directly.



ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 57 (patched)


Doc.



ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 66-69 (patched)


Out of curiousity, do we think this should be a Fail?



ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
Lines 73 (patched)


Doc.


- Jonathan Hurley


On Nov. 17, 2017, 9:40 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 9:40 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-17 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/#review191339
---



Added summary of changes to review description. Please read it for easier 
review.

- Andrew Onischuk


On Nov. 17, 2017, 2:40 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/63916/
> ---
> 
> (Updated Nov. 17, 2017, 2:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Jonathan Hurley, Myroslav 
> Papirkovskyy, Nate Cole, and Sid Wagle.
> 
> 
> Bugs: AMBARI-22467
> https://issues.apache.org/jira/browse/AMBARI-22467
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Changes Summary:
> - Do not install lzo if GPL license if not accepted, instead give out 
> logger.warning and skip
> - Remove installation of lzo from metainfo.xml. Since we need to install lzo 
> on restart if lzo is enabled/license accepted, we still need to do this in 
> Python code. No need to duplicate.
> Clients will also use that code during install.
> - Move function get_lzo_packages to lzo_utils.py
> - Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
> lzo for HDFS,Oozie to use lzo_utils.py 
> 
> Notes:
> - Impl is Patch Upgrade friendly as uses the same code to detect 
> {stack_version} as all the other packages. That code (not implemented here) 
> gets version depending component specific repo info not stack specific.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
>  f144b2da32 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
>  cfbb7d81e8 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
>  31e78b9e60 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
> 2df1b9f647 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
>  6f702d3616 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
>  fe488c3004 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  5ca012b796 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  abbe59ed9f 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
>  7d22603184 
>   
> ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
>  cd1f7ad36f 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
>  f6e8391a54 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
> 07a9ef8e94 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
> 63e85e363f 
>   ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
> 86ca03a4a3 
> 
> 
> Diff: https://reviews.apache.org/r/63916/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 63916: YARN, MapReduce2, Hive, and Oozie Should Conditionally Install LZO

2017-11-17 Thread Andrew Onischuk 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/63916/
---

Review request for Ambari, Attila Doroszlai, Myroslav Papirkovskyy, Nate Cole, 
and Sid Wagle.


Bugs: AMBARI-22467
https://issues.apache.org/jira/browse/AMBARI-22467


Repository: ambari


Description
---

Changes Summary:
- Do not install lzo if GPL license if not accepted, instead give out 
logger.warning and skip
- Remove installation of lzo from metainfo.xml. Since we need to install lzo on 
restart if lzo is enabled/license accepted, we still need to do this in Python 
code. No need to duplicate.
Clients will also use that code during install.
- Move function get_lzo_packages to lzo_utils.py
- Add code for Yarn, MR2, Hive to install lzo, edit existing code to install 
lzo for HDFS,Oozie to use lzo_utils.py 

Notes:
- Impl is Patch Upgrade friendly as uses the same code to detect 
{stack_version} as all the other packages. That code (not implemented here) 
gets version depending component specific repo info not stack specific.


Diffs
-

  
ambari-common/src/main/python/resource_management/libraries/functions/__init__.py
 f144b2da32 
  
ambari-common/src/main/python/resource_management/libraries/functions/get_lzo_packages.py
 cfbb7d81e8 
  
ambari-common/src/main/python/resource_management/libraries/functions/package_conditions.py
 31e78b9e60 
  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
2df1b9f647 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs.py
 6f702d3616 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/install_params.py
 fe488c3004 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
 5ca012b796 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 abbe59ed9f 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/oozie.py
 7d22603184 
  
ambari-server/src/main/resources/common-services/OOZIE/4.0.0.2.0/package/scripts/params_linux.py
 cd1f7ad36f 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
 f6e8391a54 
  ambari-server/src/main/resources/stacks/HDP/2.2/services/HDFS/metainfo.xml 
07a9ef8e94 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/HDFS/metainfo.xml 
63e85e363f 
  ambari-server/src/test/python/stacks/2.2/configs/oozie-upgrade.json 
86ca03a4a3 


Diff: https://reviews.apache.org/r/63916/diff/1/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk