Re: Review Request 50297: AMBARI-17788: Refactor spooler code in OutputHDFSFile to be reusable for OutputS3File

[Hemanth Yamijala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50297/
---

(Updated July 26, 2016, 5:55 a.m.)


Review request for Ambari, Hayat Behlim and Dharmesh Makwana.


Changes
---

Missed moving a log line with the earlier change. Just updated that.


Bugs: AMBARI-17788
https://issues.apache.org/jira/browse/AMBARI-17788


Repository: ambari


Description
---

This patch extracts spooling functions outside OutputHDFSFile so that it can be 
used for AMBARI-17785. The patch allows callbacks to check the condition on 
which spool files should be rolled over and what to do on rollover.


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java
 87cc0eb 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpooler.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerContext.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerException.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverCondition.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverHandler.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/50297/diff/


Testing
---

* New unit tests.
* Have tested by installing new RPM on an existing Ambari installation and 
configuring the output for one of the services to write to HDFS. Tested that 
upload to HDFS, rollover is working correctly for this service.


Thanks,

Hemanth Yamijala

Re: Review Request 50297: AMBARI-17788: Refactor spooler code in OutputHDFSFile to be reusable for OutputS3File

[Hemanth Yamijala]

> On July 25, 2016, 1:04 p.m., Hayat Behlim wrote:
> > ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpooler.java,
> >  line 126
> > 
> >
> > I think just for sake of understanding initializeSpoolFile() will come 
> > after handleRollover method is invoked.

Thanks for taking a look, Hayat. I agree with your comment. Have swapped the 
order.


- Hemanth


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50297/#review143372
---


On July 26, 2016, 5:46 a.m., Hemanth Yamijala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50297/
> ---
> 
> (Updated July 26, 2016, 5:46 a.m.)
> 
> 
> Review request for Ambari, Hayat Behlim and Dharmesh Makwana.
> 
> 
> Bugs: AMBARI-17788
> https://issues.apache.org/jira/browse/AMBARI-17788
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch extracts spooling functions outside OutputHDFSFile so that it can 
> be used for AMBARI-17785. The patch allows callbacks to check the condition 
> on which spool files should be rolled over and what to do on rollover.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java
>  87cc0eb 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpooler.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerContext.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerException.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverCondition.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverHandler.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50297/diff/
> 
> 
> Testing
> ---
> 
> * New unit tests.
> * Have tested by installing new RPM on an existing Ambari installation and 
> configuring the output for one of the services to write to HDFS. Tested that 
> upload to HDFS, rollover is working correctly for this service.
> 
> 
> Thanks,
> 
> Hemanth Yamijala
> 
>

Re: Review Request 50297: AMBARI-17788: Refactor spooler code in OutputHDFSFile to be reusable for OutputS3File

[Hemanth Yamijala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50297/
---

(Updated July 26, 2016, 5:46 a.m.)


Review request for Ambari, Hayat Behlim and Dharmesh Makwana.


Changes
---

Incorporated review comment from Hayat.


Bugs: AMBARI-17788
https://issues.apache.org/jira/browse/AMBARI-17788


Repository: ambari


Description
---

This patch extracts spooling functions outside OutputHDFSFile so that it can be 
used for AMBARI-17785. The patch allows callbacks to check the condition on 
which spool files should be rolled over and what to do on rollover.


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java
 87cc0eb 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpooler.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerContext.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerException.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverCondition.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverHandler.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/50297/diff/


Testing
---

* New unit tests.
* Have tested by installing new RPM on an existing Ambari installation and 
configuring the output for one of the services to write to HDFS. Tested that 
upload to HDFS, rollover is working correctly for this service.


Thanks,

Hemanth Yamijala

Re: Review Request 50431: AMBARI-17896: Fix falcon hook classname and classpath for atlas

[Venkat Ranganathan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50431/
---

(Updated July 25, 2016, 10:12 p.m.)


Review request for Ambari and Alejandro Fernandez.


Changes
---

Modified patch to not duplicate code for falcon advisor


Bugs: AMBARI-17896
https://issues.apache.org/jira/browse/AMBARI-17896


Repository: ambari


Description
---

Fixed stack advisor for 2.5 and also the param_linux for setting the right conf 
path


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 ebe3634 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
8d5cdc9 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
f8ba3e9 

Diff: https://reviews.apache.org/r/50431/diff/


Testing
---


Thanks,

Venkat Ranganathan

Re: Review Request 50407: Installing Components After Registering A New Repository Uses The Wrong Version

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50407/#review143471
---


Ship it!




Ship It!

- Jonathan Hurley


On July 25, 2016, 4:26 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50407/
> ---
> 
> (Updated July 25, 2016, 4:26 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17883
> https://issues.apache.org/jira/browse/AMBARI-17883
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 1. Install ZK only cluster with HDP-2.4
> 2. Install repo for HDP-2.5, and distribute bits
> 3. Do not upgrade, but then add HDFS.
> 
> The system will pick HDP-2.5 which will fail.  Fix is to check for 
> roleCommand == 'INSTALL' and the passed json 'version' to be present, which 
> is true after initial install.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/script/script.py 
> 5deeb52 
>   ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py 
> f5ed727 
> 
> Diff: https://reviews.apache.org/r/50407/diff/
> 
> 
> Testing
> ---
> 
> Manual testing of above STR, and newly added test:
> 
> --
> Total run:1033
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 50350: Config changes for Atlas in HDP 2.5 related to atlas.rest.address, atlas.cluster.name, etc

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143469
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 (line 139)


I don't see where `version_for_stack_feature_checks` is being initialized 
from.



ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
 (line 80)


It would be nice to extract these to a common location for constants of 
service names.


- Jonathan Hurley


On July 25, 2016, 9:22 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> ---
> 
> (Updated July 25, 2016, 9:22 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
> https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a384f69 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  f0a3f32 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  4305965 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  0590244 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  c53dd39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> ec14979 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  3a3879b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 8d5cdc9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  a1b93e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 96b1400 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> b2cc1c4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml

Re: Review Request 50350: Config changes for Atlas in HDP 2.5 related to atlas.rest.address, atlas.cluster.name, etc

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143467
---


Ship it!




Ship It!

- Swapan Shridhar


On July 26, 2016, 1:22 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> ---
> 
> (Updated July 26, 2016, 1:22 a.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
> https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a384f69 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  f0a3f32 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  4305965 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  0590244 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  c53dd39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> ec14979 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  3a3879b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 8d5cdc9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  a1b93e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 96b1400 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> b2cc1c4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  86e0964 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 2099958 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-atlas-application.properties.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> 88c2aae 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml
>  PRE-CREATION 
>   
> 

Review Request 50431: AMBARI-17896: Fix falcon hook classname and classpath for atlas

[Venkat Ranganathan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50431/
---

Review request for Ambari and Alejandro Fernandez.


Bugs: AMBARI-17896
https://issues.apache.org/jira/browse/AMBARI-17896


Repository: ambari


Description
---

Fixed stack advisor for 2.5 and also the param_linux for setting the right conf 
path


Diffs
-

  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/params_linux.py
 ebe3634 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
315020b 

Diff: https://reviews.apache.org/r/50431/diff/


Testing
---


Thanks,

Venkat Ranganathan

Re: Review Request 50350: Config changes for Atlas in HDP 2.5 related to atlas.rest.address, atlas.cluster.name, etc

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143462
---


Ship it!




Ship It!

- Sumit Mohanty


On July 26, 2016, 1:22 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> ---
> 
> (Updated July 26, 2016, 1:22 a.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
> https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a384f69 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  f0a3f32 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  4305965 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  0590244 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  c53dd39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> ec14979 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  3a3879b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 8d5cdc9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  a1b93e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 96b1400 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> b2cc1c4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  86e0964 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 2099958 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-atlas-application.properties.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> 88c2aae 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml
>  PRE-CREATION 
>   
> 

Re: Review Request 50350: Config changes for Atlas in HDP 2.5 related to atlas.rest.address, atlas.cluster.name, etc

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143461
---




ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 
217)


Any reason its in 2.3 as it seems to be only used in 2.5.


- Sumit Mohanty


On July 26, 2016, 1:22 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> ---
> 
> (Updated July 26, 2016, 1:22 a.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
> https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a384f69 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  f0a3f32 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  4305965 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  0590244 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  c53dd39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> ec14979 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  3a3879b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 8d5cdc9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  a1b93e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 96b1400 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> b2cc1c4 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  86e0964 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 2099958 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/configuration/hive-atlas-application.properties.xml
>  PRE-CREATION 
>   

Re: Review Request 50350: Config changes for Atlas in HDP 2.5 related to atlas.rest.address, atlas.cluster.name, etc

[Alejandro Fernandez]

> On July 26, 2016, 1:25 a.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml,
> >  line 28
> > 
> >
> > Delete this prop.

Marked this prop with  true since it should not be in 
sqoop-site.


> On July 26, 2016, 1:25 a.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml,
> >  line 118
> > 
> >
> > Deleted prop

Marked this prop with  true since it should not be in 
storm-site.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143457
---


On July 26, 2016, 1:22 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> ---
> 
> (Updated July 26, 2016, 1:22 a.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
> https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a384f69 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  f0a3f32 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  4305965 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  0590244 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  c53dd39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> ec14979 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  3a3879b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 8d5cdc9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  a1b93e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 96b1400 
>   

Re: Review Request 50350: Config changes for Atlas in HDP 2.5 related to atlas.rest.address, atlas.cluster.name, etc

[Alejandro Fernandez]

> On July 25, 2016, 11:53 p.m., Swapan Shridhar wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py,
> >  line 141
> > 
> >
> > Question: Do we need 'application_properties["atlas.rest.address"]' set 
> > here also ?

No, in earlier versions of ambari atlas.rest.address did not exist in the DB so 
the python code would construct the value on-the-fly and add it to 
application_properties dictionary so it would be written out to the config.
Today, that's not needed for HDP 2.5 since atlas.rest.address is now a proper 
config.


- Alejandro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143449
---


On July 26, 2016, 1:22 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> ---
> 
> (Updated July 26, 2016, 1:22 a.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
> https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a384f69 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  f0a3f32 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  4305965 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  0590244 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  c53dd39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> ec14979 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  3a3879b 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 8d5cdc9 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  a1b93e3 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 96b1400 
>   

Re: Review Request 50350: Config changes for Atlas in HDP 2.5 related to atlas.rest.address, atlas.cluster.name, etc

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143457
---




ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
 (line 97)


Needed to pass in service name to determine if it's a non-client service, 
which has slightly different configs. E.g., Sqoop.



ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 (line 140)


This value now comes from a config instead of being constructed on-the-fly.



ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 


This file was moved to HDP 2.5 stack.



ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
 (line 118)


Deleted prop



ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/configuration/sqoop-site.xml
 (line 28)


Delete this prop.



ambari-server/src/main/resources/stacks/HDP/2.5/services/SQOOP/kerberos.json 
(line 7)


When Sqoop is kerberized, add these props.


- Alejandro Fernandez


On July 26, 2016, 1:22 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> ---
> 
> (Updated July 26, 2016, 1:22 a.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
> https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  a384f69 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  f0a3f32 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  4305965 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-atlas-application.properties.xml
>  0590244 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  c53dd39 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/metainfo.xml 
> ec14979 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> 

Re: Review Request 49387: Support password type for custom properties

[Keta Patel]

> On July 25, 2016, 9:08 p.m., Alexandr Antonenko wrote:
> > Ship It!

Thank you Alexandr!
Could you please help with pushing in the changes?


- Keta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/#review143428
---


On July 25, 2016, 8:47 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49387/
> ---
> 
> (Updated July 25, 2016, 8:47 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17041
> https://issues.apache.org/jira/browse/AMBARI-17041
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, services can define properties in the XML configuration files that 
> is flagged as type password:
>   
> my.special.password
> 
> PASSWORD
> Password to be masked
>  
> and it will be masked properly in the UI as well as blueprint.
> 
> Custom property should also support this option so that password can be added 
> as custom property and treat accordingly.
> 
> 
> ==
> Proposed Design for the fix:
> ==
> At present only the key-value information of the service properties is stored 
> in the DB ("clusterconfig" table in the "config_data" column). 
> The "config_attributes" column stores only certain attributes like "final" 
> indicating the list of properties set with the Final flag = true. 
> The information about the property-type (i.e PASSWORD, USER, GROUP, 
> ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, 
> etc) is extracted from the corresponding service's property file (e.g. 
> hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
> information of the existing properties only. Custom Properties added by 
> ambari user have no provision to store their additional attributes. 
> 
> Since, for this Jira we are concerned with only  attribute for 
> Custom Properties, we could add an additional field called "Property Type" in 
> the "Add Property" pop-up which shows up on clicking "Add Property ..." in 
> the Custom property section for a service. For now, only 2 options are shown 
> in the drop-down list: NONE and PASSWORD .
> A few sample test properties are created using the new "Add Property" pop-up 
> as can be seen in the following attachments. 
> Attachments: 
> "add_property_pop_up.tiff"
> "custom_property_password_type.tiff"
> "custom_property_regular_type.tiff"
> "custom_properties_after_save.tiff"
> 
> The  information for these Custom properties is stored in the 
> DB in "clusterconfig" table, "config_attributes" column.
> The schema for "clusterconfig" table can be seen in the attachment:
> "schema_of_clusterconfig_table.tiff"
> 
> The content of the "config_attributes" column with the  
> information from the new Custom properties can be seen in the attachment:
> "cluster_config_with_password_type_in_config_attributes_column.tiff"
> 
> 
> Note: The fix so far is performed only for new Custom properties. The 
>  information for existing properties is extracted from the 
> corresponding property xml files for the service.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  aba45bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationResponse.java
>  eef3474 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/SecretReference.java
>  84f3109 
>   ambari-web/app/messages.js 24c9cfa 
>   ambari-web/app/mixins/common/configs/configs_saver.js 43a77ce 
>   ambari-web/app/models/configs/objects/service_config_property.js 844806f 
>   ambari-web/app/templates/common/configs/addPropertyWindow.hbs 659435c 
>   ambari-web/app/utils/config.js 9bb977a 
>   ambari-web/app/views/common/configs/service_configs_by_category_view.js 
> 964e193 
>   ambari-web/test/mixins/common/configs/configs_saver_test.js 4baf303 
> 
> Diff: https://reviews.apache.org/r/49387/diff/
> 
> 
> Testing
> ---
> 
> No new test cases are written in the patch apart from the fix required in 
> "configs_saver_test.js" to avoid test failure of existing tests.
> If the proposed solution is acceptable, I will create a new patch with the 
> necessary new tests.
> 
> The existing ambari-web tests after applying the patch:
> 
>   28977 tests complete (37 seconds)
>   154 tests pending
>   
>   
> With the patch "AMBARI-17041-trunk-July08.patch" the following is result of 
> ambari-web tests:
> 
>   29017 tests complete (48 seconds)
>   154 tests pending
>   
> With the patch 

Re: Review Request 50350: PREVIEW. Config changes for Atlas in HDP 2.5 related to atlas.rest.address, atlas.cluster.name, etc

[Swapan Shridhar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50350/#review143449
---




ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
 (line 140)


Question: Do we need 'application_properties["atlas.rest.address"]' set 
here also ?



ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
 (line 234)


Whitespace.



ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 
229)


Removal required.



ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py (line 
242)


Removal required.


- Swapan Shridhar


On July 22, 2016, 11:31 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50350/
> ---
> 
> (Updated July 22, 2016, 11:31 p.m.)
> 
> 
> Review request for Ambari, Di Li, Jonathan Hurley, Jayush Luniya, Madhan 
> Neethiraj, Nate Cole, Sumit Mohanty, Swapan Shridhar, Suma Shivaprasad, Sid 
> Wagle, and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17782
> https://issues.apache.org/jira/browse/AMBARI-17782
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> To support Atlas in HDP 2.5, make several config changes:
> 
> /etc/hive/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/hive/conf/hive-site.xml:
> 
> * atlas.cluster.name (remove this property)
> * atlas.hook.hive.maxThreads (remove this property)
> * atlas.hook.hive.minThreads (remove this property)
> * atlas.rest.address (remove this property)
> 
> /etc/storm/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
>  
> /etc/storm/conf/storm.yaml:
> 
> * atlas.cluster.name (remove this property)
>  
> /etc/falcon/conf/atlas-application.properties:
> 
> * atlas.rest.address (add this property)
> * atlas.cluster.name (this value is empty; need to set to correct value)
>  
>  
> /etc/sqoop/conf/atlas-application.properties:
> 
> * atlas.jaas.KafkaClient.option.keyTab (remove this property)
> * atlas.jaas.KafkaClient.option.principal (remove this property)
> * atlas.jaas.KafkaClient.option.storeKey (remove this property)
> * atlas.jaas.KafkaClient.option.useKeyTab (remove this property)
> * atlas.jaas.KafkaClient.option.useTicketCache=true (add this property)
> * atlas.jaas.KafkaClient.option.renewTicket=true (add this property)
> * atlas.rest.address (add this property)
>  
> /etc/sqoop/conf/sqoop-site.xml:
> 
> * atlas.cluster.name (remove this property)
> 
> Also, there is no 'Custom sqoop-atlas-application.properties' section in Sqoop
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  947d441 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  939e232 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/params.py
>  e4d2392 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  c3bcbc3 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  6111c34 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hcat.py
>  f53625c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive.py
>  6b05134 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat.py
>  4f38055 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  d78c5be 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/package/scripts/sqoop.py
>  963d169 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  d765ca3 
>   
> ambari-server/src/main/resources/common-services/STORM/1.0.1/configuration/storm-site.xml
>  d4ca263 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 6081a04 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> bfdb3d3 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.5.xml
>  7d96155 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> 09007a6 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> bd1bef2 
>   
> 

Re: Review Request 50047: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

[Robert Levas]

> On July 25, 2016, 6:21 p.m., Robert Levas wrote:
> > Ship It!

Thanks for making those changes.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50047/#review143440
---


On July 25, 2016, 5:12 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50047/
> ---
> 
> (Updated July 25, 2016, 5:12 p.m.)
> 
> 
> Review request for Ambari, Di Li and Robert Levas.
> 
> 
> Bugs: AMBARI-17694
> https://issues.apache.org/jira/browse/AMBARI-17694
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When kerberos is enabled, the protocol for listeners in 
> /etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
> even though the Ambari UI shows otherwise
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  66be3bf 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
>  ac7b0ae 
>   ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
> 2b1c01b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
>  ee2a671 
> 
> Diff: https://reviews.apache.org/r/50047/diff/
> 
> 
> Testing
> ---
> 
> Added 1 new test case,
>  Ran mvn test
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 50047: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50047/#review143440
---


Ship it!




Ship It!

- Robert Levas


On July 25, 2016, 5:12 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50047/
> ---
> 
> (Updated July 25, 2016, 5:12 p.m.)
> 
> 
> Review request for Ambari, Di Li and Robert Levas.
> 
> 
> Bugs: AMBARI-17694
> https://issues.apache.org/jira/browse/AMBARI-17694
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When kerberos is enabled, the protocol for listeners in 
> /etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
> even though the Ambari UI shows otherwise
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  66be3bf 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
>  ac7b0ae 
>   ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
> 2b1c01b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
>  ee2a671 
> 
> Diff: https://reviews.apache.org/r/50047/diff/
> 
> 
> Testing
> ---
> 
> Added 1 new test case,
>  Ran mvn test
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 50377: Update Atlas Metadata server security status for the new version

[Sumit Mohanty]

> On July 25, 2016, 6:09 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py,
> >  line 161
> > 
> >
> > small nitpick here, can remove outside parentheses.

Well, looks like "()" is used here to avoid having to use line breaks "\". So 
leaving it unchanged as most of the other security_status() calls follow this 
pattern.


- Sumit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50377/#review143403
---


On July 24, 2016, 3:31 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50377/
> ---
> 
> (Updated July 24, 2016, 3:31 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Robert Levas.
> 
> 
> Bugs: AMBARI-17869
> https://issues.apache.org/jira/browse/AMBARI-17869
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add support for new properties to detect security status
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  f18809f 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 3797cf3 
> 
> Diff: https://reviews.apache.org/r/50377/diff/
> 
> 
> Testing
> ---
> 
> Manually ran UT and tested on a kerberized cluster. UTs pass.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 50047: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

[Anita Jebaraj]

> On July 21, 2016, 6:30 p.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java,
> >  line 242
> > 
> >
> > Why is 'SASL' hardcoded here?
> 
> Anita Jebaraj wrote:
> When kerberos is enabled, the listeners protocol can be PLAINTEXTSASL or 
> SASL_PLAINTEXT, in HDP PLAINTEXTSASL is used throughout, but the user can 
> still customize the protocol as SASL_PLAINTEXT, so I thought checking for the 
> text SASL would be a better approach to avoid this issue or I can add the 
> value "SASL" as third argument to replace() function.
> 
> Robert Levas wrote:
> In this case, I think we are better off if the replace function took in a 
> regular expression...  Then you could use something like
> 
> ```
> "listeners": "${kafka-broker/listeners|replace(\PLAINTEXT\b, 
> PLAINTEXTSASL)}"
> ```
> 
> The code for the replace function could be something like:
> 
> ```
> public String perform(String[] args, String data) {
>   if ((args == null) || (args.length != 2)) {
> throw new IllegalArgumentException("Invalid number of arguments 
> encountered");
>   }
> 
>   if (data != null) {
> StringBuffer builder = new StringBuffer();
> 
> String regex = args[0];
> String replacement = args[1];
> 
> Pattern pattern = Pattern.compile(regex);
> Matcher matcher = pattern.matcher(data);
> 
> while(matcher.find()) {
>   matcher.appendReplacement(builder, replacement);
> }
> matcher.appendTail(builder);
> 
> return builder.toString();
>   }
> 
>   return "";
> }
> 
> ```
> 
> NOTE: I used the above code to translate `SASL_PLAINTEXT://host1, 
> PLAINTEXT://host2` to `SASL_PLAINTEXT://host1, PLAINTEXTSASL://host2`
> 
> Robert Levas wrote:
> From the above comment:
> 
> ```
> "listeners": "${kafka-broker/listeners|replace(\PLAINTEXT\b, 
> PLAINTEXTSASL)}"
> ```
> 
> Should have been 
> 
> ```
> "listeners": "${kafka-broker/listeners|replace(\bPLAINTEXT\b, 
> PLAINTEXTSASL)}"
> ```

Thank you Robert, I have updated the patch based on your comments


- Anita


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50047/#review143119
---


On July 25, 2016, 9:12 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50047/
> ---
> 
> (Updated July 25, 2016, 9:12 p.m.)
> 
> 
> Review request for Ambari, Di Li and Robert Levas.
> 
> 
> Bugs: AMBARI-17694
> https://issues.apache.org/jira/browse/AMBARI-17694
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When kerberos is enabled, the protocol for listeners in 
> /etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
> even though the Ambari UI shows otherwise
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
>  66be3bf 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
>  ac7b0ae 
>   ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
> 2b1c01b 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
>  ee2a671 
> 
> Diff: https://reviews.apache.org/r/50047/diff/
> 
> 
> Testing
> ---
> 
> Added 1 new test case,
>  Ran mvn test
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 50047: Kafka listeners property does not show SASL_PLAINTEXT protocol when Kerberos is enabled

[Anita Jebaraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50047/
---

(Updated July 25, 2016, 9:12 p.m.)


Review request for Ambari, Di Li and Robert Levas.


Bugs: AMBARI-17694
https://issues.apache.org/jira/browse/AMBARI-17694


Repository: ambari


Description
---

When kerberos is enabled, the protocol for listeners in 
/etc/kafka/conf/server.properties is updated from PLAINTEXT to PLAINTEXTSASL, 
even though the Ambari UI shows otherwise


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelper.java
 66be3bf 
  
ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/kafka.py
 ac7b0ae 
  ambari-server/src/main/resources/common-services/KAFKA/0.9.0/kerberos.json 
2b1c01b 
  
ambari-server/src/test/java/org/apache/ambari/server/state/kerberos/VariableReplacementHelperTest.java
 ee2a671 

Diff: https://reviews.apache.org/r/50047/diff/


Testing
---

Added 1 new test case,
 Ran mvn test


Thanks,

Anita Jebaraj

Re: Review Request 49387: Support password type for custom properties

[Alexandr Antonenko]

> On July 20, 2016, 11:17 p.m., Alexandr Antonenko wrote:
> > ambari-web/app/utils/config.js, line 264
> > 
> >
> > I do not see FINAL populated anywhere. I think this code will trigger 
> > fatal error. Have you tested this part manually ? 
> > 
> > Also, there is no need to check attribute[0], you just created it, and 
> > anyway there will be one and more values in that array
> > 
> > In code : attributes[0][FINAL][index] && attributes[0][FINAL][index] 
> > === "true"
> > 
> > You can skip cheking attributes[0][FINAL][index], just check 
> > attributes[0][FINAL][index] === "true".
> > 
> > If property "attributes[0][FINAL][index]" does not exist 
> > attributes[0][FINAL][index] === "true" will not trigger any error
> 
> Keta Patel wrote:
> The updated patch "AMBARI-17041-July21-updated.patch" conatins this 
> correction. I have kept the original statement with the name of the variable 
> updated to "attributes[0][FINAL]".
> 
> Thank you!
> 
> Alexandr Antonenko wrote:
> You are using your newly created variable FINAL only one time in 
> attributes[0][FINAL][index]. Why not simply use 
> attributes[0]["FINAL"][index], passing needed string directly, without 
> creating variable FINAL. especially when you have file with tons of variables 
> and functions which names have FINAL in it.
> 
> Keta Patel wrote:
> Hello Alexandr,
> I have updated the latest patch "AMBARI-17041-July22.patch" by removing 
> the "FINAL" variable. I have still kept the variable "PASSWORD" as it is used 
> in 2 functions (just once in both functions though).
> 
> Thank you!

Good job


- Alexandr


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/#review143024
---


On July 25, 2016, 8:47 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49387/
> ---
> 
> (Updated July 25, 2016, 8:47 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17041
> https://issues.apache.org/jira/browse/AMBARI-17041
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, services can define properties in the XML configuration files that 
> is flagged as type password:
>   
> my.special.password
> 
> PASSWORD
> Password to be masked
>  
> and it will be masked properly in the UI as well as blueprint.
> 
> Custom property should also support this option so that password can be added 
> as custom property and treat accordingly.
> 
> 
> ==
> Proposed Design for the fix:
> ==
> At present only the key-value information of the service properties is stored 
> in the DB ("clusterconfig" table in the "config_data" column). 
> The "config_attributes" column stores only certain attributes like "final" 
> indicating the list of properties set with the Final flag = true. 
> The information about the property-type (i.e PASSWORD, USER, GROUP, 
> ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, 
> etc) is extracted from the corresponding service's property file (e.g. 
> hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
> information of the existing properties only. Custom Properties added by 
> ambari user have no provision to store their additional attributes. 
> 
> Since, for this Jira we are concerned with only  attribute for 
> Custom Properties, we could add an additional field called "Property Type" in 
> the "Add Property" pop-up which shows up on clicking "Add Property ..." in 
> the Custom property section for a service. For now, only 2 options are shown 
> in the drop-down list: NONE and PASSWORD .
> A few sample test properties are created using the new "Add Property" pop-up 
> as can be seen in the following attachments. 
> Attachments: 
> "add_property_pop_up.tiff"
> "custom_property_password_type.tiff"
> "custom_property_regular_type.tiff"
> "custom_properties_after_save.tiff"
> 
> The  information for these Custom properties is stored in the 
> DB in "clusterconfig" table, "config_attributes" column.
> The schema for "clusterconfig" table can be seen in the attachment:
> "schema_of_clusterconfig_table.tiff"
> 
> The content of the "config_attributes" column with the  
> information from the new Custom properties can be seen in the attachment:
> "cluster_config_with_password_type_in_config_attributes_column.tiff"
> 
> 
> Note: The fix so far is performed only for new Custom properties. The 
>  information 

Re: Review Request 49387: Support password type for custom properties

[Alexandr Antonenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/#review143428
---


Ship it!




Ship It!

- Alexandr Antonenko


On July 25, 2016, 8:47 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49387/
> ---
> 
> (Updated July 25, 2016, 8:47 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17041
> https://issues.apache.org/jira/browse/AMBARI-17041
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, services can define properties in the XML configuration files that 
> is flagged as type password:
>   
> my.special.password
> 
> PASSWORD
> Password to be masked
>  
> and it will be masked properly in the UI as well as blueprint.
> 
> Custom property should also support this option so that password can be added 
> as custom property and treat accordingly.
> 
> 
> ==
> Proposed Design for the fix:
> ==
> At present only the key-value information of the service properties is stored 
> in the DB ("clusterconfig" table in the "config_data" column). 
> The "config_attributes" column stores only certain attributes like "final" 
> indicating the list of properties set with the Final flag = true. 
> The information about the property-type (i.e PASSWORD, USER, GROUP, 
> ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, 
> etc) is extracted from the corresponding service's property file (e.g. 
> hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
> information of the existing properties only. Custom Properties added by 
> ambari user have no provision to store their additional attributes. 
> 
> Since, for this Jira we are concerned with only  attribute for 
> Custom Properties, we could add an additional field called "Property Type" in 
> the "Add Property" pop-up which shows up on clicking "Add Property ..." in 
> the Custom property section for a service. For now, only 2 options are shown 
> in the drop-down list: NONE and PASSWORD .
> A few sample test properties are created using the new "Add Property" pop-up 
> as can be seen in the following attachments. 
> Attachments: 
> "add_property_pop_up.tiff"
> "custom_property_password_type.tiff"
> "custom_property_regular_type.tiff"
> "custom_properties_after_save.tiff"
> 
> The  information for these Custom properties is stored in the 
> DB in "clusterconfig" table, "config_attributes" column.
> The schema for "clusterconfig" table can be seen in the attachment:
> "schema_of_clusterconfig_table.tiff"
> 
> The content of the "config_attributes" column with the  
> information from the new Custom properties can be seen in the attachment:
> "cluster_config_with_password_type_in_config_attributes_column.tiff"
> 
> 
> Note: The fix so far is performed only for new Custom properties. The 
>  information for existing properties is extracted from the 
> corresponding property xml files for the service.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  aba45bf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationResponse.java
>  eef3474 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/SecretReference.java
>  84f3109 
>   ambari-web/app/messages.js 24c9cfa 
>   ambari-web/app/mixins/common/configs/configs_saver.js 43a77ce 
>   ambari-web/app/models/configs/objects/service_config_property.js 844806f 
>   ambari-web/app/templates/common/configs/addPropertyWindow.hbs 659435c 
>   ambari-web/app/utils/config.js 9bb977a 
>   ambari-web/app/views/common/configs/service_configs_by_category_view.js 
> 964e193 
>   ambari-web/test/mixins/common/configs/configs_saver_test.js 4baf303 
> 
> Diff: https://reviews.apache.org/r/49387/diff/
> 
> 
> Testing
> ---
> 
> No new test cases are written in the patch apart from the fix required in 
> "configs_saver_test.js" to avoid test failure of existing tests.
> If the proposed solution is acceptable, I will create a new patch with the 
> necessary new tests.
> 
> The existing ambari-web tests after applying the patch:
> 
>   28977 tests complete (37 seconds)
>   154 tests pending
>   
>   
> With the patch "AMBARI-17041-trunk-July08.patch" the following is result of 
> ambari-web tests:
> 
>   29017 tests complete (48 seconds)
>   154 tests pending
>   
> With the patch "AMBARI-17041-trunk-July13.patch" the following is the result 
> of ambari-web tests:
> 
>   29021 tests complete (26 seconds)
>   154 

Re: Review Request 50409: Adding Services After A Stack/Ambari Upgrade Shows Empty Required Values

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50409/#review143427
---




ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 (lines 1247 - 1271)


This is the step we were missing - iterate over all "new" configurations 
and see if the property existed in the current stack's default configs as well.

If it existed in the current stack's _default_ configs but doesn't exist in 
the current configs, then it has no business being added back.


- Jonathan Hurley


On July 25, 2016, 5:04 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50409/
> ---
> 
> (Updated July 25, 2016, 5:04 p.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.
> 
> 
> Bugs: AMBARI-17894
> https://issues.apache.org/jira/browse/AMBARI-17894
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> - Install Ambari 2.2.2 with Hive on HDP 2.4.2.0 and Kerberize
> - Upgrade to Ambari 2.4.0
> - Upgrade to HDP 2.5
> - Distribute Keytabs
> - Add a new service
> 
> At this point, the UI flags Hive as having configurations which need 
> attention. The follow are all blank and are marked as required:
> 
> {code}
> hive-site/hive.server2.authentication.ldap.url
> hiveserver2-site/hive.conf.restricted.list
> hiveserver2-site/hive.security.authenticator.manager
> hiveserver2-site/hive.security.authorization.manager
> {code}
> 
> There are actually two problems here:
> 
> - The Kerberos wizard interprets the stack advisor "delete" attribute and 
> improperly sets config properties to blank instead of actually removing them
> 
> - The upgrade logic is adding properties back when it should not.
> 
> The Kerberos issue aside, we can't be adding properties back during upgrade 
> stack merging when those properties were specifically removed by the stack 
> advisor prior.
> 
> For example (Ambari 2.2.2 installed with HDP 2.x)
> - Ambari 2.2.2 does not have {{foo-site/property}} for HDP 2.x
> - Ambari 2.4.0 adds {{foo-site/property}} for HDP 2.x, but instructs the 
> upgrade not to add it
> - An upgrade to HDP 2.y sees that {{foo-site/property}} doesn't exist and 
> thinks it's brand new and needs to be merged.
> 
> The upgrade logic should check to see if {{foo-site/property}} is part of 
> both 2.x and 2.y default configurations. If it's part of both of them and is 
> not currently set, then upgrade should NOT set it.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
>  55e9dbd 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hiveserver2-site.xml
>  ad37114 
> 
> Diff: https://reviews.apache.org/r/50409/diff/
> 
> 
> Testing
> ---
> 
> PENDING UNIT TESTS
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Review Request 50409: Adding Services After A Stack/Ambari Upgrade Shows Empty Required Values

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50409/
---

Review request for Ambari, Dmytro Grinenko, Dmitro Lisnichenko, and Nate Cole.


Bugs: AMBARI-17894
https://issues.apache.org/jira/browse/AMBARI-17894


Repository: ambari


Description
---

STR:

- Install Ambari 2.2.2 with Hive on HDP 2.4.2.0 and Kerberize
- Upgrade to Ambari 2.4.0
- Upgrade to HDP 2.5
- Distribute Keytabs
- Add a new service

At this point, the UI flags Hive as having configurations which need attention. 
The follow are all blank and are marked as required:

{code}
hive-site/hive.server2.authentication.ldap.url
hiveserver2-site/hive.conf.restricted.list
hiveserver2-site/hive.security.authenticator.manager
hiveserver2-site/hive.security.authorization.manager
{code}

There are actually two problems here:

- The Kerberos wizard interprets the stack advisor "delete" attribute and 
improperly sets config properties to blank instead of actually removing them

- The upgrade logic is adding properties back when it should not.

The Kerberos issue aside, we can't be adding properties back during upgrade 
stack merging when those properties were specifically removed by the stack 
advisor prior.

For example (Ambari 2.2.2 installed with HDP 2.x)
- Ambari 2.2.2 does not have {{foo-site/property}} for HDP 2.x
- Ambari 2.4.0 adds {{foo-site/property}} for HDP 2.x, but instructs the 
upgrade not to add it
- An upgrade to HDP 2.y sees that {{foo-site/property}} doesn't exist and 
thinks it's brand new and needs to be merged.

The upgrade logic should check to see if {{foo-site/property}} is part of both 
2.x and 2.y default configurations. If it's part of both of them and is not 
currently set, then upgrade should NOT set it.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UpgradeResourceProvider.java
 55e9dbd 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hiveserver2-site.xml
 ad37114 

Diff: https://reviews.apache.org/r/50409/diff/


Testing
---

PENDING UNIT TESTS


Thanks,

Jonathan Hurley

Re: Review Request 49387: Support password type for custom properties

[Keta Patel]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/
---

(Updated July 25, 2016, 8:47 p.m.)


Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
Jonathan Hurley.


Bugs: AMBARI-17041
https://issues.apache.org/jira/browse/AMBARI-17041


Repository: ambari


Description
---

Currently, services can define properties in the XML configuration files that 
is flagged as type password:
  
my.special.password

PASSWORD
Password to be masked
 
and it will be masked properly in the UI as well as blueprint.

Custom property should also support this option so that password can be added 
as custom property and treat accordingly.


==
Proposed Design for the fix:
==
At present only the key-value information of the service properties is stored 
in the DB ("clusterconfig" table in the "config_data" column). 
The "config_attributes" column stores only certain attributes like "final" 
indicating the list of properties set with the Final flag = true. 
The information about the property-type (i.e PASSWORD, USER, GROUP, 
ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, etc) 
is extracted from the corresponding service's property file (e.g. 
hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
information of the existing properties only. Custom Properties added by ambari 
user have no provision to store their additional attributes. 

Since, for this Jira we are concerned with only  attribute for 
Custom Properties, we could add an additional field called "Property Type" in 
the "Add Property" pop-up which shows up on clicking "Add Property ..." in the 
Custom property section for a service. For now, only 2 options are shown in the 
drop-down list: NONE and PASSWORD .
A few sample test properties are created using the new "Add Property" pop-up as 
can be seen in the following attachments. 
Attachments: 
"add_property_pop_up.tiff"
"custom_property_password_type.tiff"
"custom_property_regular_type.tiff"
"custom_properties_after_save.tiff"

The  information for these Custom properties is stored in the DB 
in "clusterconfig" table, "config_attributes" column.
The schema for "clusterconfig" table can be seen in the attachment:
"schema_of_clusterconfig_table.tiff"

The content of the "config_attributes" column with the  
information from the new Custom properties can be seen in the attachment:
"cluster_config_with_password_type_in_config_attributes_column.tiff"


Note: The fix so far is performed only for new Custom properties. The 
 information for existing properties is extracted from the 
corresponding property xml files for the service.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 aba45bf 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationResponse.java
 eef3474 
  
ambari-server/src/main/java/org/apache/ambari/server/utils/SecretReference.java 
84f3109 
  ambari-web/app/messages.js 24c9cfa 
  ambari-web/app/mixins/common/configs/configs_saver.js 43a77ce 
  ambari-web/app/models/configs/objects/service_config_property.js 844806f 
  ambari-web/app/templates/common/configs/addPropertyWindow.hbs 659435c 
  ambari-web/app/utils/config.js 9bb977a 
  ambari-web/app/views/common/configs/service_configs_by_category_view.js 
964e193 
  ambari-web/test/mixins/common/configs/configs_saver_test.js 4baf303 

Diff: https://reviews.apache.org/r/49387/diff/


Testing (updated)
---

No new test cases are written in the patch apart from the fix required in 
"configs_saver_test.js" to avoid test failure of existing tests.
If the proposed solution is acceptable, I will create a new patch with the 
necessary new tests.

The existing ambari-web tests after applying the patch:

  28977 tests complete (37 seconds)
  154 tests pending
  
  
With the patch "AMBARI-17041-trunk-July08.patch" the following is result of 
ambari-web tests:

  29017 tests complete (48 seconds)
  154 tests pending
  
With the patch "AMBARI-17041-trunk-July13.patch" the following is the result of 
ambari-web tests:

  29021 tests complete (26 seconds)
  154 tests pending
  
With the patch "AMBARI-17041-trunk-July20.patch" the following is the result of 
ambari-web tests:
  29292 tests complete (42 seconds)
  154 tests pending
  
With the patch "AMBARI-17041-July21-updated.patch" the following is the result 
of ambari-web tests:
  29343 tests complete (55 seconds)
  154 tests pending
  
**With the patch "AMBARI-17041-July22.patch" the following is the result of 
ambari-web tests:
  29371 tests complete (43 seconds)
  154 tests pending


File Attachments


AMBARI-17041-July21-ES6.patch
  

Re: Review Request 49387: Support password type for custom properties

[Keta Patel]

> On July 20, 2016, 11:17 p.m., Alexandr Antonenko wrote:
> > ambari-web/app/utils/config.js, line 264
> > 
> >
> > I do not see FINAL populated anywhere. I think this code will trigger 
> > fatal error. Have you tested this part manually ? 
> > 
> > Also, there is no need to check attribute[0], you just created it, and 
> > anyway there will be one and more values in that array
> > 
> > In code : attributes[0][FINAL][index] && attributes[0][FINAL][index] 
> > === "true"
> > 
> > You can skip cheking attributes[0][FINAL][index], just check 
> > attributes[0][FINAL][index] === "true".
> > 
> > If property "attributes[0][FINAL][index]" does not exist 
> > attributes[0][FINAL][index] === "true" will not trigger any error
> 
> Keta Patel wrote:
> The updated patch "AMBARI-17041-July21-updated.patch" conatins this 
> correction. I have kept the original statement with the name of the variable 
> updated to "attributes[0][FINAL]".
> 
> Thank you!
> 
> Alexandr Antonenko wrote:
> You are using your newly created variable FINAL only one time in 
> attributes[0][FINAL][index]. Why not simply use 
> attributes[0]["FINAL"][index], passing needed string directly, without 
> creating variable FINAL. especially when you have file with tons of variables 
> and functions which names have FINAL in it.

Hello Alexandr,
I have updated the latest patch "AMBARI-17041-July22.patch" by removing the 
"FINAL" variable. I have still kept the variable "PASSWORD" as it is used in 2 
functions (just once in both functions though).

Thank you!


- Keta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/#review143024
---


On July 21, 2016, 11:11 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49387/
> ---
> 
> (Updated July 21, 2016, 11:11 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17041
> https://issues.apache.org/jira/browse/AMBARI-17041
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, services can define properties in the XML configuration files that 
> is flagged as type password:
>   
> my.special.password
> 
> PASSWORD
> Password to be masked
>  
> and it will be masked properly in the UI as well as blueprint.
> 
> Custom property should also support this option so that password can be added 
> as custom property and treat accordingly.
> 
> 
> ==
> Proposed Design for the fix:
> ==
> At present only the key-value information of the service properties is stored 
> in the DB ("clusterconfig" table in the "config_data" column). 
> The "config_attributes" column stores only certain attributes like "final" 
> indicating the list of properties set with the Final flag = true. 
> The information about the property-type (i.e PASSWORD, USER, GROUP, 
> ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, 
> etc) is extracted from the corresponding service's property file (e.g. 
> hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
> information of the existing properties only. Custom Properties added by 
> ambari user have no provision to store their additional attributes. 
> 
> Since, for this Jira we are concerned with only  attribute for 
> Custom Properties, we could add an additional field called "Property Type" in 
> the "Add Property" pop-up which shows up on clicking "Add Property ..." in 
> the Custom property section for a service. For now, only 2 options are shown 
> in the drop-down list: NONE and PASSWORD .
> A few sample test properties are created using the new "Add Property" pop-up 
> as can be seen in the following attachments. 
> Attachments: 
> "add_property_pop_up.tiff"
> "custom_property_password_type.tiff"
> "custom_property_regular_type.tiff"
> "custom_properties_after_save.tiff"
> 
> The  information for these Custom properties is stored in the 
> DB in "clusterconfig" table, "config_attributes" column.
> The schema for "clusterconfig" table can be seen in the attachment:
> "schema_of_clusterconfig_table.tiff"
> 
> The content of the "config_attributes" column with the  
> information from the new Custom properties can be seen in the attachment:
> "cluster_config_with_password_type_in_config_attributes_column.tiff"
> 
> 
> Note: The fix so far is performed only for new Custom properties. The 
>  information for existing properties is extracted from the 
> corresponding 

Review Request 50407: Installing Components After Registering A New Repository Uses The Wrong Version

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50407/
---

Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and 
Jonathan Hurley.


Bugs: AMBARI-17883
https://issues.apache.org/jira/browse/AMBARI-17883


Repository: ambari


Description
---

STR:
1. Install ZK only cluster with HDP-2.4
2. Install repo for HDP-2.5, and distribute bits
3. Do not upgrade, but then add HDFS.

The system will pick HDP-2.5 which will fail.  Fix is to check for roleCommand 
== 'INSTALL' and the passed json 'version' to be present, which is true after 
initial install.


Diffs
-

  ambari-common/src/main/python/resource_management/libraries/script/script.py 
5deeb52 
  ambari-server/src/test/python/stacks/2.0.6/HBASE/test_hbase_master.py f5ed727 

Diff: https://reviews.apache.org/r/50407/diff/


Testing
---

Manual testing of above STR, and newly added test:

--
Total run:1033
Total errors:0
Total failures:0
OK


Thanks,

Nate Cole

Review Request 50406: HBase OOM while booting

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50406/
---

Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, and Sumit Mohanty.


Bugs: AMBARI-17893
https://issues.apache.org/jira/browse/AMBARI-17893


Repository: ambari


Description
---

Caused by: java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:658)
at java.nio.DirectByteBuffer.(DirectByteBuffer.java:123)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:311)
at org.apache.hadoop.hbase.util.ByteBufferArray.(ByteBufferArray.java:65)
at 
org.apache.hadoop.hbase.io.hfile.bucket.ByteBufferIOEngine.(ByteBufferIOEngine.java:47)
at 
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.getIOEngineFromName(BucketCache.java:307)
at 
org.apache.hadoop.hbase.io.hfile.bucket.BucketCache.(BucketCache.java:217)
at 
org.apache.hadoop.hbase.io.hfile.CacheConfig.getBucketCache(CacheConfig.java:614)
at org.apache.hadoop.hbase.io.hfile.CacheConfig.getL2(CacheConfig.java:553)
at 
org.apache.hadoop.hbase.io.hfile.CacheConfig.instantiateBlockCache(CacheConfig.java:637)
at org.apache.hadoop.hbase.io.hfile.CacheConfig.(CacheConfig.java:231)
at 
org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:561)
at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:410)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2507)
This issue has been resolved by adding below config entry in hbase-env.
export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS
{% if hbase_max_direct_memory_size %}
-XX:MaxDirectMemorySize={{hbase_max_direct_memory_size}}m
{% endif %}
"


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.2/services/HBASE/configuration/hbase-env.xml
 1bcc967 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/configuration/hbase-env.xml
 b1f8491 

Diff: https://reviews.apache.org/r/50406/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/#review143414
---


Ship it!




Ship It!

- Sumit Mohanty


On July 25, 2016, 6:17 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50395/
> ---
> 
> (Updated July 25, 2016, 6:17 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17888
> https://issues.apache.org/jira/browse/AMBARI-17888
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A common method of addressing failed disks, which is to set unwritable 
> permissions on a directory such as /hadoop10 within the root filesystem.
> 
> Once mounted, the mounted filesystem will overlay the correct permissions 
> structure, allowing hdfs to write to the mounted disk.
> 
> In the case of a failed mount operation, the associated /hadoop10 directory 
> will be unwritable by hdfs.
> 
> However, Ambari will see the visible directory with incorrect permissions, 
> and "fix" the permissions back to a writable directory-- resulting in the 
> datanode filling the root partition of a machine.
> 
> We need to prevent ambari from changing permissions on YARN local and HDFS 
> data dirs.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
> 9fa986b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  ba59a92 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  89e05d7 
>   ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 06b976e 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
>   ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
> 934007b 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
> daf6e80 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
> d1365ac 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
> 7a68a07 
> 
> Diff: https://reviews.apache.org/r/50395/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Re: Review Request 50377: Update Atlas Metadata server security status for the new version

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50377/#review143408
---


Ship it!




Ship It!

- Robert Levas


On July 23, 2016, 11:31 p.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50377/
> ---
> 
> (Updated July 23, 2016, 11:31 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Robert Levas.
> 
> 
> Bugs: AMBARI-17869
> https://issues.apache.org/jira/browse/AMBARI-17869
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add support for new properties to detect security status
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  f18809f 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 3797cf3 
> 
> Diff: https://reviews.apache.org/r/50377/diff/
> 
> 
> Testing
> ---
> 
> Manually ran UT and tested on a kerberized cluster. UTs pass.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/
---

(Updated July 25, 2016, 6:17 p.m.)


Review request for Ambari, Andrew Onischuk and Sumit Mohanty.


Bugs: AMBARI-17888
https://issues.apache.org/jira/browse/AMBARI-17888


Repository: ambari


Description
---

A common method of addressing failed disks, which is to set unwritable 
permissions on a directory such as /hadoop10 within the root filesystem.

Once mounted, the mounted filesystem will overlay the correct permissions 
structure, allowing hdfs to write to the mounted disk.

In the case of a failed mount operation, the associated /hadoop10 directory 
will be unwritable by hdfs.

However, Ambari will see the visible directory with incorrect permissions, and 
"fix" the permissions back to a writable directory-- resulting in the datanode 
filling the root partition of a machine.

We need to prevent ambari from changing permissions on YARN local and HDFS data 
dirs.


Diffs (updated)
-

  ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
9fa986b 
  
ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 ba59a92 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
89e05d7 
  ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py 06b976e 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
934007b 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
daf6e80 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
d1365ac 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
7a68a07 

Diff: https://reviews.apache.org/r/50395/diff/


Testing
---


Thanks,

Sid Wagle

Re: Review Request 50377: Update Atlas Metadata server security status for the new version

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50377/#review143403
---


Ship it!





ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
 (line 161)


small nitpick here, can remove outside parentheses.



ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
 (line 169)


small nitpick here, can remove outside parentheses.


- Alejandro Fernandez


On July 24, 2016, 3:31 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50377/
> ---
> 
> (Updated July 24, 2016, 3:31 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Robert Levas.
> 
> 
> Bugs: AMBARI-17869
> https://issues.apache.org/jira/browse/AMBARI-17869
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add support for new properties to detect security status
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py
>  f18809f 
>   ambari-server/src/test/python/stacks/2.5/ATLAS/test_atlas_server.py 3797cf3 
> 
> Diff: https://reviews.apache.org/r/50377/diff/
> 
> 
> Testing
> ---
> 
> Manually ran UT and tested on a kerberized cluster. UTs pass.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 50396: Bad error message if user has not performed ambari-server setup for jdbc driver before configuring custom DB

[Robert Levas]

> On July 25, 2016, 1:42 p.m., Robert Levas wrote:
> > Ship It!

I nit-picked on the wording of the error messages.  Esstually all is good, but 
I think the error messages could have been clearer.  My comment read "may not 
be correct", but I really meant "may not be clear".  That is ironc, being that 
my comment wasn't clear. ;)


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50396/#review143399
---


On July 25, 2016, 1:18 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50396/
> ---
> 
> (Updated July 25, 2016, 1:18 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, Robert Levas, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17889
> https://issues.apache.org/jira/browse/AMBARI-17889
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While installing a cluster, I forgot to call:
> {{ambari-server setup --jdbc-db=mysql 
> --jdbc-driver=/usr/share/java/mysql-connector-java.jar}}
> 
> Check DB connection failed with:
> {code}
> 2016-07-13 00:56:32,751 - There was an unknown error while checking database 
> connectivity: cannot concatenate 'str' and 'NoneType' objects
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", 
> line 144, in actionexecute
> db_connection_check_structured_output = 
> self.execute_db_connection_check(config, tmp_dir)
>   File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", 
> line 285, in execute_db_connection_check
> jdbc_url = jdk_location + jdbc_driver_mysql_name
> TypeError: cannot concatenate 'str' and 'NoneType' objects
> 2016-07-13 00:56:32,752 - Check db_connection_check was unsuccessful. Exit 
> code: 1. Message: cannot concatenate 'str' and 'NoneType' objects
> 
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", 
> line 506, in 
> CheckHost().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", 
> line 206, in actionexecute
> raise Fail(error_message)
> resource_management.core.exceptions.Fail: Check db_connection_check was 
> unsuccessful. Exit code: 1. Message: cannot concatenate 'str' and 'NoneType' 
> objects
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/custom_actions/scripts/check_host.py 
> 3d9dfa8 
> 
> Diff: https://reviews.apache.org/r/50396/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 50396: Bad error message if user has not performed ambari-server setup for jdbc driver before configuring custom DB

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50396/#review143399
---


Fix it, then Ship it!




Ship It!


ambari-server/src/main/resources/custom_actions/scripts/check_host.py (line 287)


The error message may not be correct. Maybe use:

"The MySQL JDBC driver has not been set.  Please ensure that you have 
executed 'ambari-server setup --jdbc-db=mysql 
--jdbc-driver=/path/to/jdbc_driver'."



ambari-server/src/main/resources/custom_actions/scripts/check_host.py (line 295)


The error message may not be correct. Maybe use:

"The Oracle JDBC driver has not been set.  Please ensure that you have 
executed 'ambari-server setup --jdbc-db=oracle 
--jdbc-driver=/path/to/jdbc_driver'.



ambari-server/src/main/resources/custom_actions/scripts/check_host.py (line 303)


The error message may not be correct. Maybe use:

"The Postgres JDBC driver has not been set.  Please ensure that you have 
executed 'ambari-server setup --jdbc-db=postgres 
--jdbc-driver=/path/to/jdbc_driver'.



ambari-server/src/main/resources/custom_actions/scripts/check_host.py (line 311)


The error message may not be correct. Maybe use:

"The MSSQL JDBC driver has not been set.  Please ensure that you have 
executed 'ambari-server setup --jdbc-db=mssql 
--jdbc-driver=/path/to/jdbc_driver'.



ambari-server/src/main/resources/custom_actions/scripts/check_host.py (line 319)


The error message may not be correct. Maybe use:

"The SQLAnywhere JDBC driver has not been set.  Please ensure that you have 
executed 'ambari-server setup --jdbc-db=sqlanywhere 
--jdbc-driver=/path/to/jdbc_driver'.


- Robert Levas


On July 25, 2016, 1:18 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50396/
> ---
> 
> (Updated July 25, 2016, 1:18 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, Robert Levas, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17889
> https://issues.apache.org/jira/browse/AMBARI-17889
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> While installing a cluster, I forgot to call:
> {{ambari-server setup --jdbc-db=mysql 
> --jdbc-driver=/usr/share/java/mysql-connector-java.jar}}
> 
> Check DB connection failed with:
> {code}
> 2016-07-13 00:56:32,751 - There was an unknown error while checking database 
> connectivity: cannot concatenate 'str' and 'NoneType' objects
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", 
> line 144, in actionexecute
> db_connection_check_structured_output = 
> self.execute_db_connection_check(config, tmp_dir)
>   File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", 
> line 285, in execute_db_connection_check
> jdbc_url = jdk_location + jdbc_driver_mysql_name
> TypeError: cannot concatenate 'str' and 'NoneType' objects
> 2016-07-13 00:56:32,752 - Check db_connection_check was unsuccessful. Exit 
> code: 1. Message: cannot concatenate 'str' and 'NoneType' objects
> 
> Traceback (most recent call last):
>   File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", 
> line 506, in 
> CheckHost().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", 
> line 206, in actionexecute
> raise Fail(error_message)
> resource_management.core.exceptions.Fail: Check db_connection_check was 
> unsuccessful. Exit code: 1. Message: cannot concatenate 'str' and 'NoneType' 
> objects
> {code}
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/custom_actions/scripts/check_host.py 
> 3d9dfa8 
> 
> Diff: https://reviews.apache.org/r/50396/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/#review143395
---




ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 (line 131)


Can this throw a KeyError ?



ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 (line 216)


typo: point


- Sid Wagle


On July 25, 2016, 5:06 p.m., Sid Wagle wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50395/
> ---
> 
> (Updated July 25, 2016, 5:06 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17888
> https://issues.apache.org/jira/browse/AMBARI-17888
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A common method of addressing failed disks, which is to set unwritable 
> permissions on a directory such as /hadoop10 within the root filesystem.
> 
> Once mounted, the mounted filesystem will overlay the correct permissions 
> structure, allowing hdfs to write to the mounted disk.
> 
> In the case of a failed mount operation, the associated /hadoop10 directory 
> will be unwritable by hdfs.
> 
> However, Ambari will see the visible directory with incorrect permissions, 
> and "fix" the permissions back to a writable directory-- resulting in the 
> datanode filling the root partition of a machine.
> 
> We need to prevent ambari from changing permissions on YARN local and HDFS 
> data dirs.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
> 9fa986b 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
>  ba59a92 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml
>  89e05d7 
>   ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
>   ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
> 934007b 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
> daf6e80 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
> d1365ac 
>   ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
> 7a68a07 
> 
> Diff: https://reviews.apache.org/r/50395/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Sid Wagle
> 
>

Review Request 50396: Bad error message if user has not performed ambari-server setup for jdbc driver before configuring custom DB

[Vitalyi Brodetskyi]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50396/
---

Review request for Ambari, Dmitro Lisnichenko, Dmytro Sen, Robert Levas, and 
Sumit Mohanty.


Bugs: AMBARI-17889
https://issues.apache.org/jira/browse/AMBARI-17889


Repository: ambari


Description
---

While installing a cluster, I forgot to call:
{{ambari-server setup --jdbc-db=mysql 
--jdbc-driver=/usr/share/java/mysql-connector-java.jar}}

Check DB connection failed with:
{code}
2016-07-13 00:56:32,751 - There was an unknown error while checking database 
connectivity: cannot concatenate 'str' and 'NoneType' objects
Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 
144, in actionexecute
db_connection_check_structured_output = 
self.execute_db_connection_check(config, tmp_dir)
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 
285, in execute_db_connection_check
jdbc_url = jdk_location + jdbc_driver_mysql_name
TypeError: cannot concatenate 'str' and 'NoneType' objects
2016-07-13 00:56:32,752 - Check db_connection_check was unsuccessful. Exit 
code: 1. Message: cannot concatenate 'str' and 'NoneType' objects

Traceback (most recent call last):
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 
506, in 
CheckHost().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File "/var/lib/ambari-agent/cache/custom_actions/scripts/check_host.py", line 
206, in actionexecute
raise Fail(error_message)
resource_management.core.exceptions.Fail: Check db_connection_check was 
unsuccessful. Exit code: 1. Message: cannot concatenate 'str' and 'NoneType' 
objects
{code}


Diffs
-

  ambari-server/src/main/resources/custom_actions/scripts/check_host.py 3d9dfa8 

Diff: https://reviews.apache.org/r/50396/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Review Request 50395: Create configuration flag to prevent changing of directory permissions

[Sid Wagle]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50395/
---

Review request for Ambari, Andrew Onischuk and Sumit Mohanty.


Bugs: AMBARI-17888
https://issues.apache.org/jira/browse/AMBARI-17888


Repository: ambari


Description
---

A common method of addressing failed disks, which is to set unwritable 
permissions on a directory such as /hadoop10 within the root filesystem.

Once mounted, the mounted filesystem will overlay the correct permissions 
structure, allowing hdfs to write to the mounted disk.

In the case of a failed mount operation, the associated /hadoop10 directory 
will be unwritable by hdfs.

However, Ambari will see the visible directory with incorrect permissions, and 
"fix" the permissions back to a writable directory-- resulting in the datanode 
filling the root partition of a machine.

We need to prevent ambari from changing permissions on YARN local and HDFS data 
dirs.


Diffs
-

  ambari-agent/src/test/python/resource_management/TestDatanodeHelper.py 
9fa986b 
  
ambari-common/src/main/python/resource_management/libraries/functions/mounted_dirs_helper.py
 ba59a92 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/configuration/cluster-env.xml 
89e05d7 
  ambari-server/src/test/python/stacks/2.0.6/configs/default.json 696b2de 
  ambari-server/src/test/python/stacks/2.0.6/configs/secured.json 7a58ec5 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-default.json 
934007b 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-admin-secured.json 
daf6e80 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-default.json 
d1365ac 
  ambari-server/src/test/python/stacks/2.5/configs/ranger-kms-secured.json 
7a68a07 

Diff: https://reviews.apache.org/r/50395/diff/


Testing
---


Thanks,

Sid Wagle

Re: Review Request 50394: Ranger Atlas plugin status needs to be shown in Ranger summary tab

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50394/#review143394
---


Ship it!




Ship It!

- Jayush Luniya


On July 25, 2016, 5:03 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50394/
> ---
> 
> (Updated July 25, 2016, 5:03 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Zhe (Joe) Wang, Richard Zang, Xi 
> Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-17887
> https://issues.apache.org/jira/browse/AMBARI-17887
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ranger Atlas plugin status needs to be shown in Ranger summary tab
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/service/info/summary.js 47727b7 
> 
> Diff: https://reviews.apache.org/r/50394/diff/
> 
> 
> Testing
> ---
> 
> Verified that the patch fixes the issue on a deployed cluster
> Verified that all unit tests passes with the patch:
> 
> 29223 tests complete (37 seconds)
> 154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Review Request 50394: Ranger Atlas plugin status needs to be shown in Ranger summary tab

[Jaimin Jetly]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50394/
---

Review request for Ambari, Zhe (Joe) Wang, Richard Zang, Xi Wang, and Yusaku 
Sako.


Bugs: AMBARI-17887
https://issues.apache.org/jira/browse/AMBARI-17887


Repository: ambari


Description
---

Ranger Atlas plugin status needs to be shown in Ranger summary tab


Diffs
-

  ambari-web/app/controllers/main/service/info/summary.js 47727b7 

Diff: https://reviews.apache.org/r/50394/diff/


Testing
---

Verified that the patch fixes the issue on a deployed cluster
Verified that all unit tests passes with the patch:

29223 tests complete (37 seconds)
154 tests pending


Thanks,

Jaimin Jetly

Re: Review Request 50390: after upgrade is finalized add service operation hang in Preparing to execute Set Keytab

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50390/#review143382
---


Ship it!




Ship It!

- Robert Levas


On July 25, 2016, 10:12 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50390/
> ---
> 
> (Updated July 25, 2016, 10:12 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, Nate 
> Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17878
> https://issues.apache.org/jira/browse/AMBARI-17878
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *add service operation hangs in Preparing to execute Set Keytab*
> 
> 1. install ambari2.4.0 with stack 2.4
> 2. express upgrade to stack 2.5
> 3. finalize the upgrade
> 4. add new services ( kms,atlas,slider)
> 
> Null pointer exception is thrown in ambari-server logs
> 
> {code}
> 
> 18 Jul 2016 07:12:32,025 ERROR [ambari-metrics-retrieval-service-thread-105] 
> MetricsRetrievalService:421 - Unable to retrieve metrics from 
> http://host:8744/api/v1/cluster/summary. Subsequent failures will be 
> suppressed from the log for 5 minutes.
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
> at sun.net.www.http.HttpClient.(HttpClient.java:211)
> at sun.net.www.http.HttpClient.New(HttpClient.java:308)
> at sun.net.www.http.HttpClient.New(HttpClient.java:326)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
> at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
> at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
> at 
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
> at 
> org.apache.ambari.server.state.services.MetricsRetrievalService$MetricRunnable.run(MetricsRetrievalService.java:382)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 
> {code}
> 
> 
> 18 Jul 2016 07:42:39,979  WARN [ambari-action-scheduler] ActionScheduler:202 
> - Exception received
> java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at org.apache.ambari.server.agent.ActionQueue.getQueue(ActionQueue.java:52)
> at org.apache.ambari.server.agent.ActionQueue.enqueue(ActionQueue.java:61)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.cancelCommandOnTimeout(ActionScheduler.java:1084)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.processInProgressStage(ActionScheduler.java:725)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:290)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:197)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 
> Note: this is an upgraded cluster from hdp-2.4 to hdp-2.5 , addition of the 
> service is done after upgrade.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  90ffa5b 
>   ambari-server/src/main/java/org/apache/ambari/server/agent/ActionQueue.java 
> 24640d3 
>   
> ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
>  0b24765 
> 
> Diff: 

Re: Review Request 50349: YARN service check failed during EU from HDP-2.4.0.0 to Erie

[Dmitro Lisnichenko]

> On July 22, 2016, 11:18 p.m., Nate Cole wrote:
> > Do all existing tests pass?

yes


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50349/#review143262
---


On July 25, 2016, 5:31 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50349/
> ---
> 
> (Updated July 25, 2016, 5:31 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17859
> https://issues.apache.org/jira/browse/AMBARI-17859
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *Steps*
> # Deploy HDP-2.4.0.0 cluster with Ambari 2.2.1.1 (secure, non-HA cluster, 
> customized service users)
> # Upgrade Ambari to 2.4.0.0
> # Perform EU to 2.5.0.0-934
> 
> There is not live cluster to see what happened. We had a similar issue 
> previously, when core-site was not provided by UI (e.g. when deleting 
> services) because Atlas had not dependencies on it. Similar issue was 
> addressed by AMBARI-17536 (The FE should always send the cluster-env config 
> when invoking the Recommendations REST API), so changing logic to depend on 
> cluster-env instead of core-site at least should not harm. 
> 
> ```
> Traceback (most recent call last):\n  File 
> \"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py\",
>  line 159, in \nServiceCheck().execute()\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
>  line 280, in execute\nmethod(env)\n  File 
> \"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py\",
>  line 117, in service_check\nuser=params.smokeuser,\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
> 71, in inner\nresult = function(command, **kwargs)\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
> 93, in checked_call\ntries=tries, try_sleep=try_sleep)\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
> 141, in _call_wrapper\nresult = _call(command, **kwargs_copy)\n  File 
> \"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\"
 , line 294, in _call\nraise 
Fail(err_msg)\nresource_management.core.exceptions.Fail: Execution of 
'/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab 
smk_rndalxit7oqf2pq3erlqg6r...@hwqe.hortonworks.com; yarn 
org.apache.hadoop.yarn.applications.distributedshell.Client -shell_command ls 
-num_containers 1 -jar 
/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar
 -timeout 30 --queue default' returned 2.  Hortonworks 
#\nThis is MOTD message, added for testing in qe infra\n16/07/09 
11:15:01 INFO impl.TimelineClientImpl: Timeline service address: 
http://host:8188/ws/v1/timeline/\n16/07/09 11:15:01 INFO 
distributedshell.Client: Initializing Client\n16/07/09 11:15:01 INFO 
distributedshell.Client: Running Client\n16/07/09 11:15:01 INFO client.RMProxy: 
Connecting to ResourceManager at host-5.domainlocal/10.0.113.157:8050\n16/07/09 
11:15:03 INFO distributedshell.Client: Got Cluster metric info from ASM, 
numNodeManagers=3\n
 16/07/09 11:15:03 INFO distributedshell.Client: Got Cluster node info from 
ASM\n16/07/09 11:15:03 INFO distributedshell.Client: Got node report from ASM 
for, nodeId=host:25454, nodeAddresshost:8042, nodeRackName/default-rack, 
nodeNumContainers0\n16/07/09 11:15:03 INFO distributedshell.Client: Got node 
report from ASM for, nodeId=host-5.domainlocal:25454, 
nodeAddresshost-5.domainlocal:8042, nodeRackName/default-rack, 
nodeNumContainers0\n
> ```
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> 5b7b7a4 
> 
> Diff: https://reviews.apache.org/r/50349/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 50349: YARN service check failed during EU from HDP-2.4.0.0 to Erie

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50349/
---

(Updated July 25, 2016, 5:31 p.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, and Robert Levas.


Changes
---

Fixed review comments.

Python tests:
 Total run:1028
Total errors:0
Total failures:0
OK


Bugs: AMBARI-17859
https://issues.apache.org/jira/browse/AMBARI-17859


Repository: ambari


Description
---

*Steps*
# Deploy HDP-2.4.0.0 cluster with Ambari 2.2.1.1 (secure, non-HA cluster, 
customized service users)
# Upgrade Ambari to 2.4.0.0
# Perform EU to 2.5.0.0-934

There is not live cluster to see what happened. We had a similar issue 
previously, when core-site was not provided by UI (e.g. when deleting services) 
because Atlas had not dependencies on it. Similar issue was addressed by 
AMBARI-17536 (The FE should always send the cluster-env config when invoking 
the Recommendations REST API), so changing logic to depend on cluster-env 
instead of core-site at least should not harm. 

```
Traceback (most recent call last):\n  File 
\"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py\",
 line 159, in \nServiceCheck().execute()\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py\",
 line 280, in execute\nmethod(env)\n  File 
\"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/service_check.py\",
 line 117, in service_check\nuser=params.smokeuser,\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
71, in inner\nresult = function(command, **kwargs)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
93, in checked_call\ntries=tries, try_sleep=try_sleep)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", line 
141, in _call_wrapper\nresult = _call(command, **kwargs_copy)\n  File 
\"/usr/lib/python2.6/site-packages/resource_management/core/shell.py\", 
 line 294, in _call\nraise 
Fail(err_msg)\nresource_management.core.exceptions.Fail: Execution of 
'/usr/bin/kinit -kt /etc/security/keytabs/smokeuser.headless.keytab 
smk_rndalxit7oqf2pq3erlqg6r...@hwqe.hortonworks.com; yarn 
org.apache.hadoop.yarn.applications.distributedshell.Client -shell_command ls 
-num_containers 1 -jar 
/usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell.jar
 -timeout 30 --queue default' returned 2.  Hortonworks 
#\nThis is MOTD message, added for testing in qe infra\n16/07/09 
11:15:01 INFO impl.TimelineClientImpl: Timeline service address: 
http://host:8188/ws/v1/timeline/\n16/07/09 11:15:01 INFO 
distributedshell.Client: Initializing Client\n16/07/09 11:15:01 INFO 
distributedshell.Client: Running Client\n16/07/09 11:15:01 INFO client.RMProxy: 
Connecting to ResourceManager at host-5.domainlocal/10.0.113.157:8050\n16/07/09 
11:15:03 INFO distributedshell.Client: Got Cluster metric info from ASM, 
numNodeManagers=3\n16
 /07/09 11:15:03 INFO distributedshell.Client: Got Cluster node info from 
ASM\n16/07/09 11:15:03 INFO distributedshell.Client: Got node report from ASM 
for, nodeId=host:25454, nodeAddresshost:8042, nodeRackName/default-rack, 
nodeNumContainers0\n16/07/09 11:15:03 INFO distributedshell.Client: Got node 
report from ASM for, nodeId=host-5.domainlocal:25454, 
nodeAddresshost-5.domainlocal:8042, nodeRackName/default-rack, 
nodeNumContainers0\n
```


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
5b7b7a4 

Diff: https://reviews.apache.org/r/50349/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 50390: after upgrade is finalized add service operation hang in Preparing to execute Set Keytab

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50390/
---

(Updated July 25, 2016, 5:12 p.m.)


Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, Nate 
Cole, and Robert Levas.


Changes
---

new patch


Bugs: AMBARI-17878
https://issues.apache.org/jira/browse/AMBARI-17878


Repository: ambari


Description
---

*add service operation hangs in Preparing to execute Set Keytab*

1. install ambari2.4.0 with stack 2.4
2. express upgrade to stack 2.5
3. finalize the upgrade
4. add new services ( kms,atlas,slider)

Null pointer exception is thrown in ambari-server logs

{code}

18 Jul 2016 07:12:32,025 ERROR [ambari-metrics-retrieval-service-thread-105] 
MetricsRetrievalService:421 - Unable to retrieve metrics from 
http://host:8744/api/v1/cluster/summary. Subsequent failures will be suppressed 
from the log for 5 minutes.
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at 
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at 
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
at 
org.apache.ambari.server.state.services.MetricsRetrievalService$MetricRunnable.run(MetricsRetrievalService.java:382)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}

{code}


18 Jul 2016 07:42:39,979  WARN [ambari-action-scheduler] ActionScheduler:202 - 
Exception received
java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
at org.apache.ambari.server.agent.ActionQueue.getQueue(ActionQueue.java:52)
at org.apache.ambari.server.agent.ActionQueue.enqueue(ActionQueue.java:61)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.cancelCommandOnTimeout(ActionScheduler.java:1084)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.processInProgressStage(ActionScheduler.java:725)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:290)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:197)
at java.lang.Thread.run(Thread.java:745)
{code}

Note: this is an upgraded cluster from hdp-2.4 to hdp-2.5 , addition of the 
service is done after upgrade.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 90ffa5b 
  ambari-server/src/main/java/org/apache/ambari/server/agent/ActionQueue.java 
24640d3 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
 0b24765 

Diff: https://reviews.apache.org/r/50390/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 50392: AMBARI-17856: quicklink url protocol type check should allow type specified as HTTP and HTTPS

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50392/#review143377
---


Ship it!




Ship It!

- Tim Thorpe


On July 25, 2016, 12:40 p.m., Di Li wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50392/
> ---
> 
> (Updated July 25, 2016, 12:40 p.m.)
> 
> 
> Review request for Ambari and Tim Thorpe.
> 
> 
> Bugs: AMBARI-17856
> https://issues.apache.org/jira/browse/AMBARI-17856
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> quicklink url protocol type check should allow type specified as HTTP and 
> HTTPS in order to have the types be consistent (case-wise) with the other two 
> possible cases: HTTPS_ONLY and HTTP_ONLY
> 
> 
> Diffs
> -
> 
>   ambari-web/app/views/common/quick_view_link_view.js 17d7c04 
>   ambari-web/test/views/common/quick_link_view_test.js f9a52b1 
> 
> Diff: https://reviews.apache.org/r/50392/diff/
> 
> 
> Testing
> ---
> 
> updated unit tests
> 
> 
> Thanks,
> 
> Di Li
> 
>

Review Request 50393: AMBARI-17884 : Stack advisor doesnot recommend Yarn as requiring restart for Ranger plugin side config changes

[Gautam Borad]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50393/
---

Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Sumit Mohanty, 
Srimanth Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17884
https://issues.apache.org/jira/browse/AMBARI-17884


Repository: ambari


Description
---

For a config change on plugin side for Ranger, Yarn is not identified as part 
of services requiring restart.

As a fix added the required dependency in YARN/2.1.0.2.0/metainfo.xml


Diffs
-

  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml 
635cec748ac07056de23ee21ec709815d6804f5e 

Diff: https://reviews.apache.org/r/50393/diff/


Testing
---

Tested on a local centos6 cluster with Yarn.


Thanks,

Gautam Borad

Re: Review Request 50297: AMBARI-17788: Refactor spooler code in OutputHDFSFile to be reusable for OutputS3File

[Hayat Behlim]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50297/#review143372
---




ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpooler.java
 (line 126)


I think just for sake of understanding initializeSpoolFile() will come 
after handleRollover method is invoked.


- Hayat Behlim


On July 25, 2016, 7:17 a.m., Hemanth Yamijala wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50297/
> ---
> 
> (Updated July 25, 2016, 7:17 a.m.)
> 
> 
> Review request for Ambari, Hayat Behlim and Dharmesh Makwana.
> 
> 
> Bugs: AMBARI-17788
> https://issues.apache.org/jira/browse/AMBARI-17788
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch extracts spooling functions outside OutputHDFSFile so that it can 
> be used for AMBARI-17785. The patch allows callbacks to check the condition 
> on which spool files should be rolled over and what to do on rollover.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java
>  87cc0eb 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpooler.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerContext.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerException.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverCondition.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverHandler.java
>  PRE-CREATION 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/50297/diff/
> 
> 
> Testing
> ---
> 
> * New unit tests.
> * Have tested by installing new RPM on an existing Ambari installation and 
> configuring the output for one of the services to write to HDFS. Tested that 
> upload to HDFS, rollover is working correctly for this service.
> 
> 
> Thanks,
> 
> Hemanth Yamijala
> 
>

Review Request 50392: AMBARI-17856: quicklink url protocol type check should allow type specified as HTTP and HTTPS

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50392/
---

Review request for Ambari and Tim Thorpe.


Bugs: AMBARI-17856
https://issues.apache.org/jira/browse/AMBARI-17856


Repository: ambari


Description
---

quicklink url protocol type check should allow type specified as HTTP and HTTPS 
in order to have the types be consistent (case-wise) with the other two 
possible cases: HTTPS_ONLY and HTTP_ONLY


Diffs
-

  ambari-web/app/views/common/quick_view_link_view.js 17d7c04 
  ambari-web/test/views/common/quick_link_view_test.js f9a52b1 

Diff: https://reviews.apache.org/r/50392/diff/


Testing
---

updated unit tests


Thanks,

Di Li

Re: Review Request 50390: after upgrade is finalized add service operation hang in Preparing to execute Set Keytab

[Robert Levas]

> On July 25, 2016, 6:49 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java,
> >  line 1043
> > 
> >
> > Wouldn't it be better to do:
> > 
> > ```
> > if (Role.AMBARI_SERVER_ACTION != hostRoleCommand.getRole())
> > ```
> 
> Nate Cole wrote:
> I think actually 
> !Role.AMBARI_SERVER_ACTION.equals(hostRoleCommand.getRole()) as Role isn't 
> actually an Enum.

Interesting.. I always thought it was.  Nice catch.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50390/#review143361
---


On July 25, 2016, 6:03 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50390/
> ---
> 
> (Updated July 25, 2016, 6:03 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, Nate 
> Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17878
> https://issues.apache.org/jira/browse/AMBARI-17878
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *add service operation hangs in Preparing to execute Set Keytab*
> 
> 1. install ambari2.4.0 with stack 2.4
> 2. express upgrade to stack 2.5
> 3. finalize the upgrade
> 4. add new services ( kms,atlas,slider)
> 
> Null pointer exception is thrown in ambari-server logs
> 
> {code}
> 
> 18 Jul 2016 07:12:32,025 ERROR [ambari-metrics-retrieval-service-thread-105] 
> MetricsRetrievalService:421 - Unable to retrieve metrics from 
> http://host:8744/api/v1/cluster/summary. Subsequent failures will be 
> suppressed from the log for 5 minutes.
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
> at sun.net.www.http.HttpClient.(HttpClient.java:211)
> at sun.net.www.http.HttpClient.New(HttpClient.java:308)
> at sun.net.www.http.HttpClient.New(HttpClient.java:326)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
> at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
> at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
> at 
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
> at 
> org.apache.ambari.server.state.services.MetricsRetrievalService$MetricRunnable.run(MetricsRetrievalService.java:382)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 
> {code}
> 
> 
> 18 Jul 2016 07:42:39,979  WARN [ambari-action-scheduler] ActionScheduler:202 
> - Exception received
> java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at org.apache.ambari.server.agent.ActionQueue.getQueue(ActionQueue.java:52)
> at org.apache.ambari.server.agent.ActionQueue.enqueue(ActionQueue.java:61)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.cancelCommandOnTimeout(ActionScheduler.java:1084)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.processInProgressStage(ActionScheduler.java:725)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:290)
> at 
> 

Re: Review Request 50391: Kerberos keytab regeneration stack advisor bug

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50391/#review143364
---


Ship it!




Ship It!

- Robert Levas


On July 25, 2016, 7:13 a.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50391/
> ---
> 
> (Updated July 25, 2016, 7:13 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Mahadev Konar, Robert Levas, Robert 
> Nettleton, Sandor Magyari, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17880
> https://issues.apache.org/jira/browse/AMBARI-17880
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If the kerberos keytabs are regenerated via ambari then the stack advisor 
> also runs, but instead of deleting those properties which are recommended to 
> do so it sets the value to an empty string, which may lead to an inconsistent 
> state.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/PrepareKerberosIdentitiesServerAction.java
>  0368881 
> 
> Diff: https://reviews.apache.org/r/50391/diff/
> 
> 
> Testing
> ---
> 
> Tested on openstack cluster created for the bug.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 50390: after upgrade is finalized add service operation hang in Preparing to execute Set Keytab

[Nate Cole]

> On July 25, 2016, 6:49 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java,
> >  line 1043
> > 
> >
> > Wouldn't it be better to do:
> > 
> > ```
> > if (Role.AMBARI_SERVER_ACTION != hostRoleCommand.getRole())
> > ```

I think actually !Role.AMBARI_SERVER_ACTION.equals(hostRoleCommand.getRole()) 
as Role isn't actually an Enum.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50390/#review143361
---


On July 25, 2016, 6:03 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50390/
> ---
> 
> (Updated July 25, 2016, 6:03 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, Nate 
> Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17878
> https://issues.apache.org/jira/browse/AMBARI-17878
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *add service operation hangs in Preparing to execute Set Keytab*
> 
> 1. install ambari2.4.0 with stack 2.4
> 2. express upgrade to stack 2.5
> 3. finalize the upgrade
> 4. add new services ( kms,atlas,slider)
> 
> Null pointer exception is thrown in ambari-server logs
> 
> {code}
> 
> 18 Jul 2016 07:12:32,025 ERROR [ambari-metrics-retrieval-service-thread-105] 
> MetricsRetrievalService:421 - Unable to retrieve metrics from 
> http://host:8744/api/v1/cluster/summary. Subsequent failures will be 
> suppressed from the log for 5 minutes.
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
> at sun.net.www.http.HttpClient.(HttpClient.java:211)
> at sun.net.www.http.HttpClient.New(HttpClient.java:308)
> at sun.net.www.http.HttpClient.New(HttpClient.java:326)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
> at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
> at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
> at 
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
> at 
> org.apache.ambari.server.state.services.MetricsRetrievalService$MetricRunnable.run(MetricsRetrievalService.java:382)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 
> {code}
> 
> 
> 18 Jul 2016 07:42:39,979  WARN [ambari-action-scheduler] ActionScheduler:202 
> - Exception received
> java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at org.apache.ambari.server.agent.ActionQueue.getQueue(ActionQueue.java:52)
> at org.apache.ambari.server.agent.ActionQueue.enqueue(ActionQueue.java:61)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.cancelCommandOnTimeout(ActionScheduler.java:1084)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.processInProgressStage(ActionScheduler.java:725)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:290)
> at 
> org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:197)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 
> Note: this is 

Re: Review Request 50390: after upgrade is finalized add service operation hang in Preparing to execute Set Keytab

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50390/#review143361
---



This solution seems to be limited. I think it would be better to protected 
calls to `org.apache.ambari.server.agent.ActionQueue#getQueue` by ensuring 
hostname is not null before attempting to getting a quueue from the 
`hostQueues` map. It appears that the callers of this method handle a `null` 
host queue properly.


ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 (line 1043)


Wouldn't it be better to do:

```
if (Role.AMBARI_SERVER_ACTION != hostRoleCommand.getRole())
```



ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 (line 1077)


Wouldn't it be better to do:

```
if (Role.AMBARI_SERVER_ACTION != hostRoleCommand.getRole())
```


- Robert Levas


On July 25, 2016, 6:03 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50390/
> ---
> 
> (Updated July 25, 2016, 6:03 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, Nate 
> Cole, and Robert Levas.
> 
> 
> Bugs: AMBARI-17878
> https://issues.apache.org/jira/browse/AMBARI-17878
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> *add service operation hangs in Preparing to execute Set Keytab*
> 
> 1. install ambari2.4.0 with stack 2.4
> 2. express upgrade to stack 2.5
> 3. finalize the upgrade
> 4. add new services ( kms,atlas,slider)
> 
> Null pointer exception is thrown in ambari-server logs
> 
> {code}
> 
> 18 Jul 2016 07:12:32,025 ERROR [ambari-metrics-retrieval-service-thread-105] 
> MetricsRetrievalService:421 - Unable to retrieve metrics from 
> http://host:8744/api/v1/cluster/summary. Subsequent failures will be 
> suppressed from the log for 5 minutes.
> java.net.ConnectException: Connection refused
> at java.net.PlainSocketImpl.socketConnect(Native Method)
> at 
> java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
> at 
> java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
> at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
> at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> at java.net.Socket.connect(Socket.java:589)
> at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
> at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
> at sun.net.www.http.HttpClient.(HttpClient.java:211)
> at sun.net.www.http.HttpClient.New(HttpClient.java:308)
> at sun.net.www.http.HttpClient.New(HttpClient.java:326)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
> at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
> at 
> sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
> at 
> sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
> at 
> org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
> at 
> org.apache.ambari.server.state.services.MetricsRetrievalService$MetricRunnable.run(MetricsRetrievalService.java:382)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
> 
> {code}
> 
> 
> 18 Jul 2016 07:42:39,979  WARN [ambari-action-scheduler] ActionScheduler:202 
> - Exception received
> java.lang.NullPointerException
> at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
> at org.apache.ambari.server.agent.ActionQueue.getQueue(ActionQueue.java:52)
> at org.apache.ambari.server.agent.ActionQueue.enqueue(ActionQueue.java:61)
> at 
> 

Re: Review Request 50390: after upgrade is finalized add service operation hang in Preparing to execute Set Keytab

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50390/
---

(Updated July 25, 2016, 1:03 p.m.)


Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, Nate 
Cole, and Robert Levas.


Bugs: AMBARI-17878
https://issues.apache.org/jira/browse/AMBARI-17878


Repository: ambari


Description (updated)
---

*add service operation hangs in Preparing to execute Set Keytab*

1. install ambari2.4.0 with stack 2.4
2. express upgrade to stack 2.5
3. finalize the upgrade
4. add new services ( kms,atlas,slider)

Null pointer exception is thrown in ambari-server logs

{code}

18 Jul 2016 07:12:32,025 ERROR [ambari-metrics-retrieval-service-thread-105] 
MetricsRetrievalService:421 - Unable to retrieve metrics from 
http://host:8744/api/v1/cluster/summary. Subsequent failures will be suppressed 
from the log for 5 minutes.
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at 
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at 
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
at 
org.apache.ambari.server.state.services.MetricsRetrievalService$MetricRunnable.run(MetricsRetrievalService.java:382)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}

{code}


18 Jul 2016 07:42:39,979  WARN [ambari-action-scheduler] ActionScheduler:202 - 
Exception received
java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
at org.apache.ambari.server.agent.ActionQueue.getQueue(ActionQueue.java:52)
at org.apache.ambari.server.agent.ActionQueue.enqueue(ActionQueue.java:61)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.cancelCommandOnTimeout(ActionScheduler.java:1084)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.processInProgressStage(ActionScheduler.java:725)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:290)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:197)
at java.lang.Thread.run(Thread.java:745)
{code}

Note: this is an upgraded cluster from hdp-2.4 to hdp-2.5 , addition of the 
service is done after upgrade.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 205ef9f 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
 0b24765 

Diff: https://reviews.apache.org/r/50390/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Review Request 50390: after upgrade is finalized add service operation hang in Preparing to execute Set Keytab

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50390/
---

Review request for Ambari, Alejandro Fernandez, Myroslav Papirkovskyy, Nate 
Cole, and Robert Levas.


Bugs: AMBARI-17878
https://issues.apache.org/jira/browse/AMBARI-17878


Repository: ambari


Description
---

*add service operation hangs in Preparing to execute Set Keytab*

1. install ambari2.4.0 with stack 2.4
2. express upgrade to stack 2.5
3. finalize the upgrade
4. add new services ( kms,atlas,slider)

Null pointer exception is thrown in ambari-server logs

{code}

18 Jul 2016 07:12:32,025 ERROR [ambari-metrics-retrieval-service-thread-105] 
MetricsRetrievalService:421 - Unable to retrieve metrics from 
http://host:8744/api/v1/cluster/summary. Subsequent failures will be suppressed 
from the log for 5 minutes.
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at 
java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.net.NetworkClient.doConnect(NetworkClient.java:175)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:432)
at sun.net.www.http.HttpClient.openServer(HttpClient.java:527)
at sun.net.www.http.HttpClient.(HttpClient.java:211)
at sun.net.www.http.HttpClient.New(HttpClient.java:308)
at sun.net.www.http.HttpClient.New(HttpClient.java:326)
at 
sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1169)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1105)
at 
sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:999)
at 
sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:933)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1513)
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:218)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.processURL(URLStreamProvider.java:142)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:116)
at 
org.apache.ambari.server.controller.internal.URLStreamProvider.readFrom(URLStreamProvider.java:121)
at 
org.apache.ambari.server.state.services.MetricsRetrievalService$MetricRunnable.run(MetricsRetrievalService.java:382)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{code}

{code}


18 Jul 2016 07:42:39,979  WARN [ambari-action-scheduler] ActionScheduler:202 - 
Exception received
java.lang.NullPointerException
at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
at org.apache.ambari.server.agent.ActionQueue.getQueue(ActionQueue.java:52)
at org.apache.ambari.server.agent.ActionQueue.enqueue(ActionQueue.java:61)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.cancelCommandOnTimeout(ActionScheduler.java:1084)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.processInProgressStage(ActionScheduler.java:725)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.doWork(ActionScheduler.java:290)
at 
org.apache.ambari.server.actionmanager.ActionScheduler.run(ActionScheduler.java:197)
at java.lang.Thread.run(Thread.java:745)
{code}

Note: this is an upgraded cluster from hdp-2.4 to hdp-2.5 , addition of the 
service is done after upgrade.

cluster:
172.22.108.134

cc [~skoneru][~ssanthosh]


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
 205ef9f 
  
ambari-server/src/test/java/org/apache/ambari/server/actionmanager/TestActionScheduler.java
 0b24765 

Diff: https://reviews.apache.org/r/50390/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 50386: Zeppelin quick link does not respect zeppelin.server.port config

[Gaurav Nagar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50386/#review143359
---


Ship it!




Ship It!

- Gaurav Nagar


On July 25, 2016, 8:01 a.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50386/
> ---
> 
> (Updated July 25, 2016, 8:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Nitiraj Rathore, Pallav Kulshreshtha, Prabhjyot Singh, Rohit 
> Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17870
> https://issues.apache.org/jira/browse/AMBARI-17870
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> updated quicklinks.json
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/quicklinks/quicklinks.json
>  fcc 
> 
> Diff: https://reviews.apache.org/r/50386/diff/
> 
> 
> Testing
> ---
> 
> tested manually on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 50386: Zeppelin quick link does not respect zeppelin.server.port config

[Prabhjyot Singh]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50386/#review143358
---


Ship it!




Ship It!

- Prabhjyot Singh


On July 25, 2016, 8:01 a.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50386/
> ---
> 
> (Updated July 25, 2016, 8:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Nitiraj Rathore, Pallav Kulshreshtha, Prabhjyot Singh, Rohit 
> Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17870
> https://issues.apache.org/jira/browse/AMBARI-17870
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> updated quicklinks.json
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/quicklinks/quicklinks.json
>  fcc 
> 
> Diff: https://reviews.apache.org/r/50386/diff/
> 
> 
> Testing
> ---
> 
> tested manually on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 50386: Zeppelin quick link does not respect zeppelin.server.port config

[Rohit Choudhary]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50386/#review143357
---


Ship it!




Ship It!

- Rohit Choudhary


On July 25, 2016, 8:01 a.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50386/
> ---
> 
> (Updated July 25, 2016, 8:01 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Nitiraj Rathore, Pallav Kulshreshtha, Prabhjyot Singh, Rohit 
> Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17870
> https://issues.apache.org/jira/browse/AMBARI-17870
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> updated quicklinks.json
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/quicklinks/quicklinks.json
>  fcc 
> 
> Diff: https://reviews.apache.org/r/50386/diff/
> 
> 
> Testing
> ---
> 
> tested manually on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Review Request 50386: Zeppelin quick link does not respect zeppelin.server.port config

[Renjith Kamath]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50386/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Nitiraj Rathore, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and 
Sumit Mohanty.


Bugs: AMBARI-17870
https://issues.apache.org/jira/browse/AMBARI-17870


Repository: ambari


Description
---

updated quicklinks.json


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/quicklinks/quicklinks.json
 fcc 

Diff: https://reviews.apache.org/r/50386/diff/


Testing
---

tested manually on CentOS


Thanks,

Renjith Kamath

Re: Review Request 50297: AMBARI-17788: Refactor spooler code in OutputHDFSFile to be reusable for OutputS3File

[Hemanth Yamijala]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50297/
---

(Updated July 25, 2016, 7:17 a.m.)


Review request for Ambari, Hayat Behlim and Dharmesh Makwana.


Changes
---

Rebased to trunk after changes in AMBARI-17834


Bugs: AMBARI-17788
https://issues.apache.org/jira/browse/AMBARI-17788


Repository: ambari


Description
---

This patch extracts spooling functions outside OutputHDFSFile so that it can be 
used for AMBARI-17785. The patch allows callbacks to check the condition on 
which spool files should be rolled over and what to do on rollover.


Diffs (updated)
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/OutputHDFSFile.java
 87cc0eb 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpooler.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerContext.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerException.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverCondition.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/spool/RolloverHandler.java
 PRE-CREATION 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/test/java/org/apache/ambari/logfeeder/output/spool/LogSpoolerTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/50297/diff/


Testing
---

* New unit tests.
* Have tested by installing new RPM on an existing Ambari installation and 
configuring the output for one of the services to write to HDFS. Tested that 
upload to HDFS, rollover is working correctly for this service.


Thanks,

Hemanth Yamijala