Re: Review Request 49387: Support password type for custom properties

[Matt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/#review141391
---




ambari-web/app/mixins/common/configs/configs_saver.js (line 469)


Isn't it possible to simplify all these if conditions (considering your 
property types and attribute names are the same, besides the upper/lowercase)?

```
if (Em.get(property,'propertyName') != null) {
  Em.get(property,'propertyName').map(function(propType) { 
attributes[propType.toLowerCase()][Em.get(property,'name')] = true;
  });
}
```


- Matt


On July 8, 2016, 2:15 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49387/
> ---
> 
> (Updated July 8, 2016, 2:15 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17041
> https://issues.apache.org/jira/browse/AMBARI-17041
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, services can define properties in the XML configuration files that 
> is flagged as type password:
>   
> my.special.password
> 
> PASSWORD
> Password to be masked
>  
> and it will be masked properly in the UI as well as blueprint.
> 
> Custom property should also support this option so that password can be added 
> as custom property and treat accordingly.
> 
> 
> ==
> Proposed Design for the fix:
> ==
> At present only the key-value information of the service properties is stored 
> in the DB ("clusterconfig" table in the "config_data" column). 
> The "config_attributes" column stores only certain attributes like "final" 
> indicating the list of properties set with the Final flag = true. 
> The information about the property-type (i.e PASSWORD, USER, GROUP, 
> ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, 
> etc) is extracted from the corresponding service's property file (e.g. 
> hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
> information of the existing properties only. Custom Properties added by 
> ambari user have no provision to store their additional attributes. 
> 
> Since, for this Jira we are concerned with only  attribute for 
> Custom Properties, we could add an additional field called "Property Type" in 
> the "Add Property" pop-up which shows up on clicking "Add Property ..." in 
> the Custom property section for a service. For now, only 2 options are shown 
> in the drop-down list: NONE and PASSWORD .
> A few sample test properties are created using the new "Add Property" pop-up 
> as can be seen in the following attachments. 
> Attachments: 
> "add_property_pop_up.tiff"
> "custom_property_password_type.tiff"
> "custom_property_regular_type.tiff"
> "custom_properties_after_save.tiff"
> 
> The  information for these Custom properties is stored in the 
> DB in "clusterconfig" table, "config_attributes" column.
> The schema for "clusterconfig" table can be seen in the attachment:
> "schema_of_clusterconfig_table.tiff"
> 
> The content of the "config_attributes" column with the  
> information from the new Custom properties can be seen in the attachment:
> "cluster_config_with_password_type_in_config_attributes_column.tiff"
> 
> 
> Note: The fix so far is performed only for new Custom properties. The 
>  information for existing properties is extracted from the 
> corresponding property xml files for the service.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationResponse.java
>  eef3474 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/SecretReference.java
>  84f3109 
>   ambari-web/app/messages.js bbac1bd 
>   ambari-web/app/mixins/common/configs/configs_saver.js 43a77ce 
>   ambari-web/app/models/configs/objects/service_config_property.js 861d599 
>   ambari-web/app/templates/common/configs/addPropertyWindow.hbs 659435c 
>   ambari-web/app/utils/config.js 6c6e7f6 
>   ambari-web/app/views/common/configs/service_configs_by_category_view.js 
> bdd0530 
>   ambari-web/test/mixins/common/configs/configs_saver_test.js 4baf303 
> 
> Diff: https://reviews.apache.org/r/49387/diff/
> 
> 
> Testing
> ---
> 
> No new test cases are written in the patch apart from the fix required in 
> "configs_saver_test.js" to avoid test failure of existing tests.
> If the proposed solution is acceptable, I will create a new patch with the 
> necessary new tests.
> 
> The existing ambari-web tests after applying the patch:
> 
>   28977 tests complete (37 seconds)
>   154 

Review Request 49838: AMBARI-17636 Service Configs page: can't see all config versions in dropdown

[Vivek Ratnavel Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49838/
---

Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, and Yusaku Sako.


Bugs: AMBARI-17636
https://issues.apache.org/jira/browse/AMBARI-17636


Repository: ambari


Description
---

* Go to Service->Configs page (that have more than ~ 30 versions)
* Open versions dropdown

*Result:*
Number of visible versions limited by window height.


Diffs
-

  ambari-web/app/styles/config_history_flow.less fff3151 
  ambari-web/app/templates/common/configs/config_history_flow.hbs 73d6913 
  ambari-web/app/views/common/configs/config_history_flow.js ad2b1ab 

Diff: https://reviews.apache.org/r/49838/diff/


Testing
---

Local Ambari web test passed.
28944 tests complete (25 seconds)
154 tests pending
Manual testing done.


Thanks,

Vivek Ratnavel Subramanian

Review Request 49834: Logfeeder is not starting with non-root agent install

[Oliver Szabo]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49834/
---

Review request for Ambari, Miklos Gergely, Sumit Mohanty, and Sebastian Toader.


Bugs: AMBARI-17634
https://issues.apache.org/jira/browse/AMBARI-17634


Repository: ambari


Description
---

in Execute environment field is set env variable for agent user, so its not 
working right with non-root install


Diffs
-

  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/package/scripts/logfeeder.py
 9567db2 
  ambari-server/src/test/python/stacks/2.4/LOGSEARCH/test_logfeeder.py 78e2c5c 

Diff: https://reviews.apache.org/r/49834/diff/


Testing
---

Total run:1016
Total errors:0
Total failures:0


Thanks,

Oliver Szabo

Review Request 49809: Ambari server log flooded with error messages related to LogSearch service

[Miklos Gergely]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49809/
---

Review request for Ambari, Oliver Szabo, Robert Nettleton, and Sumit Mohanty.


Bugs: AMBARI-17625
https://issues.apache.org/jira/browse/AMBARI-17625


Repository: ambari


Description
---

ambari-server.log contains entries like this if Log Search is not installed:

06 Jul 2016 19:26:28,798 ERROR [ambari-client-thread-2843] 
LoggingSearchPropertyProvider:59 - Error occurred while making request to 
LogSearch service, unable to populate logging properties on this resource


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProvider.java
 a28e04a 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProviderTest.java
 8b71b65 

Diff: https://reviews.apache.org/r/49809/diff/


Testing
---

Tested on local cluster


Thanks,

Miklos Gergely

Re: Review Request 49387: Support password type for custom properties

[Keta Patel]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/
---

(Updated July 8, 2016, 9:15 p.m.)


Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
Jonathan Hurley.


Bugs: AMBARI-17041
https://issues.apache.org/jira/browse/AMBARI-17041


Repository: ambari


Description
---

Currently, services can define properties in the XML configuration files that 
is flagged as type password:
  
my.special.password

PASSWORD
Password to be masked
 
and it will be masked properly in the UI as well as blueprint.

Custom property should also support this option so that password can be added 
as custom property and treat accordingly.


==
Proposed Design for the fix:
==
At present only the key-value information of the service properties is stored 
in the DB ("clusterconfig" table in the "config_data" column). 
The "config_attributes" column stores only certain attributes like "final" 
indicating the list of properties set with the Final flag = true. 
The information about the property-type (i.e PASSWORD, USER, GROUP, 
ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, etc) 
is extracted from the corresponding service's property file (e.g. 
hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
information of the existing properties only. Custom Properties added by ambari 
user have no provision to store their additional attributes. 

Since, for this Jira we are concerned with only  attribute for 
Custom Properties, we could add an additional field called "Property Type" in 
the "Add Property" pop-up which shows up on clicking "Add Property ..." in the 
Custom property section for a service. For now, only 2 options are shown in the 
drop-down list: NONE and PASSWORD .
A few sample test properties are created using the new "Add Property" pop-up as 
can be seen in the following attachments. 
Attachments: 
"add_property_pop_up.tiff"
"custom_property_password_type.tiff"
"custom_property_regular_type.tiff"
"custom_properties_after_save.tiff"

The  information for these Custom properties is stored in the DB 
in "clusterconfig" table, "config_attributes" column.
The schema for "clusterconfig" table can be seen in the attachment:
"schema_of_clusterconfig_table.tiff"

The content of the "config_attributes" column with the  
information from the new Custom properties can be seen in the attachment:
"cluster_config_with_password_type_in_config_attributes_column.tiff"


Note: The fix so far is performed only for new Custom properties. The 
 information for existing properties is extracted from the 
corresponding property xml files for the service.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationResponse.java
 eef3474 
  
ambari-server/src/main/java/org/apache/ambari/server/utils/SecretReference.java 
84f3109 
  ambari-web/app/messages.js bbac1bd 
  ambari-web/app/mixins/common/configs/configs_saver.js 43a77ce 
  ambari-web/app/models/configs/objects/service_config_property.js 861d599 
  ambari-web/app/templates/common/configs/addPropertyWindow.hbs 659435c 
  ambari-web/app/utils/config.js 6c6e7f6 
  ambari-web/app/views/common/configs/service_configs_by_category_view.js 
bdd0530 
  ambari-web/test/mixins/common/configs/configs_saver_test.js 4baf303 

Diff: https://reviews.apache.org/r/49387/diff/


Testing (updated)
---

No new test cases are written in the patch apart from the fix required in 
"configs_saver_test.js" to avoid test failure of existing tests.
If the proposed solution is acceptable, I will create a new patch with the 
necessary new tests.

The existing ambari-web tests after applying the patch:

  28977 tests complete (37 seconds)
  154 tests pending
  
  
With the new latest patch "AMBARI-17041-trunk-July08.patch" the following is 
result of ambari-web tests:

  29017 tests complete (48 seconds)
  154 tests pending


Thanks,

Keta Patel

Review Request 49831: AMBARI-17562 - Adding single stack, extension and service should be removed from management pack support

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49831/
---

Review request for Ambari, Alejandro Fernandez, Jayush Luniya, Mahadev Konar, 
and Sumit Mohanty.


Bugs: AMBARI-17562
https://issues.apache.org/jira/browse/AMBARI-17562


Repository: ambari


Description
---

This code is duplicated by the ability to add multiple stacks, extensions and 
addon services. There is no reason to maintain both code paths.


Diffs
-

  ambari-server/src/main/python/ambari_server/setupMpacks.py 5acc92c 
  ambari-server/src/test/python/TestMpacks.py 0995280 
  
ambari-server/src/test/python/mpacks/myservice-ambari-mpack-1.0.0.0/mpack.json 
0af6949 

Diff: https://reviews.apache.org/r/49831/diff/


Testing
---

mvn clean test -DskipSurefireTests

No errors related to the mpack tests


Thanks,

Tim Thorpe

Re: Review Request 48972: AMBARI-17253 Ambari Alert causes too many wanings in ZooKeeper logs.

[Masahiro Tanaka]

> On July 8, 2016, 8:45 p.m., Nate Cole wrote:
> > ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py, lines 68-71
> > 
> >
> > We use underscore notation in our json, not dotted names.  Match for 
> > consistency.
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing! I checked other `alerts.json` which uses 
> `parameters` attributes, and they use dotted name (see 
> [this](https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/alerts.json#L194))
> 
> Should I use underscore? (Like, `socket_command`)?
> 
> Nate Cole wrote:
> Bah! :)  Ok, I guess dotted is fine.  We are supposed to keep things 
> consistent, but I guess not everyone knows that.  You can drop the issue.  
> Just make sure Jonathan reviews before pushing.

OK, I'll wait Jonathan review. Thanks again!


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48972/#review141332
---


On July 8, 2016, 8:20 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48972/
> ---
> 
> (Updated July 8, 2016, 8:20 p.m.)
> 
> 
> Review request for Ambari, Florian Barca, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-17253
> https://issues.apache.org/jira/browse/AMBARI-17253
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are too many WARNING in ZooKeeper log.
> ```
> 2016-06-15 21:02:15,405 - WARN  
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of 
> stream exception
> EndOfStreamException: Unable to read additional data from client sessionid 
> 0x0, likely client has closed socket
> at 
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> at java.lang.Thread.run(Thread.java:745)
> ```
> 
> It may be because of Ambari Alert. Ambari Alert pings to the zookeeper port 
> to do monitoring.
> We should use 'ruok' to monitor zookeepers.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 1918327 
>   ambari-agent/src/test/python/ambari_agent/TestPortAlert.py dffa56c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java
>  d7279de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7ef12a7 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/alerts.json 
> 469036a 
> 
> Diff: https://reviews.apache.org/r/48972/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> ```
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12811835/AMBARI-17253.2.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in .
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//console
> This message is automatically generated.
> ```
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49387: Support password type for custom properties

[Keta Patel]

> On July 8, 2016, 2:12 p.m., Di Li wrote:
> > please add Jonathan Hurley to the review

Thank you Di. I have added him.


- Keta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/#review141280
---


On July 8, 2016, 9:05 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49387/
> ---
> 
> (Updated July 8, 2016, 9:05 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-17041
> https://issues.apache.org/jira/browse/AMBARI-17041
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, services can define properties in the XML configuration files that 
> is flagged as type password:
>   
> my.special.password
> 
> PASSWORD
> Password to be masked
>  
> and it will be masked properly in the UI as well as blueprint.
> 
> Custom property should also support this option so that password can be added 
> as custom property and treat accordingly.
> 
> 
> ==
> Proposed Design for the fix:
> ==
> At present only the key-value information of the service properties is stored 
> in the DB ("clusterconfig" table in the "config_data" column). 
> The "config_attributes" column stores only certain attributes like "final" 
> indicating the list of properties set with the Final flag = true. 
> The information about the property-type (i.e PASSWORD, USER, GROUP, 
> ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, 
> etc) is extracted from the corresponding service's property file (e.g. 
> hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
> information of the existing properties only. Custom Properties added by 
> ambari user have no provision to store their additional attributes. 
> 
> Since, for this Jira we are concerned with only  attribute for 
> Custom Properties, we could add an additional field called "Property Type" in 
> the "Add Property" pop-up which shows up on clicking "Add Property ..." in 
> the Custom property section for a service. For now, only 2 options are shown 
> in the drop-down list: NONE and PASSWORD .
> A few sample test properties are created using the new "Add Property" pop-up 
> as can be seen in the following attachments. 
> Attachments: 
> "add_property_pop_up.tiff"
> "custom_property_password_type.tiff"
> "custom_property_regular_type.tiff"
> "custom_properties_after_save.tiff"
> 
> The  information for these Custom properties is stored in the 
> DB in "clusterconfig" table, "config_attributes" column.
> The schema for "clusterconfig" table can be seen in the attachment:
> "schema_of_clusterconfig_table.tiff"
> 
> The content of the "config_attributes" column with the  
> information from the new Custom properties can be seen in the attachment:
> "cluster_config_with_password_type_in_config_attributes_column.tiff"
> 
> 
> Note: The fix so far is performed only for new Custom properties. The 
>  information for existing properties is extracted from the 
> corresponding property xml files for the service.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationResponse.java
>  eef3474 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/SecretReference.java
>  84f3109 
>   ambari-web/app/messages.js bbac1bd 
>   ambari-web/app/mixins/common/configs/configs_saver.js 43a77ce 
>   ambari-web/app/models/configs/objects/service_config_property.js 861d599 
>   ambari-web/app/templates/common/configs/addPropertyWindow.hbs 659435c 
>   ambari-web/app/utils/config.js 6c6e7f6 
>   ambari-web/app/views/common/configs/service_configs_by_category_view.js 
> bdd0530 
>   ambari-web/test/mixins/common/configs/configs_saver_test.js 4baf303 
> 
> Diff: https://reviews.apache.org/r/49387/diff/
> 
> 
> Testing
> ---
> 
> No new test cases are written in the patch apart from the fix required in 
> "configs_saver_test.js" to avoid test failure of existing tests.
> If the proposed solution is acceptable, I will create a new patch with the 
> necessary new tests.
> 
> The existing ambari-web tests after applying the patch:
> 
>   28977 tests complete (37 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

Re: Review Request 49387: Support password type for custom properties

[Keta Patel]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/
---

(Updated July 8, 2016, 9:05 p.m.)


Review request for Ambari, Alexandr Antonenko, Di Li, Dmitro Lisnichenko, and 
Jonathan Hurley.


Changes
---

Updated the patch to allow multiple selection of 


Bugs: AMBARI-17041
https://issues.apache.org/jira/browse/AMBARI-17041


Repository: ambari


Description
---

Currently, services can define properties in the XML configuration files that 
is flagged as type password:
  
my.special.password

PASSWORD
Password to be masked
 
and it will be masked properly in the UI as well as blueprint.

Custom property should also support this option so that password can be added 
as custom property and treat accordingly.


==
Proposed Design for the fix:
==
At present only the key-value information of the service properties is stored 
in the DB ("clusterconfig" table in the "config_data" column). 
The "config_attributes" column stores only certain attributes like "final" 
indicating the list of properties set with the Final flag = true. 
The information about the property-type (i.e PASSWORD, USER, GROUP, 
ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, etc) 
is extracted from the corresponding service's property file (e.g. 
hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
information of the existing properties only. Custom Properties added by ambari 
user have no provision to store their additional attributes. 

Since, for this Jira we are concerned with only  attribute for 
Custom Properties, we could add an additional field called "Property Type" in 
the "Add Property" pop-up which shows up on clicking "Add Property ..." in the 
Custom property section for a service. For now, only 2 options are shown in the 
drop-down list: NONE and PASSWORD .
A few sample test properties are created using the new "Add Property" pop-up as 
can be seen in the following attachments. 
Attachments: 
"add_property_pop_up.tiff"
"custom_property_password_type.tiff"
"custom_property_regular_type.tiff"
"custom_properties_after_save.tiff"

The  information for these Custom properties is stored in the DB 
in "clusterconfig" table, "config_attributes" column.
The schema for "clusterconfig" table can be seen in the attachment:
"schema_of_clusterconfig_table.tiff"

The content of the "config_attributes" column with the  
information from the new Custom properties can be seen in the attachment:
"cluster_config_with_password_type_in_config_attributes_column.tiff"


Note: The fix so far is performed only for new Custom properties. The 
 information for existing properties is extracted from the 
corresponding property xml files for the service.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationResponse.java
 eef3474 
  
ambari-server/src/main/java/org/apache/ambari/server/utils/SecretReference.java 
84f3109 
  ambari-web/app/messages.js bbac1bd 
  ambari-web/app/mixins/common/configs/configs_saver.js 43a77ce 
  ambari-web/app/models/configs/objects/service_config_property.js 861d599 
  ambari-web/app/templates/common/configs/addPropertyWindow.hbs 659435c 
  ambari-web/app/utils/config.js 6c6e7f6 
  ambari-web/app/views/common/configs/service_configs_by_category_view.js 
bdd0530 
  ambari-web/test/mixins/common/configs/configs_saver_test.js 4baf303 

Diff: https://reviews.apache.org/r/49387/diff/


Testing
---

No new test cases are written in the patch apart from the fix required in 
"configs_saver_test.js" to avoid test failure of existing tests.
If the proposed solution is acceptable, I will create a new patch with the 
necessary new tests.

The existing ambari-web tests after applying the patch:

  28977 tests complete (37 seconds)
  154 tests pending


Thanks,

Keta Patel

Re: Review Request 48972: AMBARI-17253 Ambari Alert causes too many wanings in ZooKeeper logs.

[Nate Cole]

> On July 8, 2016, 4:45 p.m., Nate Cole wrote:
> > ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py, lines 68-71
> > 
> >
> > We use underscore notation in our json, not dotted names.  Match for 
> > consistency.
> 
> Masahiro Tanaka wrote:
> Thank you for reviewing! I checked other `alerts.json` which uses 
> `parameters` attributes, and they use dotted name (see 
> [this](https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/alerts.json#L194))
> 
> Should I use underscore? (Like, `socket_command`)?

Bah! :)  Ok, I guess dotted is fine.  We are supposed to keep things 
consistent, but I guess not everyone knows that.  You can drop the issue.  Just 
make sure Jonathan reviews before pushing.


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48972/#review141332
---


On July 8, 2016, 4:20 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48972/
> ---
> 
> (Updated July 8, 2016, 4:20 p.m.)
> 
> 
> Review request for Ambari, Florian Barca, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-17253
> https://issues.apache.org/jira/browse/AMBARI-17253
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are too many WARNING in ZooKeeper log.
> ```
> 2016-06-15 21:02:15,405 - WARN  
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of 
> stream exception
> EndOfStreamException: Unable to read additional data from client sessionid 
> 0x0, likely client has closed socket
> at 
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> at java.lang.Thread.run(Thread.java:745)
> ```
> 
> It may be because of Ambari Alert. Ambari Alert pings to the zookeeper port 
> to do monitoring.
> We should use 'ruok' to monitor zookeepers.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 1918327 
>   ambari-agent/src/test/python/ambari_agent/TestPortAlert.py dffa56c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java
>  d7279de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7ef12a7 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/alerts.json 
> 469036a 
> 
> Diff: https://reviews.apache.org/r/48972/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> ```
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12811835/AMBARI-17253.2.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in .
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//console
> This message is automatically generated.
> ```
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 48972: AMBARI-17253 Ambari Alert causes too many wanings in ZooKeeper logs.

[Masahiro Tanaka]

> On July 8, 2016, 8:45 p.m., Nate Cole wrote:
> > ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py, lines 68-71
> > 
> >
> > We use underscore notation in our json, not dotted names.  Match for 
> > consistency.

Thank you for reviewing! I checked other `alerts.json` which uses `parameters` 
attributes, and they use dotted name (see 
[this](https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/alerts.json#L194))

Should I use underscore? (Like, `socket_command`)?


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48972/#review141332
---


On July 8, 2016, 8:20 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48972/
> ---
> 
> (Updated July 8, 2016, 8:20 p.m.)
> 
> 
> Review request for Ambari, Florian Barca, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-17253
> https://issues.apache.org/jira/browse/AMBARI-17253
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are too many WARNING in ZooKeeper log.
> ```
> 2016-06-15 21:02:15,405 - WARN  
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of 
> stream exception
> EndOfStreamException: Unable to read additional data from client sessionid 
> 0x0, likely client has closed socket
> at 
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> at java.lang.Thread.run(Thread.java:745)
> ```
> 
> It may be because of Ambari Alert. Ambari Alert pings to the zookeeper port 
> to do monitoring.
> We should use 'ruok' to monitor zookeepers.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 1918327 
>   ambari-agent/src/test/python/ambari_agent/TestPortAlert.py dffa56c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java
>  d7279de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7ef12a7 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/alerts.json 
> 469036a 
> 
> Diff: https://reviews.apache.org/r/48972/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> ```
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12811835/AMBARI-17253.2.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in .
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//console
> This message is automatically generated.
> ```
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 48972: AMBARI-17253 Ambari Alert causes too many wanings in ZooKeeper logs.

[Nate Cole]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48972/#review141332
---


Fix it, then Ship it!




Make sure Jonathan reviews this before pushing it.


ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py (lines 68 - 71)


We use underscore notation in our json, not dotted names.  Match for 
consistency.


- Nate Cole


On July 8, 2016, 4:20 p.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48972/
> ---
> 
> (Updated July 8, 2016, 4:20 p.m.)
> 
> 
> Review request for Ambari, Florian Barca, Jonathan Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-17253
> https://issues.apache.org/jira/browse/AMBARI-17253
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There are too many WARNING in ZooKeeper log.
> ```
> 2016-06-15 21:02:15,405 - WARN  
> [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of 
> stream exception
> EndOfStreamException: Unable to read additional data from client sessionid 
> 0x0, likely client has closed socket
> at 
> org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
> at 
> org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> at java.lang.Thread.run(Thread.java:745)
> ```
> 
> It may be because of Ambari Alert. Ambari Alert pings to the zookeeper port 
> to do monitoring.
> We should use 'ruok' to monitor zookeepers.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 1918327 
>   ambari-agent/src/test/python/ambari_agent/TestPortAlert.py dffa56c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java
>  d7279de 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  7ef12a7 
>   
> ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/alerts.json 
> 469036a 
> 
> Diff: https://reviews.apache.org/r/48972/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> ```
> +1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12811835/AMBARI-17253.2.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> +1 tests included. The patch appears to include 1 new or modified test files.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in .
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/7427//console
> This message is automatically generated.
> ```
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 49721: AMBARI-17580: Unable to add MASTER component on all nodes

[Jaimin Jetly]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49721/#review141328
---


Ship it!




Ship It!

- Jaimin Jetly


On July 6, 2016, 7:20 p.m., Shantanu Mundkur wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49721/
> ---
> 
> (Updated July 6, 2016, 7:20 p.m.)
> 
> 
> Review request for Ambari and Jaimin Jetly.
> 
> 
> Bugs: AMBARI-17580
> https://issues.apache.org/jira/browse/AMBARI-17580
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-17580: Unable to add MASTER component on all nodes
> 
> 
> While trying to add a MASTER component with cardinality ALL:
> 
> org.apache.ambari.server.controller.spi.ResourceAlreadyExistsException: 
> Attempted to create a host_component which already exists: [clusterName=HDP, 
> hostName=node1.mydomain.com, componentName=SERVICE_MASTER]
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step8_controller.js 3971cf5 
>   ambari-web/test/controllers/wizard/step8_test.js 74e042b 
> 
> Diff: https://reviews.apache.org/r/49721/diff/
> 
> 
> Testing
> ---
> 
> 1) Tried to add a custom service with cardinality as ALL for a MASTER 
> component (that was previously the failing case).
> 
> 2) Unit tests:
> Under ambari-web:
> mvn clean test
> 
> Results:
> 28946 tests complete (37 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Shantanu Mundkur
> 
>

Re: Review Request 48972: AMBARI-17253 Ambari Alert causes too many wanings in ZooKeeper logs.

[Masahiro Tanaka]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48972/
---

(Updated July 8, 2016, 8:20 p.m.)


Review request for Ambari, Florian Barca, Jonathan Hurley, and Nate Cole.


Changes
---

Update the patch
* Check response message
* Dealing with Ambari Upgrade


Bugs: AMBARI-17253
https://issues.apache.org/jira/browse/AMBARI-17253


Repository: ambari


Description
---

There are too many WARNING in ZooKeeper log.
```
2016-06-15 21:02:15,405 - WARN  
[NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@357] - caught end of 
stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x0, 
likely client has closed socket
at 
org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at 
org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
```

It may be because of Ambari Alert. Ambari Alert pings to the zookeeper port to 
do monitoring.
We should use 'ruok' to monitor zookeepers.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/alerts/port_alert.py 1918327 
  ambari-agent/src/test/python/ambari_agent/TestPortAlert.py dffa56c 
  
ambari-server/src/main/java/org/apache/ambari/server/state/alert/PortSource.java
 d7279de 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 7ef12a7 
  ambari-server/src/main/resources/common-services/ZOOKEEPER/3.4.5/alerts.json 
469036a 

Diff: https://reviews.apache.org/r/48972/diff/


Testing
---

mvn clean test

```
+1 overall. Here are the results of testing the latest attachment 
http://issues.apache.org/jira/secure/attachment/12811835/AMBARI-17253.2.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 1 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac 
compiler warnings.
+1 release audit. The applied patch does not increase the total number of 
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7427//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7427//console
This message is automatically generated.
```


Thanks,

Masahiro Tanaka

Review Request 49826: AUTH_TO_LOCAL rules are not updated when adding services to a Blueprint-installed cluster

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49826/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sebastian Toader.


Bugs: AMBARI-17629
https://issues.apache.org/jira/browse/AMBARI-17629


Repository: ambari


Description
---

When adding new services and components to a cluster that was initially created 
via Blueprints (rather than via the Ambari UI), auth-to-local rules that are 
expected to be created as indicated by the Kerberos descriptor are not.

It occurs because the components being installed are in the `INIT` state where 
the logic to determine whether to include the new auth-to-local rules or not 
expects the components to be in either the `INSTALLED` or `STARTED` states. 
This is due to logic added when resolving AMBARI-14232.

#Solution:
Allow for auth-to-local rules for new services and components to be added when 
the state of the components are  `INIT` as well as `INSTALLED` and `STARTED`, 
when the cluster was installed via Blueprints.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelperImpl.java
 cc48240 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
 66ed68d 

Diff: https://reviews.apache.org/r/49826/diff/


Testing
---

Manually tested..

# Local unit tests: 
```
Running org.apache.ambari.server.controller.KerberosHelperTest
Tests run: 63, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.402 sec - in 
org.apache.ambari.server.controller.KerberosHelperTest
```

Previous existing and unrelated unit test error:
``` 
Running org.apache.ambari.server.state.ServicePropertiesTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.505 sec <<< 
FAILURE! - in org.apache.ambari.server.state.ServicePropertiesTest
validatePropertySchemaOfServiceXMLs(org.apache.ambari.server.state.ServicePropertiesTest)
  Time elapsed: 0.504 sec  <<< ERROR!
org.apache.ambari.server.AmbariException: File 
/Users/rlevas/git/ambari/ambari-server/target/test-classes/TestAmbaryServer.samples/../../../src/main/resources/common-services/RANGER/0.4.0/configuration/admin-properties.xml
 didn't pass the validation. Error message is : cvc-complex-type.3.2.2: 
Attribute 'update' is not allowed to appear in element 'on-ambari-upgrade'.
at 
org.apache.ambari.server.state.ServicePropertiesTest.validatePropertySchemaOfServiceXMLs(ServicePropertiesTest.java:48)
```


# Jenkins unit tests: PENDING


Thanks,

Robert Levas

Re: Review Request 49328: AMBARI-17465 - Management packs should be able to install extensions

[Jayush Luniya]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49328/#review141290
---


Ship it!




Ship It!

- Jayush Luniya


On July 8, 2016, 2:43 p.m., Tim Thorpe wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49328/
> ---
> 
> (Updated July 8, 2016, 2:43 p.m.)
> 
> 
> Review request for Ambari, Jayush Luniya, Mahadev Konar, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17465
> https://issues.apache.org/jira/browse/AMBARI-17465
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently management packs (AMBARI-14854) can only add stacks and addon 
> services. Now that AMBARI-12885 has been resolved, the management packs 
> should be able to add extensions as well.
> 
> 
> Diffs
> -
> 
>   ambari-server/conf/unix/ambari.properties abd72ad 
>   ambari-server/src/main/python/ambari_server/serverConfiguration.py e868f96 
>   ambari-server/src/main/python/ambari_server/setupMpacks.py 98811f5 
>   ambari-server/src/test/python/TestMpacks.py a7c6092 
>   
> ambari-server/src/test/python/mpacks/myextension-ambari-mpack-1.0.0.0/extensions/MYEXTENSION/1.0/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/python/mpacks/myextension-ambari-mpack-1.0.0.0/extensions/MYEXTENSION/1.1/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/test/python/mpacks/myextension-ambari-mpack-1.0.0.0/mpack.json
>  PRE-CREATION 
>   
> ambari-server/src/test/python/mpacks/myservice-ambari-mpack-1.0.0.0/mpack.json
>  faade4a 
> 
> Diff: https://reviews.apache.org/r/49328/diff/
> 
> 
> Testing
> ---
> 
> Manual testing
> 
> ambari-server install-mpack 
> --mpack=/root/mpacks/hdp-ambari-mpack-1.0.0.0.tar.gz --purge -v
> ambari-server install-mpack 
> --mpack=/root/mpacks/myservice-ambari-mpack-1.0.0.0.tar.gz -v
> ambari-server install-mpack 
> --mpack=/root/mpacks/myservice2-ambari-mpack-1.0.0.0.tar.gz -v
> ambari-server install-mpack --mpack=/root/mpacks/myextension-1.0.0.0.tar.gz -v
> 
> 
> Thanks,
> 
> Tim Thorpe
> 
>

Review Request 49808: Host service mapping has incorrect default mapping

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49808/
---

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-17624
https://issues.apache.org/jira/browse/AMBARI-17624


Repository: ambari


Description
---

Activity Analyzer and Explorer are salves with cardinality
[0-1].  
However the default selection maps to all hosts. This used to work fine in
previous builds, but seems to be broken recently

Ambari version:



[root@cn030 ~]# rpm -qa | grep ambari
ambari-agent-2.4.0.0-764.x86_64
ambari-server-2.4.0.0-764.x86_64


Diffs
-

  ambari-server/src/main/resources/stacks/stack_advisor.py 23c0320 
  ambari-server/src/test/python/stacks/2.0.6/common/test_stack_advisor.py 
931d59b 

Diff: https://reviews.apache.org/r/49808/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49328: AMBARI-17465 - Management packs should be able to install extensions

[Tim Thorpe]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49328/
---

(Updated July 8, 2016, 2:43 p.m.)


Review request for Ambari, Jayush Luniya, Mahadev Konar, and Sumit Mohanty.


Changes
---

Added and updated unit tests


Bugs: AMBARI-17465
https://issues.apache.org/jira/browse/AMBARI-17465


Repository: ambari


Description
---

Currently management packs (AMBARI-14854) can only add stacks and addon 
services. Now that AMBARI-12885 has been resolved, the management packs should 
be able to add extensions as well.


Diffs (updated)
-

  ambari-server/conf/unix/ambari.properties abd72ad 
  ambari-server/src/main/python/ambari_server/serverConfiguration.py e868f96 
  ambari-server/src/main/python/ambari_server/setupMpacks.py 98811f5 
  ambari-server/src/test/python/TestMpacks.py a7c6092 
  
ambari-server/src/test/python/mpacks/myextension-ambari-mpack-1.0.0.0/extensions/MYEXTENSION/1.0/metainfo.xml
 PRE-CREATION 
  
ambari-server/src/test/python/mpacks/myextension-ambari-mpack-1.0.0.0/extensions/MYEXTENSION/1.1/metainfo.xml
 PRE-CREATION 
  
ambari-server/src/test/python/mpacks/myextension-ambari-mpack-1.0.0.0/mpack.json
 PRE-CREATION 
  
ambari-server/src/test/python/mpacks/myservice-ambari-mpack-1.0.0.0/mpack.json 
faade4a 

Diff: https://reviews.apache.org/r/49328/diff/


Testing
---

Manual testing

ambari-server install-mpack 
--mpack=/root/mpacks/hdp-ambari-mpack-1.0.0.0.tar.gz --purge -v
ambari-server install-mpack 
--mpack=/root/mpacks/myservice-ambari-mpack-1.0.0.0.tar.gz -v
ambari-server install-mpack 
--mpack=/root/mpacks/myservice2-ambari-mpack-1.0.0.0.tar.gz -v
ambari-server install-mpack --mpack=/root/mpacks/myextension-1.0.0.0.tar.gz -v


Thanks,

Tim Thorpe

Re: Review Request 49387: Support password type for custom properties

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49387/#review141280
---



please add Jonathan Hurley to the review

- Di Li


On June 29, 2016, 9:05 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49387/
> ---
> 
> (Updated June 29, 2016, 9:05 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Di Li, and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17041
> https://issues.apache.org/jira/browse/AMBARI-17041
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently, services can define properties in the XML configuration files that 
> is flagged as type password:
>   
> my.special.password
> 
> PASSWORD
> Password to be masked
>  
> and it will be masked properly in the UI as well as blueprint.
> 
> Custom property should also support this option so that password can be added 
> as custom property and treat accordingly.
> 
> 
> ==
> Proposed Design for the fix:
> ==
> At present only the key-value information of the service properties is stored 
> in the DB ("clusterconfig" table in the "config_data" column). 
> The "config_attributes" column stores only certain attributes like "final" 
> indicating the list of properties set with the Final flag = true. 
> The information about the property-type (i.e PASSWORD, USER, GROUP, 
> ADDITIONAL_USER_PROPERTY, VALUE_FROM_PROPERTY_FILE, NOT_MANAGED_HDFS_PATH, 
> etc) is extracted from the corresponding service's property file (e.g. 
> hive-site.xml, core-site.xml, webhcat-env.xml, etc). These files contain 
> information of the existing properties only. Custom Properties added by 
> ambari user have no provision to store their additional attributes. 
> 
> Since, for this Jira we are concerned with only  attribute for 
> Custom Properties, we could add an additional field called "Property Type" in 
> the "Add Property" pop-up which shows up on clicking "Add Property ..." in 
> the Custom property section for a service. For now, only 2 options are shown 
> in the drop-down list: NONE and PASSWORD .
> A few sample test properties are created using the new "Add Property" pop-up 
> as can be seen in the following attachments. 
> Attachments: 
> "add_property_pop_up.tiff"
> "custom_property_password_type.tiff"
> "custom_property_regular_type.tiff"
> "custom_properties_after_save.tiff"
> 
> The  information for these Custom properties is stored in the 
> DB in "clusterconfig" table, "config_attributes" column.
> The schema for "clusterconfig" table can be seen in the attachment:
> "schema_of_clusterconfig_table.tiff"
> 
> The content of the "config_attributes" column with the  
> information from the new Custom properties can be seen in the attachment:
> "cluster_config_with_password_type_in_config_attributes_column.tiff"
> 
> 
> Note: The fix so far is performed only for new Custom properties. The 
>  information for existing properties is extracted from the 
> corresponding property xml files for the service.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ConfigurationResponse.java
>  eef3474 
>   
> ambari-server/src/main/java/org/apache/ambari/server/utils/SecretReference.java
>  84f3109 
>   ambari-web/app/messages.js d40a84a 
>   ambari-web/app/mixins/common/configs/configs_saver.js 43a77ce 
>   ambari-web/app/models/configs/objects/service_config_property.js 73960b3 
>   ambari-web/app/templates/common/configs/addPropertyWindow.hbs 659435c 
>   ambari-web/app/utils/config.js a9cb5e9 
>   ambari-web/app/views/common/configs/service_configs_by_category_view.js 
> bdd0530 
>   ambari-web/test/mixins/common/configs/configs_saver_test.js 4baf303 
> 
> Diff: https://reviews.apache.org/r/49387/diff/
> 
> 
> Testing
> ---
> 
> No new test cases are written in the patch apart from the fix required in 
> "configs_saver_test.js" to avoid test failure of existing tests.
> If the proposed solution is acceptable, I will create a new patch with the 
> necessary new tests.
> 
> The existing ambari-web tests after applying the patch:
> 
>   28977 tests complete (37 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

Re: Review Request 49801: Syntax error in 'setup_ranger_xml.py' logging.

[Andrew Onischuk]

> On July 8, 2016, 11:06 a.m., Sebastian Toader wrote:
> > ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py,
> >  line 289
> > 
> >
> > I'd replace {{db_name}} in the error message with the value of 
> > {db_flavor}

Done


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49801/#review141270
---


On July 8, 2016, 11:56 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49801/
> ---
> 
> (Updated July 8, 2016, 11:56 a.m.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-17620
> https://issues.apache.org/jira/browse/AMBARI-17620
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> File "/var/lib/ambari-agent/cache/common-
> services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 291, in
> copy_jdbc_connector  
> Please run 'ambari-server setup --jdbc-db=
> 
> {db_name}
> 
> \--jdbc-driver=
> 
> {path_to_jdbc}
> 
> on server host.'".format(params.db_flavor, params.jdk_location)  
> KeyError: 'db_name'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  9dbf6d4 
> 
> Diff: https://reviews.apache.org/r/49801/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49801: Syntax error in 'setup_ranger_xml.py' logging.

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49801/
---

(Updated July 8, 2016, 11:56 a.m.)


Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-17620
https://issues.apache.org/jira/browse/AMBARI-17620


Repository: ambari


Description
---

File "/var/lib/ambari-agent/cache/common-
services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 291, in
copy_jdbc_connector  
Please run 'ambari-server setup --jdbc-db=

{db_name}

\--jdbc-driver=

{path_to_jdbc}

on server host.'".format(params.db_flavor, params.jdk_location)  
KeyError: 'db_name'


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 9dbf6d4 

Diff: https://reviews.apache.org/r/49801/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49783: Multiple fixes for Atlas integration in versions 0.1.0 and 0.7.0

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49783/#review141274
---


Ship it!




Ship It!

- Robert Levas


On July 7, 2016, 10:59 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49783/
> ---
> 
> (Updated July 7, 2016, 10:59 p.m.)
> 
> 
> Review request for Ambari, Madhan Neethiraj, Sumit Mohanty, and Suma 
> Shivaprasad.
> 
> 
> Bugs: AMBARI-17616
> https://issues.apache.org/jira/browse/AMBARI-17616
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This Jira contains several fixes.
> 1. Separate Atlas versions 0.1.0 (HDP 2.3 and 2.4), and 0.7.0 (HDP 2.5) for 
> Atlas.
> This way, all of the LDAP/AD changes only affect HDP 2.5.
> This means that HDP 2.5 will have the smart configs theme.
> 
> 2. Allow atlas.authentication.method.ldap.type to contain the value "none"
> If the value is none, then do not require any of the LDAP or AD configs.
> Stack advisor must support this change.
> 
> 
> 3. For the LDAP and AD configs that are required, do not allow an empty 
> value. Only these 4 LDAP/AD properties can be optional:
> atlas.authentication.method.ldap.referral
> atlas.authentication.method.ldap.default.role
> atlas.authentication.method.ldap.ad.referral
> atlas.authentication.method.ldap.ad.default.role
> 
> 
> 4. Set all of the configs in atlas-application.properties to have,
> 
> ```
> 
> ```
> 
> 
> 5. When generating the file atlas-application.properties.xml for Hive, 
> Falcon, Storm, and Sqoop, only copy a subset of the configs from Atlas' 
> application.properties, these configs are
> 
> ```
>   "atlas.kafka.zookeeper.connect",
>   "atlas.kafka.bootstrap.servers",
>   "atlas.kafka.zookeeper.session.timeout.ms",
>   "atlas.kafka.zookeeper.connection.timeout.ms",
>   "atlas.kafka.zookeeper.sync.time.ms",
>   "atlas.kafka.hook.group.id",
>   "atlas.notification.create.topics",
>   "atlas.notification.replicas",
>   "atlas.notification.topics",
>   "atlas.notification.kafka.service.principal",
>   "atlas.notification.kafka.keytab.location",
>   "atlas.jaas.KafkaClient.loginModuleName",
>   "atlas.jaas.KafkaClient.loginModuleControlFlag",
>   "atlas.jaas.KafkaClient.option.useKeyTab",
>   "atlas.jaas.KafkaClient.option.storeKey",
>   "atlas.jaas.KafkaClient.option.serviceName",
>   "atlas.jaas.KafkaClient.option.keyTab",
>   "atlas.jaas.KafkaClient.option.principal",
>   "atlas.cluster.name"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/setup_atlas_hook.py
>  872147a 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/configuration/application-properties.xml
>  b1b945d 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/metainfo.xml 
> b2fc143 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/themes/theme.json
>  414633e 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/configuration/application-properties.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/metainfo.xml 
> PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/quicklinks/quicklinks.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/themes/theme.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  57b31a7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/kerberos.json 
> fd05edf 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/metainfo.xml 
> 020e339 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/quicklinks/quicklinks.json
>  ce266c4 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
> 1d092cd 
> 
> Diff: https://reviews.apache.org/r/49783/diff/
> 
> 
> Testing
> ---
> 
> Verified that could install Atlas on a cluster.
> Python unit tests passed.
> --
> Total run:1015
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 49805: Ambari server schema upgrade failed from 2.1.1 with ORA-12516 error

[Myroslav Papirkovskyy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49805/#review141273
---


Ship it!




Ship It!

- Myroslav Papirkovskyy


On Липень 8, 2016, 2:25 після полудня, Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49805/
> ---
> 
> (Updated Липень 8, 2016, 2:25 після полудня)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, and 
> Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-17622
> https://issues.apache.org/jira/browse/AMBARI-17622
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Steps
> Deploy HDP-2.3.0.0 cluster with Ambari 2.1.1 (Oracle DB, unsecure, HA 
> cluster). Note - the DB instance is dedicated for this cluster and Ambari 
> server and not shared
> Upgrade Ambari to 2.4.0.0-726.x86_64
> Result
> Error during Ambari DB schema upgrade. Logs show the below
> 
> 01 Jul 2016 11:47:17,686 ERROR [main] DBAccessorImpl:107 - Error while 
> creating database accessor 
> java.sql.SQLException: Listener refused the connection with the following 
> error:
> ORA-12516, TNS:listener could not find available handler with matching 
> protocol stack
>  
>   at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:419)
>   at 
> oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:536)
>   at oracle.jdbc.driver.T4CConnection.(T4CConnection.java:228)
>   at 
> oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
>   at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
>   at java.sql.DriverManager.getConnection(DriverManager.java:571)
>   at java.sql.DriverManager.getConnection(DriverManager.java:215)
>   at 
> org.apache.ambari.server.orm.DBAccessorImpl.(DBAccessorImpl.java:87)
> 01 Jul 2016 11:47:17,697 ERROR [main] DBAccessorImpl:107 - Error while 
> creating database accessor 
> java.sql.SQLException: Listener refused the connection with the following 
> error:
> ORA-12516, TNS:listener could not find available handler with matching 
> protocol stack
>  
>   at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:419)
>   at 
> oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:536)
> 01 Jul 2016 11:47:17,699 ERROR [main] SchemaUpgradeHelper:369 - Unexpected 
> error, upgrade failed
> com.google.inject.ProvisionException: Guice provision errors:
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java 
> b10d32a 
> 
> Diff: https://reviews.apache.org/r/49805/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 49805: Ambari server schema upgrade failed from 2.1.1 with ORA-12516 error

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49805/
---

Review request for Ambari, Dmitro Lisnichenko, Myroslav Papirkovskyy, and 
Vitalyi Brodetskyi.


Bugs: AMBARI-17622
https://issues.apache.org/jira/browse/AMBARI-17622


Repository: ambari


Description
---

Steps
Deploy HDP-2.3.0.0 cluster with Ambari 2.1.1 (Oracle DB, unsecure, HA cluster). 
Note - the DB instance is dedicated for this cluster and Ambari server and not 
shared
Upgrade Ambari to 2.4.0.0-726.x86_64
Result
Error during Ambari DB schema upgrade. Logs show the below

01 Jul 2016 11:47:17,686 ERROR [main] DBAccessorImpl:107 - Error while creating 
database accessor 
java.sql.SQLException: Listener refused the connection with the following error:
ORA-12516, TNS:listener could not find available handler with matching protocol 
stack
 
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:419)
at 
oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:536)
at oracle.jdbc.driver.T4CConnection.(T4CConnection.java:228)
at 
oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:32)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:521)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at 
org.apache.ambari.server.orm.DBAccessorImpl.(DBAccessorImpl.java:87)
01 Jul 2016 11:47:17,697 ERROR [main] DBAccessorImpl:107 - Error while creating 
database accessor 
java.sql.SQLException: Listener refused the connection with the following error:
ORA-12516, TNS:listener could not find available handler with matching protocol 
stack
 
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:419)
at 
oracle.jdbc.driver.PhysicalConnection.(PhysicalConnection.java:536)
01 Jul 2016 11:47:17,699 ERROR [main] SchemaUpgradeHelper:369 - Unexpected 
error, upgrade failed
com.google.inject.ProvisionException: Guice provision errors:


Diffs
-

  ambari-server/src/main/java/org/apache/ambari/server/orm/DBAccessorImpl.java 
b10d32a 

Diff: https://reviews.apache.org/r/49805/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 49801: Syntax error in 'setup_ranger_xml.py' logging.

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49801/#review141271
---


Ship it!




Ship It!

- Sebastian Toader


On July 8, 2016, 12:30 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49801/
> ---
> 
> (Updated July 8, 2016, 12:30 p.m.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-17620
> https://issues.apache.org/jira/browse/AMBARI-17620
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> File "/var/lib/ambari-agent/cache/common-
> services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 291, in
> copy_jdbc_connector  
> Please run 'ambari-server setup --jdbc-db=
> 
> {db_name}
> 
> \--jdbc-driver=
> 
> {path_to_jdbc}
> 
> on server host.'".format(params.db_flavor, params.jdk_location)  
> KeyError: 'db_name'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  9dbf6d4 
> 
> Diff: https://reviews.apache.org/r/49801/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 49804: improve resourcemanager HA description

[wang yaoxin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49804/
---

Review request for Ambari.


Bugs: AMBARI-17518
https://issues.apache.org/jira/browse/AMBARI-17518


Repository: ambari


Description
---

improve the resourcemanager HA description


Diffs
-

  ambari-web/app/models/host_component.js 973b809 

Diff: https://reviews.apache.org/r/49804/diff/


Testing
---

done


Thanks,

wang yaoxin

Review Request 49802: Host Namenode Rebalance HDFS failed

[wang yaoxin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49802/
---

Review request for Ambari.


Bugs: AMBARI-17546
https://issues.apache.org/jira/browse/AMBARI-17546


Repository: ambari


Description
---

the host view ,excute namenode rebalance HDFS, json none lead fail


Diffs
-

  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/namenode.py
 df02311 

Diff: https://reviews.apache.org/r/49802/diff/


Testing
---

done


Thanks,

wang yaoxin

Re: Review Request 49803: Kerberized Solr support for Atlas

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49803/#review141268
---


Ship it!




Ship It!

- Dmytro Sen


On Июль 8, 2016, 10:32 д.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49803/
> ---
> 
> (Updated Июль 8, 2016, 10:32 д.п.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-17621
> https://issues.apache.org/jira/browse/AMBARI-17621
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently Atlas does not support kerberized Solr communication.  
> To make kerberized Solr client work:  
> 1\. Set `java.security.auth.login.config` property, which points to a jaas-
> file (with Client block)  
> 2\. Use Kerberos http client configurer.
> 
> This option should be bind to a new property (e.g.:
> "atlas.solr.kerberos.enable")
> 
> call this before creating CloudSolrClient instance: (most likely
> Solr5Index.java)
> 
> 
> 
> 
> boolean securityEnabled = 
> PropertiesUtil.getBooleanProperty("atlas.solr.kerberos.enable", false);
> if (securityEnabled) {
>   System.setProperty("java.security.auth.login.config", 
> "/etc/atlas/conf/atlas-jaas.conf");
>   HttpClientUtil.setConfigurer(new Krb5HttpClientConfigurer());
> }
> 
> 
> some useful documentation from Ranger:  
>  d+with+Kerberos+for+Ranger+0.5>
> 
> On ambari side: we should handle this property if kerberos is enabled. The new
> property should be added only if Solr is used for Atlas
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  57b31a7 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/kerberos.json 
> fd05edf 
> 
> Diff: https://reviews.apache.org/r/49803/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 49803: Kerberized Solr support for Atlas

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49803/
---

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-17621
https://issues.apache.org/jira/browse/AMBARI-17621


Repository: ambari


Description
---

Currently Atlas does not support kerberized Solr communication.  
To make kerberized Solr client work:  
1\. Set `java.security.auth.login.config` property, which points to a jaas-
file (with Client block)  
2\. Use Kerberos http client configurer.

This option should be bind to a new property (e.g.:
"atlas.solr.kerberos.enable")

call this before creating CloudSolrClient instance: (most likely
Solr5Index.java)




boolean securityEnabled = 
PropertiesUtil.getBooleanProperty("atlas.solr.kerberos.enable", false);
if (securityEnabled) {
  System.setProperty("java.security.auth.login.config", 
"/etc/atlas/conf/atlas-jaas.conf");
  HttpClientUtil.setConfigurer(new Krb5HttpClientConfigurer());
}


some useful documentation from Ranger:  


On ambari side: we should handle this property if kerberos is enabled. The new
property should be added only if Solr is used for Atlas


Diffs
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
 57b31a7 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/kerberos.json 
fd05edf 

Diff: https://reviews.apache.org/r/49803/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49801: Syntax error in 'setup_ranger_xml.py' logging.

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49801/#review141267
---


Ship it!




Ship It!

- Dmytro Sen


On Июль 8, 2016, 10:30 д.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49801/
> ---
> 
> (Updated Июль 8, 2016, 10:30 д.п.)
> 
> 
> Review request for Ambari and Dmytro Sen.
> 
> 
> Bugs: AMBARI-17620
> https://issues.apache.org/jira/browse/AMBARI-17620
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> File "/var/lib/ambari-agent/cache/common-
> services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 291, in
> copy_jdbc_connector  
> Please run 'ambari-server setup --jdbc-db=
> 
> {db_name}
> 
> \--jdbc-driver=
> 
> {path_to_jdbc}
> 
> on server host.'".format(params.db_flavor, params.jdk_location)  
> KeyError: 'db_name'
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  9dbf6d4 
> 
> Diff: https://reviews.apache.org/r/49801/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 49801: Syntax error in 'setup_ranger_xml.py' logging.

[Andrew Onischuk]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49801/
---

Review request for Ambari and Dmytro Sen.


Bugs: AMBARI-17620
https://issues.apache.org/jira/browse/AMBARI-17620


Repository: ambari


Description
---

File "/var/lib/ambari-agent/cache/common-
services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py", line 291, in
copy_jdbc_connector  
Please run 'ambari-server setup --jdbc-db=

{db_name}

\--jdbc-driver=

{path_to_jdbc}

on server host.'".format(params.db_flavor, params.jdk_location)  
KeyError: 'db_name'


Diffs
-

  
ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
 9dbf6d4 

Diff: https://reviews.apache.org/r/49801/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49691: Autoskip failure for blueprint deployment

[Ajit Kumar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49691/
---

(Updated July 8, 2016, 6:45 a.m.)


Review request for Ambari, Alejandro Fernandez, Nahappan Somasundaram, and 
Sumit Mohanty.


Changes
---

Replaced "skip_failure" in AmbariManagementControllerImpl.java and 
HostComponentResourceProvider.java  with Setting.SETTING_NAME_SKIP_FAILURE.


Bugs: AMBARI-17599
https://issues.apache.org/jira/browse/AMBARI-17599


Repository: ambari


Description
---

Currently if a cluster deployment is scheduled and any install/start task 
fails, all other queued tasks are cancelled. Ideally it should be possible for 
user to pass a flag to skip failures for best effort deployment instead of 
failing fast on error.

While creating a blueprint json, user can provide deployment settings as 
follows :

"settings" : [
  {
  "deployment_settings" : [{
  "skip_failure" : "true"
   }]
  }
]


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 aaf69df 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java
 9d8389a 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
 de7f209 
  
ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
 c6279bc 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
 926d253 
  ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java 
0edbaea 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java
 aeb9a2d 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
 3cdca4d 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
 cee9406 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java 
a9c26cc 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java
 7ec6088 
  ambari-server/src/main/java/org/apache/ambari/server/topology/Setting.java 
c03d833 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
 714 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
 0b06eb8 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithHostsSyspreppedTest.java
 d49c21d 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java
 ad441e4 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/LogicalRequestTest.java
 e979173 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/SettingTest.java 
5615d12 

Diff: https://reviews.apache.org/r/49691/diff/


Testing
---

Tested manually by creating a cluster with blueprint (attached). It has 
deployment_settings as follows
"settings" : [
  {
  "deployment_settings" : [{
  "skip_failure" : "true"
   }]
  }
]

mvn test

[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [8.590s]
[INFO] Apache Ambari Project POM . SUCCESS [0.024s]
[INFO] Ambari Web  SUCCESS [37.028s]
[INFO] Ambari Views .. SUCCESS [3.221s]
[INFO] Ambari Admin View . SUCCESS [14.033s]
[INFO] ambari-metrics  SUCCESS [0.507s]
[INFO] Ambari Metrics Common . SUCCESS [9.610s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [5.440s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [7.033s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [3.401s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [2.075s]
[INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.573s]
[INFO] Ambari Metrics Collector .. SUCCESS [8:00.318s]
[INFO] Ambari Metrics Monitor  SUCCESS [1.513s]
[INFO] Ambari Metrics Grafana  SUCCESS [0.744s]
[INFO] Ambari Metrics Assembly ... SUCCESS [2.746s]
[INFO] Ambari Server . SUCCESS [59:14.115s]
[INFO] Ambari Functional Tests ... SUCCESS [1.808s]
[INFO] Ambari Agent .. SUCCESS [23.211s]
[INFO] Ambari Client . SUCCESS [0.029s]
[INFO] Ambari Python Client .. SUCCESS [0.389s]
[INFO] Ambari Groovy Client .. SUCCESS [11.331s]
[INFO] Ambari Shell 

Re: Review Request 49786: AMBARI-17615 : AMS metrics GET API does not work for same metric with multiple aggregation functions

[Aravindan Vijayan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49786/
---

(Updated July 8, 2016, 6:26 a.m.)


Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-17615
https://issues.apache.org/jira/browse/AMBARI-17615


Repository: ambari


Description
---

AMS API which used to work in previous versions is now broken in 2.4.0 version

GET 

http://:6188/ws/v1/timeline/metrics?metricNames=bytes_in._min,bytes_in._max,bytes_in._sum,bytes_in._avg=HOST=hours=146692800=146743920=

failed. Status:400
{
"exception": "BadRequestException",
"message": "java.lang.Exception: Multiple aggregate functions not supported.",
"javaClassName": "org.apache.hadoop.yarn.webapp.BadRequestException"
}

This impacts SmartSense capture

FIX

Changed data structure that stores the MetricName -> Aggregation Function from 
HashMap to Multimap, so that we can have 1 metric name mapping to separate 
lists of functions.


Diffs
-

  ambari-metrics/ambari-metrics-timelineservice/pom.xml a0e4adf 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStore.java
 9510574 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessor.java
 bbd6d83 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/HBaseTimelineMetricStoreTest.java
 29e2664 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/ITPhoenixHBaseAccessor.java
 ae1aa5a 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/PhoenixHBaseAccessorTest.java
 a86fa11 

Diff: https://reviews.apache.org/r/49786/diff/


Testing (updated)
---

mvn clean test

Apache

+1 overall. Here are the results of testing the latest attachment 
http://issues.apache.org/jira/secure/attachment/12816764/AMBARI-17615.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified test files.
+1 javac. The applied patch does not increase the total number of javac 
compiler warnings.
+1 release audit. The applied patch does not increase the total number of 
release audit warnings.
+1 core tests. The patch passed unit tests in 
ambari-metrics/ambari-metrics-timelineservice.
Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7732//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/7732//console
This message is automatically generated.


Thanks,

Aravindan Vijayan

Re: Review Request 49691: Autoskip failure for blueprint deployment

[Ajit Kumar]

> On July 8, 2016, 5:14 a.m., Sumit Mohanty wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java,
> >  line 2464
> > 
> >
> > Move it out as a constant? Say in ExecutionCommand.java. Is 
> > Setting.SETTING_NAME_SKIP_FAILURE accessble here?

Replaced "skip_failure" in AmbariManagementControllerImpl.java and 
HostComponentResourceProvider.java  with Setting.SETTING_NAME_SKIP_FAILURE.


- Ajit


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49691/#review141244
---


On July 7, 2016, 6:09 p.m., Ajit Kumar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49691/
> ---
> 
> (Updated July 7, 2016, 6:09 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nahappan Somasundaram, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17599
> https://issues.apache.org/jira/browse/AMBARI-17599
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently if a cluster deployment is scheduled and any install/start task 
> fails, all other queued tasks are cancelled. Ideally it should be possible 
> for user to pass a flag to skip failures for best effort deployment instead 
> of failing fast on error.
> 
> While creating a blueprint json, user can provide deployment settings as 
> follows :
> 
> "settings" : [
>   {
>   "deployment_settings" : [{
>   "skip_failure" : "true"
>}]
>   }
> ]
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  aaf69df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostComponentResourceProvider.java
>  9d8389a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/HostResourceProvider.java
>  de7f209 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c6279bc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/AmbariContext.java
>  926d253 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/Blueprint.java 
> 0edbaea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/BlueprintImpl.java
>  aeb9a2d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopology.java
>  3cdca4d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterTopologyImpl.java
>  cee9406 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/HostRequest.java
>  a9c26cc 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/LogicalRequest.java
>  7ec6088 
>   ambari-server/src/main/java/org/apache/ambari/server/topology/Setting.java 
> c03d833 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/RequestResourceProviderTest.java
>  714 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/BlueprintImplTest.java
>  0b06eb8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterDeployWithHostsSyspreppedTest.java
>  d49c21d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterInstallWithoutStartTest.java
>  ad441e4 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/LogicalRequestTest.java
>  e979173 
>   
> ambari-server/src/test/java/org/apache/ambari/server/topology/SettingTest.java
>  5615d12 
> 
> Diff: https://reviews.apache.org/r/49691/diff/
> 
> 
> Testing
> ---
> 
> Tested manually by creating a cluster with blueprint (attached). It has 
> deployment_settings as follows
> "settings" : [
>   {
>   "deployment_settings" : [{
>   "skip_failure" : "true"
>}]
>   }
> ]
> 
> mvn test
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [8.590s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.024s]
> [INFO] Ambari Web  SUCCESS [37.028s]
> [INFO] Ambari Views .. SUCCESS [3.221s]
> [INFO] Ambari Admin View . SUCCESS [14.033s]
> [INFO] ambari-metrics  SUCCESS [0.507s]
> [INFO] Ambari Metrics Common . SUCCESS [9.610s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [5.440s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [7.033s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [3.401s]
> [INFO] Ambari Metrics Storm Sink