Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST

2016-11-04 Thread Vishal Ghugare 


> On Nov. 1, 2016, 11:46 a.m., Robert Levas wrote:
> > Ship It!

Thank you Robert! I would really appreciate if you can commit these changes on 
my behalf to trunk ( I am not a committer).The latest patch is attached to 
AMBARI-12263. Thanks again.


- Vishal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/#review154434
---


On Oct. 21, 2016, 7:38 p.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52369/
> ---
> 
> (Updated Oct. 21, 2016, 7:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.
> 
> 
> Bugs: AMBARI-12263
> https://issues.apache.org/jira/browse/AMBARI-12263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hello Robert,
> 
> How are you doing? 
> 
> We have been working on PAM support into Ambari and have something ready for 
> review. Can you please take a look at the patch and documentation and provide 
> your feedback.
> 
> Please let me know if you have any questions.
> 
> Note: I have added you as a reviewer as i see some authentication related 
> commits under your name.
> 
> Thanks,
> -Vishal
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml d507b82 
>   ambari-server/sbin/ambari-server 762ae19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1fc9dbf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  5e498f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
>  ef28f61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
>  e1aa5ac 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
>  bdd73a6 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java 
> 255c5e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java 
> e4ed9c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
>  00e233e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
>  26d4da7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
>  b20df8d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
>  aa9f3e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
>  e547f05 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  185bd58 
>   ambari-server/src/main/python/ambari-server.py bb6bc0e 
>   ambari-server/src/main/python/ambari_server/setupActions.py 697bc1d 
>   ambari-server/src/main/python/ambari_server/setupSecurity.py 119a7d8 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 1d55515 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
>   ambari-server/src/main/resources/properties.json eb27878 
>   ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 500c0bf 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java
>  a80cd03 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7b6c3ad 
> 
> Diff: https://reviews.apache.org/r/52369/diff/
> 
> 
> Testing
> ---
> 
> No test cases added at this point.
> 
> 
> File Attachments
> 
> 
> AMBARI-12263.patch_base
>   
> https://reviews.apache.org/media/uploaded/files/2016/10/17/5107a016-3a83-478c-b98c-2f35ecf6cbc5__AMBARI-12263.patch_base
> 
> 
> Thanks,
> 
> Vishal Ghugare
> 
>

Re: Review Request 53512: Fix the backend for HAWQ View BETA

2016-11-04 Thread Matt 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53512/
---

(Updated Nov. 4, 2016, 10:34 p.m.)


Review request for Ambari, Alexander Denissov and Lav Jain.


Changes
---

Updated patch for trunk


Bugs: AMBARI-18779
https://issues.apache.org/jira/browse/AMBARI-18779


Repository: ambari


Description
---

Fix the backend for HAWQ View so that the backend is able to query from 
pg_stat_activity table.


Diffs
-

  contrib/views/hawq/README.md b8b51ba 
  contrib/views/hawq/pom.xml e49a23b 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/HAWQDataSource.java
 PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/HAWQViewServlet.java
 4202eef 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/JsonApiResource.java
 PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/QueryResource.java 
PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/QueryResourceProvider.java
 PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/QueryService.java 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/WEB-INF/web.xml e12f314 
  contrib/views/hawq/src/main/resources/ui/app/adapters/query.js f82c00d 
  contrib/views/hawq/src/main/resources/ui/app/app.js af4fdc4 
  contrib/views/hawq/src/main/resources/ui/app/components/query-table.js 
d28e2f2 
  contrib/views/hawq/src/main/resources/ui/app/models/query.js 5906c34 
  contrib/views/hawq/src/main/resources/ui/app/resolver.js b9eabe4 
  contrib/views/hawq/src/main/resources/ui/app/router.js f058f78 
  contrib/views/hawq/src/main/resources/ui/app/routes/application.js 4ad870b 
  contrib/views/hawq/src/main/resources/ui/app/routes/main.js a05324a 
  contrib/views/hawq/src/main/resources/ui/app/serializers/query.js 9546da4 
  contrib/views/hawq/src/main/resources/ui/app/styles/app.scss 7470867 
  contrib/views/hawq/src/main/resources/ui/app/templates/application.hbs 
4c831d1 
  
contrib/views/hawq/src/main/resources/ui/app/templates/components/query-table.hbs
 5af548a 
  contrib/views/hawq/src/main/resources/ui/app/templates/main.hbs 6baba0c 
  contrib/views/hawq/src/main/resources/ui/app/utils/utils.js 7b02cca 
  contrib/views/hawq/src/main/resources/ui/bower.json 5bab18f 
  contrib/views/hawq/src/main/resources/ui/config/environment.js 4c742ef 
  contrib/views/hawq/src/main/resources/ui/package.json f1b4116 
  contrib/views/hawq/src/main/resources/ui/tests/acceptance/application-test.js 
c7ce7c7 
  
contrib/views/hawq/src/main/resources/ui/tests/helpers/module-for-acceptance.js 
3efeee7 
  contrib/views/hawq/src/main/resources/ui/tests/helpers/test-helper.js 8c70dbc 
  
contrib/views/hawq/src/main/resources/ui/tests/integration/components/query-table-test.js
 e02f8fd 
  contrib/views/hawq/src/main/resources/ui/tests/test-helper.js 96975ee 
  contrib/views/hawq/src/main/resources/ui/tests/unit/adapters/query-test.js 
bab0260 
  contrib/views/hawq/src/main/resources/ui/tests/unit/models/query-test.js 
b83130e 
  contrib/views/hawq/src/main/resources/ui/tests/unit/routes/main-test.js 
7ad0b79 
  contrib/views/hawq/src/main/resources/ui/tests/unit/serializers/query-test.js 
6cd5f88 
  contrib/views/hawq/src/main/resources/ui/tests/unit/utils/utils-test.js 
969ee6d 
  contrib/views/hawq/src/main/resources/view.log4j.properties PRE-CREATION 
  contrib/views/hawq/src/main/resources/view.xml 1421c27 
  
contrib/views/hawq/src/test/java/org/apache/ambari/view/hawq/HAWQDataSourceTest.java
 PRE-CREATION 
  
contrib/views/hawq/src/test/java/org/apache/ambari/view/hawq/QueryResourceProviderTest.java
 PRE-CREATION 
  contrib/views/pom.xml 39560b5 

Diff: https://reviews.apache.org/r/53512/diff/


Testing (updated)
---

Added tests on backend and updated tests on frontend.
Tested on live cluster.

Hadoop QA passed: 
https://builds.apache.org/job/Ambari-trunk-test-patch/9153//testReport/


Thanks,

Matt

Review Request 53512: Fix the backend for HAWQ View BETA

2016-11-04 Thread Matt 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53512/
---

Review request for Ambari, Alexander Denissov and Lav Jain.


Bugs: AMBARI-18779
https://issues.apache.org/jira/browse/AMBARI-18779


Repository: ambari


Description
---

Fix the backend for HAWQ View so that the backend is able to query from 
pg_stat_activity table.


Diffs
-

  contrib/views/hawq/README.md b8b51ba 
  contrib/views/hawq/pom.xml e49a23b 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/HAWQDataSource.java
 PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/HAWQViewServlet.java
 4202eef 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/JsonApiResource.java
 PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/QueryResource.java 
PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/QueryResourceProvider.java
 PRE-CREATION 
  
contrib/views/hawq/src/main/java/org/apache/ambari/view/hawq/QueryService.java 
PRE-CREATION 
  contrib/views/hawq/src/main/resources/WEB-INF/web.xml e12f314 
  contrib/views/hawq/src/main/resources/ui/app/adapters/query.js f82c00d 
  contrib/views/hawq/src/main/resources/ui/app/app.js af4fdc4 
  contrib/views/hawq/src/main/resources/ui/app/components/query-table.js 
d28e2f2 
  contrib/views/hawq/src/main/resources/ui/app/models/query.js 5906c34 
  contrib/views/hawq/src/main/resources/ui/app/resolver.js b9eabe4 
  contrib/views/hawq/src/main/resources/ui/app/router.js f058f78 
  contrib/views/hawq/src/main/resources/ui/app/routes/application.js 4ad870b 
  contrib/views/hawq/src/main/resources/ui/app/routes/main.js a05324a 
  contrib/views/hawq/src/main/resources/ui/app/serializers/query.js 9546da4 
  contrib/views/hawq/src/main/resources/ui/app/styles/app.scss 7470867 
  contrib/views/hawq/src/main/resources/ui/app/templates/application.hbs 
4c831d1 
  
contrib/views/hawq/src/main/resources/ui/app/templates/components/query-table.hbs
 5af548a 
  contrib/views/hawq/src/main/resources/ui/app/templates/main.hbs 6baba0c 
  contrib/views/hawq/src/main/resources/ui/app/utils/utils.js 7b02cca 
  contrib/views/hawq/src/main/resources/ui/bower.json 5bab18f 
  contrib/views/hawq/src/main/resources/ui/config/environment.js 4c742ef 
  contrib/views/hawq/src/main/resources/ui/package.json f1b4116 
  contrib/views/hawq/src/main/resources/ui/tests/acceptance/application-test.js 
c7ce7c7 
  
contrib/views/hawq/src/main/resources/ui/tests/helpers/module-for-acceptance.js 
3efeee7 
  contrib/views/hawq/src/main/resources/ui/tests/helpers/test-helper.js 8c70dbc 
  
contrib/views/hawq/src/main/resources/ui/tests/integration/components/query-table-test.js
 e02f8fd 
  contrib/views/hawq/src/main/resources/ui/tests/test-helper.js 96975ee 
  contrib/views/hawq/src/main/resources/ui/tests/unit/adapters/query-test.js 
bab0260 
  contrib/views/hawq/src/main/resources/ui/tests/unit/models/query-test.js 
b83130e 
  contrib/views/hawq/src/main/resources/ui/tests/unit/routes/main-test.js 
7ad0b79 
  contrib/views/hawq/src/main/resources/ui/tests/unit/serializers/query-test.js 
6cd5f88 
  contrib/views/hawq/src/main/resources/ui/tests/unit/utils/utils-test.js 
969ee6d 
  contrib/views/hawq/src/main/resources/view.log4j.properties PRE-CREATION 
  contrib/views/hawq/src/main/resources/view.xml 1421c27 
  
contrib/views/hawq/src/test/java/org/apache/ambari/view/hawq/HAWQDataSourceTest.java
 PRE-CREATION 
  
contrib/views/hawq/src/test/java/org/apache/ambari/view/hawq/QueryResourceProviderTest.java
 PRE-CREATION 
  contrib/views/pom.xml 39560b5 

Diff: https://reviews.apache.org/r/53512/diff/


Testing
---

Added tests on backend and updated tests on frontend.
Tested on live cluster.
Hadoop QA passed: 
https://builds.apache.org/job/Ambari-trunk-test-patch/9153//testReport/


Thanks,

Matt

Re: Review Request 53505: AMBARI-18805 Bulk delete hosts: sending a force option to delete all components

2016-11-04 Thread Matt 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53505/#review155048
---


Ship it!




Ship It!

- Matt


On Nov. 4, 2016, 4:02 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53505/
> ---
> 
> (Updated Nov. 4, 2016, 4:02 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Richard Zang, Vivek Ratnavel 
> Subramanian, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18805
> https://issues.apache.org/jira/browse/AMBARI-18805
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Send a force option to delete all components when delete hosts
> 
> 
> Diffs
> -
> 
>   ambari-web/app/utils/ajax/ajax.js d263ba0 
> 
> Diff: https://reviews.apache.org/r/53505/diff/
> 
> 
> Testing
> ---
> 
> Local ambari-web test passed.
> 30351 tests complete (34 seconds)
> 151 tests pending
> Manual testing done
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Re: Review Request 53506: AMBARI-18761 NTP check fails even if chrony is working on CentOS7

2016-11-04 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53506/#review155046
---




ambari-agent/src/main/python/ambari_agent/HostInfo.py (line 120)


This should return a ('chronyd', 'ntp') since both are valid. Need to 
chnage the caller to do a OR.


- Sid Wagle


On Nov. 5, 2016, 1:13 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53506/
> ---
> 
> (Updated Nov. 5, 2016, 1:13 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18761
> https://issues.apache.org/jira/browse/AMBARI-18761
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> I use CentOS 7 as a host OS. When the host check runs, it shows warning. The 
> warning says "The following services should be up: Service ntpd."
> I think "chrony" is a default NTP client and server on CentOS7. So We should 
> check if there are ntpd or chronyd on a host.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/HostInfo.py 3871e72 
> 
> Diff: https://reviews.apache.org/r/53506/diff/
> 
> 
> Testing
> ---
> 
> ```
> -1 overall. Here are the results of testing the latest attachment 
> http://issues.apache.org/jira/secure/attachment/12837135/AMBARI-18761.patch
> against trunk revision .
> +1 @author. The patch does not contain any @author tags.
> -1 tests included. The patch doesn't appear to include any new or modified 
> tests.
> Please justify why no new tests are needed for this patch.
> Also please list what manual steps were performed to verify this patch.
> +1 javac. The applied patch does not increase the total number of javac 
> compiler warnings.
> +1 release audit. The applied patch does not increase the total number of 
> release audit warnings.
> +1 core tests. The patch passed unit tests in .
> Test results: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/9144//testReport/
> Console output: 
> https://builds.apache.org/job/Ambari-trunk-test-patch/9144//console
> This message is automatically generated.
> ```
> 
> Manual test: Installed Ambari2.4.1 on centos7, replaced HostInfo.py and 
> recheck host.
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Review Request 53506: AMBARI-18761 NTP check fails even if chrony is working on CentOS7

2016-11-04 Thread Masahiro Tanaka 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53506/
---

Review request for Ambari, Andrew Onischuk, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-18761
https://issues.apache.org/jira/browse/AMBARI-18761


Repository: ambari


Description
---

I use CentOS 7 as a host OS. When the host check runs, it shows warning. The 
warning says "The following services should be up: Service ntpd."
I think "chrony" is a default NTP client and server on CentOS7. So We should 
check if there are ntpd or chronyd on a host.


Diffs (updated)
-

  ambari-agent/src/main/python/ambari_agent/HostInfo.py 3871e72 

Diff: https://reviews.apache.org/r/53506/diff/


Testing
---

```
-1 overall. Here are the results of testing the latest attachment 
http://issues.apache.org/jira/secure/attachment/12837135/AMBARI-18761.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified 
tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac 
compiler warnings.
+1 release audit. The applied patch does not increase the total number of 
release audit warnings.
+1 core tests. The patch passed unit tests in .
Test results: 
https://builds.apache.org/job/Ambari-trunk-test-patch/9144//testReport/
Console output: 
https://builds.apache.org/job/Ambari-trunk-test-patch/9144//console
This message is automatically generated.
```

Manual test: Installed Ambari2.4.1 on centos7, replaced HostInfo.py and recheck 
host.


Thanks,

Masahiro Tanaka

Re: Review Request 53505: AMBARI-18805 Bulk delete hosts: sending a force option to delete all components

2016-11-04 Thread Matt 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53505/#review155037
---




ambari-web/app/utils/ajax/ajax.js (line 307)


Does this fix mean that deleting a host will go through, even if there are 
components installed and in started state (running on the host)?


- Matt


On Nov. 4, 2016, 4:02 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53505/
> ---
> 
> (Updated Nov. 4, 2016, 4:02 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Richard Zang, Vivek Ratnavel 
> Subramanian, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18805
> https://issues.apache.org/jira/browse/AMBARI-18805
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Send a force option to delete all components when delete hosts
> 
> 
> Diffs
> -
> 
>   ambari-web/app/utils/ajax/ajax.js d263ba0 
> 
> Diff: https://reviews.apache.org/r/53505/diff/
> 
> 
> Testing
> ---
> 
> Local ambari-web test passed.
> 30351 tests complete (34 seconds)
> 151 tests pending
> Manual testing done
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Re: Review Request 53505: AMBARI-18805 Bulk delete hosts: sending a force option to delete all components

2016-11-04 Thread Richard Zang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53505/#review155018
---


Ship it!




Ship It!

- Richard Zang


On Nov. 4, 2016, 11:02 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53505/
> ---
> 
> (Updated Nov. 4, 2016, 11:02 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Richard Zang, Vivek Ratnavel 
> Subramanian, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18805
> https://issues.apache.org/jira/browse/AMBARI-18805
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Send a force option to delete all components when delete hosts
> 
> 
> Diffs
> -
> 
>   ambari-web/app/utils/ajax/ajax.js d263ba0 
> 
> Diff: https://reviews.apache.org/r/53505/diff/
> 
> 
> Testing
> ---
> 
> Local ambari-web test passed.
> 30351 tests complete (34 seconds)
> 151 tests pending
> Manual testing done
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Review Request 53505: AMBARI-18805 Bulk delete hosts: sending a force option to delete all components

2016-11-04 Thread Zhe (Joe) Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53505/
---

Review request for Ambari, Jaimin Jetly, Richard Zang, Vivek Ratnavel 
Subramanian, Xi Wang, and Yusaku Sako.


Bugs: AMBARI-18805
https://issues.apache.org/jira/browse/AMBARI-18805


Repository: ambari


Description
---

Send a force option to delete all components when delete hosts


Diffs
-

  ambari-web/app/utils/ajax/ajax.js d263ba0 

Diff: https://reviews.apache.org/r/53505/diff/


Testing
---

Local ambari-web test passed.
30351 tests complete (34 seconds)
151 tests pending
Manual testing done


Thanks,

Zhe (Joe) Wang

Review Request 53504: Manage Ambari principals should be set to off when upgrading Ambari from versions < 2.4.0

2016-11-04 Thread Robert Levas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53504/
---

Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Nettleton, and 
Sandor Magyari.


Bugs: AMBARI-18804
https://issues.apache.org/jira/browse/AMBARI-18804


Repository: ambari


Description
---

Since users would have manually set up the Ambari principal after enabling 
Kerberos using `ambari-server setup-security` `option #3` ("Setup Ambari 
kerberos JAAS configuration") in Ambari versions before 2.4.0, there is no need 
to configure Ambari to automatically manage its principals after an upgrade to 
version 2.4.0 and above. 

Therefore, upon upgrade to Ambari 2.4.0 or above, the upgrade process (in 
`UpgradeCatalog240`) should ensure that `kerberos-env/create_ambari_principal` 
is set to "false".  By default this value will be set to "true" after 
`org.apache.ambari.server.upgrade.AbstractUpgradeCatalog#addNewConfigurationsFromXml`
 is executed. 

Note: This may have an effect on Ambari versions 2.4.2 and above if Kerberos 
authentication is enabled and the SPNEGO (`HTTP/_HOST`) principal and keytab 
file is already created and installed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 54afd8d 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 958758f 

Diff: https://reviews.apache.org/r/53504/diff/


Testing
---

Manually tested upgrade from 2.2.2.

# Local test results:
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:11:23.488s
[INFO] Finished at: Fri Nov 04 18:28:36 EDT 2016
[INFO] Final Memory: 61M/1909M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 53502: AMBARI-18681 Add host structured_out to manual task

2016-11-04 Thread Duc Le 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/
---

(Updated Nov. 4, 2016, 10:51 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-18681
https://issues.apache.org/jira/browse/AMBARI-18681


Repository: ambari


Description
---

AMBARI-18681 Add host structured_out to manual task


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
 b999f2f 

Diff: https://reviews.apache.org/r/53502/diff/


Testing
---

TBD


Thanks,

Duc Le

Review Request 53502: AMBARI-18681 Add host structured_out to manual task

2016-11-04 Thread Duc Le 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53502/
---

Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, and Nate Cole.


Bugs: AMBARI-18681
https://issues.apache.org/jira/browse/AMBARI-18681


Repository: ambari


Description
---

AMBARI-18681 Add host structured_out to manual task


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
 b999f2f 

Diff: https://reviews.apache.org/r/53502/diff/


Testing
---

TBD


Thanks,

Duc Le

Re: Review Request 50014: AMBARI-17703 - Hosts page: after component filter hosts have wrong list of components

2016-11-04 Thread Zhe (Joe) Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50014/#review155001
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On Nov. 4, 2016, 9:55 p.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50014/
> ---
> 
> (Updated Nov. 4, 2016, 9:55 p.m.)
> 
> 
> Review request for Ambari and Zhe (Joe) Wang.
> 
> 
> Bugs: AMBARI-17703
> https://issues.apache.org/jira/browse/AMBARI-17703
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enable preload for host component related filters
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/global/update_controller.js fda1704 
>   ambari-web/app/controllers/main/host.js cea40cb 
> 
> Diff: https://reviews.apache.org/r/50014/diff/
> 
> 
> Testing
> ---
> 
> Manually tested on live cluster, all unit tests passed.
>   29021 tests complete (27 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Re: Review Request 50014: AMBARI-17703 - Hosts page: after component filter hosts have wrong list of components

2016-11-04 Thread Richard Zang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50014/
---

(Updated Nov. 4, 2016, 9:55 p.m.)


Review request for Ambari and Zhe (Joe) Wang.


Changes
---

Fix query parameter issue in the relay request


Bugs: AMBARI-17703
https://issues.apache.org/jira/browse/AMBARI-17703


Repository: ambari


Description
---

Enable preload for host component related filters


Diffs (updated)
-

  ambari-web/app/controllers/global/update_controller.js fda1704 
  ambari-web/app/controllers/main/host.js cea40cb 

Diff: https://reviews.apache.org/r/50014/diff/


Testing
---

Manually tested on live cluster, all unit tests passed.
  29021 tests complete (27 seconds)
  154 tests pending


Thanks,

Richard Zang

Re: Review Request 53404: Component actions confirmation window (from host details page) don't have the details about action to be confirmed

2016-11-04 Thread Zhe (Joe) Wang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53404/#review154988
---


Ship it!




Ship It!

- Zhe (Joe) Wang


On Nov. 2, 2016, 8:45 p.m., Vivek Ratnavel Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53404/
> ---
> 
> (Updated Nov. 2, 2016, 8:45 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, Richard Zang, Xi 
> Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18777
> https://issues.apache.org/jira/browse/AMBARI-18777
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Component actions confirmation modal window (from host details page) don't 
> have the details about action to be confirmed. 
> 
> The window asks the user "Are you sure" only for any action selected except 
> delete e.g Stop, Start, Turn On Maintenance Mode, Restart.
> 
> Steps to reproduce the issue:
> 1. Login to Ambari
> 2. Navigate to Hosts Tab
> 3. Click on Service action drop down of any service available
> 4. Click on any component action e.g Restart/Stop/Start
> 5. The confirmation window should have the action to be performed E.g In Case 
> of Restart, "You are about to Restart Service NodeManager, Confirm?".
> But it just have text "Are you sure" for any action selected except delete
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/host/details.js 848e6f8 
>   ambari-web/app/messages.js b0ebddd 
> 
> Diff: https://reviews.apache.org/r/53404/diff/
> 
> 
> Testing
> ---
> 
> Verified Manually.
> Ambari-web unit tests pass.
> 30323 tests complete (28 seconds)
> 151 tests pending
> 
> 
> Thanks,
> 
> Vivek Ratnavel Subramanian
> 
>

Re: Review Request 53454: AMBARI-18795: Ambari-server: Stack definition to specify which services support credential store

2016-11-04 Thread Nahappan Somasundaram 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53454/
---

(Updated Nov. 4, 2016, 1:32 p.m.)


Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
Levas, and Sumit Mohanty.


Changes
---

HDP-2.3 stack definition changes for RANGER.


Bugs: AMBARI-18795
https://issues.apache.org/jira/browse/AMBARI-18795


Repository: ambari


Description
---

AMBARI-18795: Ambari-server: Stack definition to specify which services support 
credential store


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.3/services/RANGER/metainfo.xml 
c5a727ca549af8accea1eceee59bb9662785146e 
  ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
811c5e9b443b1d13bdc1884d098d0ad726f92f0d 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
a409465cbf8e6a457f8330c7a7bf073bdbb716e6 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/metainfo.xml 
8f303a54a5128520bea23f6f6c87ad7ae41965ed 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
fe97a41657dcd647707729951a4d09fa5eb870e5 

Diff: https://reviews.apache.org/r/53454/diff/


Testing
---

** Manual testing **

Copied the modified stack definition files to a VM, deployed RANGER, HIVE and 
KNOX for HDP-2.5. Verified in the database that the following entries were made:

ambari=> select service_name, credential_store_supported, 
credential_store_enabled from servicedesiredstate;

 service_name | credential_store_supported | credential_store_enabled
--++--
 HDFS |  0 |0
 HIVE |  1 |0
 KNOX |  1 |0
 MAPREDUCE2   |  0 |0
 PIG  |  0 |0
 SLIDER   |  0 |0
 TEZ  |  0 |0
 YARN |  0 |0
 ZOOKEEPER|  0 |0
 RANGER   |  1 |1


Thanks,

Nahappan Somasundaram

Re: Review Request 53454: AMBARI-18795: Ambari-server: Stack definition to specify which services support credential store

2016-11-04 Thread Jayush Luniya 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53454/#review154981
---




ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
(line 26)


What about RANGER in HDP-2.2 and HDP-2.3?


- Jayush Luniya


On Nov. 4, 2016, 4:53 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53454/
> ---
> 
> (Updated Nov. 4, 2016, 4:53 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
> Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18795
> https://issues.apache.org/jira/browse/AMBARI-18795
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18795: Ambari-server: Stack definition to specify which services 
> support credential store
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
> 811c5e9 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> a409465 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/metainfo.xml 
> 8f303a5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
> fe97a41 
> 
> Diff: https://reviews.apache.org/r/53454/diff/
> 
> 
> Testing
> ---
> 
> ** Manual testing **
> 
> Copied the modified stack definition files to a VM, deployed RANGER, HIVE and 
> KNOX for HDP-2.5. Verified in the database that the following entries were 
> made:
> 
> ambari=> select service_name, credential_store_supported, 
> credential_store_enabled from servicedesiredstate;
> 
>  service_name | credential_store_supported | credential_store_enabled
> --++--
>  HDFS |  0 |0
>  HIVE |  1 |0
>  KNOX |  1 |0
>  MAPREDUCE2   |  0 |0
>  PIG  |  0 |0
>  SLIDER   |  0 |0
>  TEZ  |  0 |0
>  YARN |  0 |0
>  ZOOKEEPER|  0 |0
>  RANGER   |  1 |1
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Re: Review Request 53480: Add a cluster version directly as INSTALLED

2016-11-04 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53480/#review154973
---


Ship it!




Ship It!

- Nate Cole


On Nov. 4, 2016, 3:23 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53480/
> ---
> 
> (Updated Nov. 4, 2016, 3:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Nate 
> Cole.
> 
> 
> Bugs: AMBARI-18708
> https://issues.apache.org/jira/browse/AMBARI-18708
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When creating a repository for a cluster, allow the API to set directly to 
> INSTALLED.  This is the case where bits are known to be preinstalled.
> 
> This API call will set direct values for {{cluster_version}} and 
> {{host_version}}.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
>  4e9fd6b 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
> b1958ef 
>   ambari-server/src/main/java/org/apache/ambari/server/state/Host.java 
> bd6cc0d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  bb46c5c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> 513d924 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
>  9837b0f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
>  dcbc435 
> 
> Diff: https://reviews.apache.org/r/53480/diff/
> 
> 
> Testing
> ---
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 34:29 min
> [INFO] Finished at: 2016-11-04T15:20:02-04:00
> [INFO] Final Memory: 45M/687M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 53480: Add a cluster version directly as INSTALLED

2016-11-04 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53480/
---

(Updated Nov. 4, 2016, 3:23 p.m.)


Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Nate 
Cole.


Bugs: AMBARI-18708
https://issues.apache.org/jira/browse/AMBARI-18708


Repository: ambari


Description
---

When creating a repository for a cluster, allow the API to set directly to 
INSTALLED.  This is the case where bits are known to be preinstalled.

This API call will set direct values for {{cluster_version}} and 
{{host_version}}.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
 4e9fd6b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
b1958ef 
  ambari-server/src/main/java/org/apache/ambari/server/state/Host.java bd6cc0d 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 bb46c5c 
  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
513d924 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
 9837b0f 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
 dcbc435 

Diff: https://reviews.apache.org/r/53480/diff/


Testing (updated)
---

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 34:29 min
[INFO] Finished at: 2016-11-04T15:20:02-04:00
[INFO] Final Memory: 45M/687M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 53494: Update Ambari LogSearch integration to only queue a LogSearch request once

2016-11-04 Thread Robert Nettleton 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53494/
---

(Updated Nov. 4, 2016, 6:46 p.m.)


Review request for Ambari, Jonathan Hurley, Miklos Gergely, and Oliver Szabo.


Changes
---

Uploading second version of patch, including a fix for the issue found by 
jhurley.


Bugs: AMBARI-17529
https://issues.apache.org/jira/browse/AMBARI-17529


Repository: ambari


Description
---

This patch resolves AMBARI-17529. 

In the initial implementation of the LogSearch integration layer in Ambari, in 
Ambari 2.4.0, it was possible that multiple requests for a given host/comopnent 
for logging information be queued and sent to the LogSearch Portal Server. 

While these requests are idempotent, it is a slight performance issue to allow 
these requests to occur multiple times.  This is particularly an issue when the 
LogSearch-related PropertyProvider might be called during various Ambari REST 
calls, even if the LogSearch information is not required. 

This patch uses an approach similar to that found in:

org.apache.ambari.server.state.services.MetricsRetrievalService

This patch implements the following:

1. Implements a request cache, to keep track of the current pending requests on 
a host/component basis, and to ensure that only one request for log file 
information occurs on a host/component basis.  
2. Refactors the Runnable implementation used by LogSearch to execute these log 
file information requests, in order to facilitate simpler unit testing.
3. Adds some DEBUG-level log statements, in order to debug any problems that 
might arise in the caching layer in the future.  
4. Adds new unit tests to verify these changes.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 4929747 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalServiceTest.java
 97c59f8 

Diff: https://reviews.apache.org/r/53494/diff/


Testing
---

1. Ran the ambari-server unit test suite ("mvn clean test"), with all tests 
passing.
2. Tested this change in a cluster deployment on top of the recent trunk code, 
and verified that the LogSearch integration UI in Ambari still functions as 
expected with this change applied.


Thanks,

Robert Nettleton

Re: Review Request 52850: Icons don't display in Windows 10 Microsoft Edge or Internet Explorer 11

2016-11-04 Thread Yusaku Sako 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52850/#review154963
---


Ship it!




Ship It!

- Yusaku Sako


On Nov. 4, 2016, 1:45 a.m., Vivek Ratnavel Subramanian wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52850/
> ---
> 
> (Updated Nov. 4, 2016, 1:45 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, Richard Zang, Xi 
> Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18595
> https://issues.apache.org/jira/browse/AMBARI-18595
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Icons do not display in any page as shown in the screen-shot attached.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  4b939fe 
> 
> Diff: https://reviews.apache.org/r/52850/diff/
> 
> 
> Testing
> ---
> 
> Verified Manually in the latest versions of Google Chrome Mac OS X, Google 
> Chrome Windows 10, Microsoft Internet Explorer 11 and Microsoft Edge.
> Ambari web unit tests pass.
> 30374 tests complete (30 seconds)
> 151 tests pending
> 
> 
> Thanks,
> 
> Vivek Ratnavel Subramanian
> 
>

Re: Review Request 53496: Perf: Automation script to manage the lifecycle of multiple Ambari Agents on a single Host

2016-11-04 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53496/#review154959
---




ambari-agent/conf/unix/agent-multiplier.py (line 35)


Each VM will contain this file.
The user is expected to
* install Ambari Agent RPMs on the VM
* modify its ambari-agent.ini file to point it to the server
* add dummy configs like,

hostname_script=foo
public_hostname_script=fo

* create the agent-multiplier.conf file with values for start, num, and 
prefix

And then run this script,
python agent-multiplier start

To start all of the agents on this VM, each with a unique port number.



ambari-agent/conf/unix/agent-multiplier.py (line 237)


Will add this in a future patch.



ambari-agent/conf/unix/agent-multiplier.py (line 279)


These are the supported commands.


- Alejandro Fernandez


On Nov. 4, 2016, 6:27 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53496/
> ---
> 
> (Updated Nov. 4, 2016, 6:27 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Aravindan Vijayan, Myroslav 
> Papirkovskyy, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18735
> https://issues.apache.org/jira/browse/AMBARI-18735
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We need a method of easily bootstrapping multiple Ambari Agents on a single 
> host.
> For example, a script that will create each agent's home directory, modify 
> its config file with unique hostname and port number, etc.
> The script should also allow starting/stopping all agents on the host, and 
> aggregating the status.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/agent-multiplier.py PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/53496/diff/
> 
> 
> Testing
> ---
> 
> Verified it works locally and on a cloud environment.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 53494: Update Ambari LogSearch integration to only queue a LogSearch request once

2016-11-04 Thread Robert Nettleton 


> On Nov. 4, 2016, 6:16 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java,
> >  line 156
> > 
> >
> > Missing space before = ... just to be consistent with the other log 
> > statement below this one.

Thanks for catching this.  I'll upload a new version of the patch with this 
minor fix.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53494/#review154955
---


On Nov. 4, 2016, 6:11 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53494/
> ---
> 
> (Updated Nov. 4, 2016, 6:11 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Miklos Gergely, and Oliver Szabo.
> 
> 
> Bugs: AMBARI-17529
> https://issues.apache.org/jira/browse/AMBARI-17529
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch resolves AMBARI-17529. 
> 
> In the initial implementation of the LogSearch integration layer in Ambari, 
> in Ambari 2.4.0, it was possible that multiple requests for a given 
> host/comopnent for logging information be queued and sent to the LogSearch 
> Portal Server. 
> 
> While these requests are idempotent, it is a slight performance issue to 
> allow these requests to occur multiple times.  This is particularly an issue 
> when the LogSearch-related PropertyProvider might be called during various 
> Ambari REST calls, even if the LogSearch information is not required. 
> 
> This patch uses an approach similar to that found in:
> 
> org.apache.ambari.server.state.services.MetricsRetrievalService
> 
> This patch implements the following:
> 
> 1. Implements a request cache, to keep track of the current pending requests 
> on a host/component basis, and to ensure that only one request for log file 
> information occurs on a host/component basis.  
> 2. Refactors the Runnable implementation used by LogSearch to execute these 
> log file information requests, in order to facilitate simpler unit testing.
> 3. Adds some DEBUG-level log statements, in order to debug any problems that 
> might arise in the caching layer in the future.  
> 4. Adds new unit tests to verify these changes.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
>  4929747 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalServiceTest.java
>  97c59f8 
> 
> Diff: https://reviews.apache.org/r/53494/diff/
> 
> 
> Testing
> ---
> 
> 1. Ran the ambari-server unit test suite ("mvn clean test"), with all tests 
> passing.
> 2. Tested this change in a cluster deployment on top of the recent trunk 
> code, and verified that the LogSearch integration UI in Ambari still 
> functions as expected with this change applied.
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>

Re: Review Request 53494: Update Ambari LogSearch integration to only queue a LogSearch request once

2016-11-04 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53494/#review154955
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 (line 156)


Missing space before = ... just to be consistent with the other log 
statement below this one.


- Jonathan Hurley


On Nov. 4, 2016, 2:11 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53494/
> ---
> 
> (Updated Nov. 4, 2016, 2:11 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Miklos Gergely, and Oliver Szabo.
> 
> 
> Bugs: AMBARI-17529
> https://issues.apache.org/jira/browse/AMBARI-17529
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch resolves AMBARI-17529. 
> 
> In the initial implementation of the LogSearch integration layer in Ambari, 
> in Ambari 2.4.0, it was possible that multiple requests for a given 
> host/comopnent for logging information be queued and sent to the LogSearch 
> Portal Server. 
> 
> While these requests are idempotent, it is a slight performance issue to 
> allow these requests to occur multiple times.  This is particularly an issue 
> when the LogSearch-related PropertyProvider might be called during various 
> Ambari REST calls, even if the LogSearch information is not required. 
> 
> This patch uses an approach similar to that found in:
> 
> org.apache.ambari.server.state.services.MetricsRetrievalService
> 
> This patch implements the following:
> 
> 1. Implements a request cache, to keep track of the current pending requests 
> on a host/component basis, and to ensure that only one request for log file 
> information occurs on a host/component basis.  
> 2. Refactors the Runnable implementation used by LogSearch to execute these 
> log file information requests, in order to facilitate simpler unit testing.
> 3. Adds some DEBUG-level log statements, in order to debug any problems that 
> might arise in the caching layer in the future.  
> 4. Adds new unit tests to verify these changes.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
>  4929747 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalServiceTest.java
>  97c59f8 
> 
> Diff: https://reviews.apache.org/r/53494/diff/
> 
> 
> Testing
> ---
> 
> 1. Ran the ambari-server unit test suite ("mvn clean test"), with all tests 
> passing.
> 2. Tested this change in a cluster deployment on top of the recent trunk 
> code, and verified that the LogSearch integration UI in Ambari still 
> functions as expected with this change applied.
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>

Re: Review Request 53454: AMBARI-18795: Ambari-server: Stack definition to specify which services support credential store

2016-11-04 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53454/#review154954
---


Ship it!




Ship It!

- Jonathan Hurley


On Nov. 4, 2016, 12:53 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53454/
> ---
> 
> (Updated Nov. 4, 2016, 12:53 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
> Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18795
> https://issues.apache.org/jira/browse/AMBARI-18795
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18795: Ambari-server: Stack definition to specify which services 
> support credential store
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
> 811c5e9 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> a409465 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/metainfo.xml 
> 8f303a5 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
> fe97a41 
> 
> Diff: https://reviews.apache.org/r/53454/diff/
> 
> 
> Testing
> ---
> 
> ** Manual testing **
> 
> Copied the modified stack definition files to a VM, deployed RANGER, HIVE and 
> KNOX for HDP-2.5. Verified in the database that the following entries were 
> made:
> 
> ambari=> select service_name, credential_store_supported, 
> credential_store_enabled from servicedesiredstate;
> 
>  service_name | credential_store_supported | credential_store_enabled
> --++--
>  HDFS |  0 |0
>  HIVE |  1 |0
>  KNOX |  1 |0
>  MAPREDUCE2   |  0 |0
>  PIG  |  0 |0
>  SLIDER   |  0 |0
>  TEZ  |  0 |0
>  YARN |  0 |0
>  ZOOKEEPER|  0 |0
>  RANGER   |  1 |1
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Re: Review Request 53446: AMBARI-18793: Ambari-server: Support stack inheritance for credential store information

2016-11-04 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53446/#review154952
---


Ship it!




Ship It!

- Jonathan Hurley


On Nov. 3, 2016, 8:30 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53446/
> ---
> 
> (Updated Nov. 3, 2016, 8:30 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18793
> https://issues.apache.org/jira/browse/AMBARI-18793
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18793: Ambari-server: Support stack inheritance for credential store 
> information
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> b6a0da53dfd796dea739da97c994e1fc37a3d155 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/CredentialStoreInfo.java
>  605de999bb1a87dcd74708a14b6cca8ff330a853 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 16042e12017745a674facbe70940ece80f83e56a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/ServiceModuleTest.java
>  47f5eb9b367e76ef7fd81b8c0ee3c8c5d82ea61f 
> 
> Diff: https://reviews.apache.org/r/53446/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [6.599s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Web  SUCCESS [1:14.514s]
> [INFO] Ambari Views .. SUCCESS [0.983s]
> [INFO] Ambari Admin View . SUCCESS [7.031s]
> [INFO] utility ... SUCCESS [0.309s]
> [INFO] ambari-metrics  SUCCESS [0.594s]
> [INFO] Ambari Metrics Common . SUCCESS [3.898s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.938s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.139s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.203s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [3.223s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.378s]
> [INFO] Ambari Metrics Collector .. SUCCESS [8.550s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.966s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.243s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:11.798s]
> [INFO] Ambari Server . SUCCESS [3:01.245s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.651s]
> [INFO] Ambari Agent .. SUCCESS [26.446s]
> [INFO] Ambari Client . SUCCESS [0.049s]
> [INFO] Ambari Python Client .. SUCCESS [0.941s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.149s]
> [INFO] Ambari Shell .. SUCCESS [0.045s]
> [INFO] Ambari Python Shell ... SUCCESS [0.695s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.932s]
> [INFO] ambari-logsearch .. SUCCESS [0.394s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.241s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.363s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [6.995s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [4.183s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.083s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6:57.041s
> [INFO] Finished at: Thu Nov 03 17:29:33 PDT 2016
> [INFO] Final Memory: 305M/1110M
> [INFO] 
> 
> 
> *** 2. mvn -DskipPythonTests -Dtest=*Provider*,*Service*,StackModuleTest **
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 2:41.191s
> [INFO] Finished at: Thu Nov 03 17:14:26 PDT 2016

Re: Review Request 53494: Update Ambari LogSearch integration to only queue a LogSearch request once

2016-11-04 Thread Robert Nettleton 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53494/
---

(Updated Nov. 4, 2016, 6:11 p.m.)


Review request for Ambari, Jonathan Hurley, Miklos Gergely, and Oliver Szabo.


Bugs: AMBARI-17529
https://issues.apache.org/jira/browse/AMBARI-17529


Repository: ambari


Description (updated)
---

This patch resolves AMBARI-17529. 

In the initial implementation of the LogSearch integration layer in Ambari, in 
Ambari 2.4.0, it was possible that multiple requests for a given host/comopnent 
for logging information be queued and sent to the LogSearch Portal Server. 

While these requests are idempotent, it is a slight performance issue to allow 
these requests to occur multiple times.  This is particularly an issue when the 
LogSearch-related PropertyProvider might be called during various Ambari REST 
calls, even if the LogSearch information is not required. 

This patch uses an approach similar to that found in:

org.apache.ambari.server.state.services.MetricsRetrievalService

This patch implements the following:

1. Implements a request cache, to keep track of the current pending requests on 
a host/component basis, and to ensure that only one request for log file 
information occurs on a host/component basis.  
2. Refactors the Runnable implementation used by LogSearch to execute these log 
file information requests, in order to facilitate simpler unit testing.
3. Adds some DEBUG-level log statements, in order to debug any problems that 
might arise in the caching layer in the future.  
4. Adds new unit tests to verify these changes.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 4929747 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalServiceTest.java
 97c59f8 

Diff: https://reviews.apache.org/r/53494/diff/


Testing
---

1. Ran the ambari-server unit test suite ("mvn clean test"), with all tests 
passing.
2. Tested this change in a cluster deployment on top of the recent trunk code, 
and verified that the LogSearch integration UI in Ambari still functions as 
expected with this change applied.


Thanks,

Robert Nettleton

Re: Review Request 53476: Rolling Upgrade From HDP 2.5.x to 2.5.y Doesn't hdp-select ZKFC

2016-11-04 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53476/#review154951
---


Ship it!




Ship It!

- Jonathan Hurley


On Nov. 4, 2016, 11:22 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53476/
> ---
> 
> (Updated Nov. 4, 2016, 11:22 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18800
> https://issues.apache.org/jira/browse/AMBARI-18800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Install HDP 2.5.0.0 with HDFS in HA mode
> - Perform a rolling upgrade to 2.5.2.0
> 
> At the end of the upgrade, the pre-finalize step fails with the upgrade 
> failing on 2 hosts for ZKFC. Taking a look at the orchestration, the ZKFC is 
> restarted. However, {{hdp-select}} is never invoked:
> 
> {noformat}
> 2016-11-02 18:38:15,577 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,597 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,598 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,600 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,621 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,621 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,623 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,644 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,644 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,646 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,667 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,668 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,669 - In the middle of a stack upgrade/downgrade for 
> Stack HDP and destination version 2.5.2.0-67, determining which hadoop conf 
> dir to use.
> 2016-11-02 18:38:15,669 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,669 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
> exists, will call conf-select on it for version 2.5.2.0-67
> 2016-11-02 18:38:15,669 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.5.2.0-67/0
> 2016-11-02 18:38:15,670 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
> 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
> '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
> 'stderr': -1}
> 2016-11-02 18:38:15,690 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
> already', '')
> 2016-11-02 18:38:15,690 - checked_call[('ambari-python-wrap', 
> '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
> '--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
> 2016-11-02 18:38:15,716 - checked_call returned (0, '')
> 2016-11-02 18:38:15,716 - Ensuring that hadoop has the correct symlink 
> structure
> 2016-11-02 18:38:15,716 - Using hadoop conf dir: 
> /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,718 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,738 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,739 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,826 - In the middle of a stack upgrade/downgrade for 
> Stack HDP and destination version 2.5.2.0-67, determining which hadoop conf 
> dir to use.
> 2016-11-02 18:38:15,826 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,826 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
> exists, will call conf-select on it for version 2.5.2.0-67
> 2016-11-02 18:38:15,826 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.5.2.0-67/0
> 2016-11-02 18:38:15,827 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
> 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
> '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
> 'stderr': -1}
> 2016-11-02 18:38:15,846 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
> already', '')
> 2016-11-02 18:38:15,846 - checked_call[('ambari-python-wrap', 
> '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
> '--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
>

Review Request 53494: Update Ambari LogSearch integration to only queue a LogSearch request once

2016-11-04 Thread Robert Nettleton 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53494/
---

Review request for Ambari, Jonathan Hurley, Miklos Gergely, and Oliver Szabo.


Bugs: AMBARI-17529
https://issues.apache.org/jira/browse/AMBARI-17529


Repository: ambari


Description
---

This patch resolved AMBARI-17529. 

In the initial implementation of the LogSearch integration layer in Ambari, in 
Ambari 2.4.0, it was possible that multiple requests for a given host/comopnent 
for logging information be queued and sent to the LogSearch Portal Server. 

While these requests are idempotent, it is a slight performance issue to allow 
these requests to occur multiple times.  This is particularly an issue when the 
LogSearch-related PropertyProvider might be called during various Ambari REST 
calls, even if the LogSearch information is not required. 

This patch uses an approach similar to that found in:

org.apache.ambari.server.state.services.MetricsRetrievalService

This patch implements the following:

1. Implements a request cache, to keep track of the current pending requests on 
a host/component basis, and to ensure that only one request for log file 
information occurs on a host/component basis.  
2. Refactors the Runnable implementation used by LogSearch to execute these log 
file information requests, in order to facilitate simpler unit testing.
3. Adds some DEBUG-level log statements, in order to debug any problems that 
might arise in the caching layer in the future.  
4. Adds new unit tests to verify these changes.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 4929747 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalServiceTest.java
 97c59f8 

Diff: https://reviews.apache.org/r/53494/diff/


Testing
---

1. Ran the ambari-server unit test suite ("mvn clean test"), with all tests 
passing.
2. Tested this change in a cluster deployment on top of the recent trunk code, 
and verified that the LogSearch integration UI in Ambari still functions as 
expected with this change applied.


Thanks,

Robert Nettleton

Re: Review Request 53476: Rolling Upgrade From HDP 2.5.x to 2.5.y Doesn't hdp-select ZKFC

2016-11-04 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53476/#review154947
---


Ship it!




Ship It!

- Alejandro Fernandez


On Nov. 4, 2016, 3:22 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53476/
> ---
> 
> (Updated Nov. 4, 2016, 3:22 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18800
> https://issues.apache.org/jira/browse/AMBARI-18800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Install HDP 2.5.0.0 with HDFS in HA mode
> - Perform a rolling upgrade to 2.5.2.0
> 
> At the end of the upgrade, the pre-finalize step fails with the upgrade 
> failing on 2 hosts for ZKFC. Taking a look at the orchestration, the ZKFC is 
> restarted. However, {{hdp-select}} is never invoked:
> 
> {noformat}
> 2016-11-02 18:38:15,577 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,597 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,598 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,600 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,621 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,621 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,623 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,644 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,644 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,646 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,667 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,668 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,669 - In the middle of a stack upgrade/downgrade for 
> Stack HDP and destination version 2.5.2.0-67, determining which hadoop conf 
> dir to use.
> 2016-11-02 18:38:15,669 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,669 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
> exists, will call conf-select on it for version 2.5.2.0-67
> 2016-11-02 18:38:15,669 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.5.2.0-67/0
> 2016-11-02 18:38:15,670 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
> 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
> '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
> 'stderr': -1}
> 2016-11-02 18:38:15,690 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
> already', '')
> 2016-11-02 18:38:15,690 - checked_call[('ambari-python-wrap', 
> '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
> '--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
> 2016-11-02 18:38:15,716 - checked_call returned (0, '')
> 2016-11-02 18:38:15,716 - Ensuring that hadoop has the correct symlink 
> structure
> 2016-11-02 18:38:15,716 - Using hadoop conf dir: 
> /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,718 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,738 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,739 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,826 - In the middle of a stack upgrade/downgrade for 
> Stack HDP and destination version 2.5.2.0-67, determining which hadoop conf 
> dir to use.
> 2016-11-02 18:38:15,826 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,826 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
> exists, will call conf-select on it for version 2.5.2.0-67
> 2016-11-02 18:38:15,826 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.5.2.0-67/0
> 2016-11-02 18:38:15,827 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
> 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
> '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
> 'stderr': -1}
> 2016-11-02 18:38:15,846 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
> already', '')
> 2016-11-02 18:38:15,846 - checked_call[('ambari-python-wrap', 
> '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
> '--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
>

Review Request 53480: Add a cluster version directly as INSTALLED

2016-11-04 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53480/
---

Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and Nate 
Cole.


Bugs: AMBARI-18708
https://issues.apache.org/jira/browse/AMBARI-18708


Repository: ambari


Description
---

When creating a repository for a cluster, allow the API to set directly to 
INSTALLED.  This is the case where bits are known to be preinstalled.

This API call will set direct values for {{cluster_version}} and 
{{host_version}}.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProvider.java
 4e9fd6b 
  ambari-server/src/main/java/org/apache/ambari/server/state/Cluster.java 
b1958ef 
  ambari-server/src/main/java/org/apache/ambari/server/state/Host.java bd6cc0d 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 bb46c5c 
  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
513d924 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ClusterStackVersionResourceProviderTest.java
 9837b0f 
  
ambari-server/src/test/java/org/apache/ambari/server/state/cluster/ClusterTest.java
 dcbc435 

Diff: https://reviews.apache.org/r/53480/diff/


Testing
---

PENDING


Thanks,

Jonathan Hurley

Re: Review Request 53454: AMBARI-18795: Ambari-server: Stack definition to specify which services support credential store

2016-11-04 Thread Nahappan Somasundaram 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53454/
---

(Updated Nov. 4, 2016, 9:53 a.m.)


Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
Levas, and Sumit Mohanty.


Changes
---

Added stack definition for RANGER HDP-2.5 since it extends from common-services 
and will not inherit HDP-2.4 properties. Missed adding it in the original patch.


Bugs: AMBARI-18795
https://issues.apache.org/jira/browse/AMBARI-18795


Repository: ambari


Description
---

AMBARI-18795: Ambari-server: Stack definition to specify which services support 
credential store


Diffs (updated)
-

  ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
811c5e9 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
a409465 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/metainfo.xml 
8f303a5 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/metainfo.xml 
fe97a41 

Diff: https://reviews.apache.org/r/53454/diff/


Testing
---

** Manual testing **

Copied the modified stack definition files to a VM, deployed RANGER, HIVE and 
KNOX for HDP-2.5. Verified in the database that the following entries were made:

ambari=> select service_name, credential_store_supported, 
credential_store_enabled from servicedesiredstate;

 service_name | credential_store_supported | credential_store_enabled
--++--
 HDFS |  0 |0
 HIVE |  1 |0
 KNOX |  1 |0
 MAPREDUCE2   |  0 |0
 PIG  |  0 |0
 SLIDER   |  0 |0
 TEZ  |  0 |0
 YARN |  0 |0
 ZOOKEEPER|  0 |0
 RANGER   |  1 |1


Thanks,

Nahappan Somasundaram

Re: Review Request 53454: AMBARI-18795: Ambari-server: Stack definition to specify which services support credential store

2016-11-04 Thread Robert Levas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53454/#review154932
---


Ship it!




Ship It!

- Robert Levas


On Nov. 3, 2016, 9:33 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53454/
> ---
> 
> (Updated Nov. 3, 2016, 9:33 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
> Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18795
> https://issues.apache.org/jira/browse/AMBARI-18795
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18795: Ambari-server: Stack definition to specify which services 
> support credential store
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
> 811c5e9 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> a409465 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/metainfo.xml 
> 8f303a5 
> 
> Diff: https://reviews.apache.org/r/53454/diff/
> 
> 
> Testing
> ---
> 
> ** Manual testing **
> 
> Copied the modified stack definition files to a VM, deployed RANGER, HIVE and 
> KNOX for HDP-2.5. Verified in the database that the following entries were 
> made:
> 
> ambari=> select service_name, credential_store_supported, 
> credential_store_enabled from servicedesiredstate;
> 
>  service_name | credential_store_supported | credential_store_enabled
> --++--
>  HDFS |  0 |0
>  HIVE |  1 |0
>  KNOX |  1 |0
>  MAPREDUCE2   |  0 |0
>  PIG  |  0 |0
>  SLIDER   |  0 |0
>  TEZ  |  0 |0
>  YARN |  0 |0
>  ZOOKEEPER|  0 |0
>  RANGER   |  1 |1
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Re: Review Request 53454: AMBARI-18795: Ambari-server: Stack definition to specify which services support credential store

2016-11-04 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53454/#review154930
---


Ship it!




Ship It!

- Nate Cole


On Nov. 3, 2016, 9:33 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53454/
> ---
> 
> (Updated Nov. 3, 2016, 9:33 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
> Levas, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-18795
> https://issues.apache.org/jira/browse/AMBARI-18795
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18795: Ambari-server: Stack definition to specify which services 
> support credential store
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/services/RANGER/metainfo.xml 
> 811c5e9 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HIVE/metainfo.xml 
> a409465 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/KNOX/metainfo.xml 
> 8f303a5 
> 
> Diff: https://reviews.apache.org/r/53454/diff/
> 
> 
> Testing
> ---
> 
> ** Manual testing **
> 
> Copied the modified stack definition files to a VM, deployed RANGER, HIVE and 
> KNOX for HDP-2.5. Verified in the database that the following entries were 
> made:
> 
> ambari=> select service_name, credential_store_supported, 
> credential_store_enabled from servicedesiredstate;
> 
>  service_name | credential_store_supported | credential_store_enabled
> --++--
>  HDFS |  0 |0
>  HIVE |  1 |0
>  KNOX |  1 |0
>  MAPREDUCE2   |  0 |0
>  PIG  |  0 |0
>  SLIDER   |  0 |0
>  TEZ  |  0 |0
>  YARN |  0 |0
>  ZOOKEEPER|  0 |0
>  RANGER   |  1 |1
> 
> 
> Thanks,
> 
> Nahappan Somasundaram
> 
>

Re: Review Request 53446: AMBARI-18793: Ambari-server: Support stack inheritance for credential store information

2016-11-04 Thread Robert Levas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53446/#review154921
---


Ship it!




Ship It!

- Robert Levas


On Nov. 3, 2016, 8:30 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53446/
> ---
> 
> (Updated Nov. 3, 2016, 8:30 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18793
> https://issues.apache.org/jira/browse/AMBARI-18793
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18793: Ambari-server: Support stack inheritance for credential store 
> information
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> b6a0da53dfd796dea739da97c994e1fc37a3d155 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/CredentialStoreInfo.java
>  605de999bb1a87dcd74708a14b6cca8ff330a853 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 16042e12017745a674facbe70940ece80f83e56a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/ServiceModuleTest.java
>  47f5eb9b367e76ef7fd81b8c0ee3c8c5d82ea61f 
> 
> Diff: https://reviews.apache.org/r/53446/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [6.599s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Web  SUCCESS [1:14.514s]
> [INFO] Ambari Views .. SUCCESS [0.983s]
> [INFO] Ambari Admin View . SUCCESS [7.031s]
> [INFO] utility ... SUCCESS [0.309s]
> [INFO] ambari-metrics  SUCCESS [0.594s]
> [INFO] Ambari Metrics Common . SUCCESS [3.898s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.938s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.139s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.203s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [3.223s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.378s]
> [INFO] Ambari Metrics Collector .. SUCCESS [8.550s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.966s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.243s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:11.798s]
> [INFO] Ambari Server . SUCCESS [3:01.245s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.651s]
> [INFO] Ambari Agent .. SUCCESS [26.446s]
> [INFO] Ambari Client . SUCCESS [0.049s]
> [INFO] Ambari Python Client .. SUCCESS [0.941s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.149s]
> [INFO] Ambari Shell .. SUCCESS [0.045s]
> [INFO] Ambari Python Shell ... SUCCESS [0.695s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.932s]
> [INFO] ambari-logsearch .. SUCCESS [0.394s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.241s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.363s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [6.995s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [4.183s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.083s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6:57.041s
> [INFO] Finished at: Thu Nov 03 17:29:33 PDT 2016
> [INFO] Final Memory: 305M/1110M
> [INFO] 
> 
> 
> *** 2. mvn -DskipPythonTests -Dtest=*Provider*,*Service*,StackModuleTest **
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 2:41.191s
> [INFO] Finished at: Thu Nov 03 17:14:26 PDT 2016
>

Re: Review Request 53476: Rolling Upgrade From HDP 2.5.x to 2.5.y Doesn't hdp-select ZKFC

2016-11-04 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53476/#review154918
---


Ship it!




Ship It!

- Nate Cole


On Nov. 4, 2016, 11:22 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53476/
> ---
> 
> (Updated Nov. 4, 2016, 11:22 a.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18800
> https://issues.apache.org/jira/browse/AMBARI-18800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Install HDP 2.5.0.0 with HDFS in HA mode
> - Perform a rolling upgrade to 2.5.2.0
> 
> At the end of the upgrade, the pre-finalize step fails with the upgrade 
> failing on 2 hosts for ZKFC. Taking a look at the orchestration, the ZKFC is 
> restarted. However, {{hdp-select}} is never invoked:
> 
> {noformat}
> 2016-11-02 18:38:15,577 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,597 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,598 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,600 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,621 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,621 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,623 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,644 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,644 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,646 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,667 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,668 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,669 - In the middle of a stack upgrade/downgrade for 
> Stack HDP and destination version 2.5.2.0-67, determining which hadoop conf 
> dir to use.
> 2016-11-02 18:38:15,669 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,669 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
> exists, will call conf-select on it for version 2.5.2.0-67
> 2016-11-02 18:38:15,669 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.5.2.0-67/0
> 2016-11-02 18:38:15,670 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
> 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
> '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
> 'stderr': -1}
> 2016-11-02 18:38:15,690 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
> already', '')
> 2016-11-02 18:38:15,690 - checked_call[('ambari-python-wrap', 
> '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
> '--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
> 2016-11-02 18:38:15,716 - checked_call returned (0, '')
> 2016-11-02 18:38:15,716 - Ensuring that hadoop has the correct symlink 
> structure
> 2016-11-02 18:38:15,716 - Using hadoop conf dir: 
> /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,718 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,738 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,739 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,826 - In the middle of a stack upgrade/downgrade for 
> Stack HDP and destination version 2.5.2.0-67, determining which hadoop conf 
> dir to use.
> 2016-11-02 18:38:15,826 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,826 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
> exists, will call conf-select on it for version 2.5.2.0-67
> 2016-11-02 18:38:15,826 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.5.2.0-67/0
> 2016-11-02 18:38:15,827 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
> 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
> '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
> 'stderr': -1}
> 2016-11-02 18:38:15,846 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
> already', '')
> 2016-11-02 18:38:15,846 - checked_call[('ambari-python-wrap', 
> '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
> '--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
> 2016-11-02

Review Request 53476: Rolling Upgrade From HDP 2.5.x to 2.5.y Doesn't hdp-select ZKFC

2016-11-04 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53476/
---

Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-18800
https://issues.apache.org/jira/browse/AMBARI-18800


Repository: ambari


Description
---

- Install HDP 2.5.0.0 with HDFS in HA mode
- Perform a rolling upgrade to 2.5.2.0

At the end of the upgrade, the pre-finalize step fails with the upgrade failing 
on 2 hosts for ZKFC. Taking a look at the orchestration, the ZKFC is restarted. 
However, {{hdp-select}} is never invoked:

{noformat}
2016-11-02 18:38:15,577 - call['ambari-python-wrap /usr/bin/hdp-select status 
hadoop-hdfs-namenode'] {'timeout': 20}
2016-11-02 18:38:15,597 - call returned (0, 'hadoop-hdfs-namenode - 2.5.2.0-67')
2016-11-02 18:38:15,598 - hadoop-hdfs-namenode is currently at version 
2.5.2.0-67
2016-11-02 18:38:15,600 - call['ambari-python-wrap /usr/bin/hdp-select status 
hadoop-hdfs-namenode'] {'timeout': 20}
2016-11-02 18:38:15,621 - call returned (0, 'hadoop-hdfs-namenode - 2.5.2.0-67')
2016-11-02 18:38:15,621 - hadoop-hdfs-namenode is currently at version 
2.5.2.0-67
2016-11-02 18:38:15,623 - call['ambari-python-wrap /usr/bin/hdp-select status 
hadoop-hdfs-namenode'] {'timeout': 20}
2016-11-02 18:38:15,644 - call returned (0, 'hadoop-hdfs-namenode - 2.5.2.0-67')
2016-11-02 18:38:15,644 - hadoop-hdfs-namenode is currently at version 
2.5.2.0-67
2016-11-02 18:38:15,646 - call['ambari-python-wrap /usr/bin/hdp-select status 
hadoop-hdfs-namenode'] {'timeout': 20}
2016-11-02 18:38:15,667 - call returned (0, 'hadoop-hdfs-namenode - 2.5.2.0-67')
2016-11-02 18:38:15,668 - hadoop-hdfs-namenode is currently at version 
2.5.2.0-67
2016-11-02 18:38:15,669 - In the middle of a stack upgrade/downgrade for Stack 
HDP and destination version 2.5.2.0-67, determining which hadoop conf dir to 
use.
2016-11-02 18:38:15,669 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
2016-11-02 18:38:15,669 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
exists, will call conf-select on it for version 2.5.2.0-67
2016-11-02 18:38:15,669 - Checking if need to create versioned conf dir 
/etc/hadoop/2.5.2.0-67/0
2016-11-02 18:38:15,670 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
'--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
'stderr': -1}
2016-11-02 18:38:15,690 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
already', '')
2016-11-02 18:38:15,690 - checked_call[('ambari-python-wrap', 
'/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
'--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
'sudo': True, 'quiet': False}
2016-11-02 18:38:15,716 - checked_call returned (0, '')
2016-11-02 18:38:15,716 - Ensuring that hadoop has the correct symlink structure
2016-11-02 18:38:15,716 - Using hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
2016-11-02 18:38:15,718 - call['ambari-python-wrap /usr/bin/hdp-select status 
hadoop-hdfs-namenode'] {'timeout': 20}
2016-11-02 18:38:15,738 - call returned (0, 'hadoop-hdfs-namenode - 2.5.2.0-67')
2016-11-02 18:38:15,739 - hadoop-hdfs-namenode is currently at version 
2.5.2.0-67
2016-11-02 18:38:15,826 - In the middle of a stack upgrade/downgrade for Stack 
HDP and destination version 2.5.2.0-67, determining which hadoop conf dir to 
use.
2016-11-02 18:38:15,826 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
2016-11-02 18:38:15,826 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
exists, will call conf-select on it for version 2.5.2.0-67
2016-11-02 18:38:15,826 - Checking if need to create versioned conf dir 
/etc/hadoop/2.5.2.0-67/0
2016-11-02 18:38:15,827 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
'--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
'stderr': -1}
2016-11-02 18:38:15,846 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
already', '')
2016-11-02 18:38:15,846 - checked_call[('ambari-python-wrap', 
'/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
'--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
'sudo': True, 'quiet': False}
2016-11-02 18:38:15,867 - checked_call returned (0, '')
2016-11-02 18:38:15,868 - Ensuring that hadoop has the correct symlink structure
2016-11-02 18:38:15,869 - Using hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
2016-11-02 18:38:15,870 - Group['hadoop'] {}
2016-11-02 18:38:15,871 - Group['users'] {}
2016-11-02 18:38:15,871 - User['hive'] {'gid': 'hadoop', 
'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-11-02 18:38:15,872 - User['mapred'] {'gid': 'hadoop', 
'fetch_nonlocal_groups': True, 'groups': ['hadoop']}
2016-11-02 18:38:15,872 - User['ambari-qa'] {'gid':

Re: Review Request 53476: Rolling Upgrade From HDP 2.5.x to 2.5.y Doesn't hdp-select ZKFC

2016-11-04 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53476/#review154916
---




ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 


reason: method was not properly intended (static method vs instance 
method). That was not a syntax error due to fallback logic at script.py


- Dmitro Lisnichenko


On Nov. 4, 2016, 5:22 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53476/
> ---
> 
> (Updated Nov. 4, 2016, 5:22 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18800
> https://issues.apache.org/jira/browse/AMBARI-18800
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - Install HDP 2.5.0.0 with HDFS in HA mode
> - Perform a rolling upgrade to 2.5.2.0
> 
> At the end of the upgrade, the pre-finalize step fails with the upgrade 
> failing on 2 hosts for ZKFC. Taking a look at the orchestration, the ZKFC is 
> restarted. However, {{hdp-select}} is never invoked:
> 
> {noformat}
> 2016-11-02 18:38:15,577 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,597 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,598 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,600 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,621 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,621 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,623 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,644 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,644 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,646 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,667 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,668 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,669 - In the middle of a stack upgrade/downgrade for 
> Stack HDP and destination version 2.5.2.0-67, determining which hadoop conf 
> dir to use.
> 2016-11-02 18:38:15,669 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,669 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
> exists, will call conf-select on it for version 2.5.2.0-67
> 2016-11-02 18:38:15,669 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.5.2.0-67/0
> 2016-11-02 18:38:15,670 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
> 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
> '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
> 'stderr': -1}
> 2016-11-02 18:38:15,690 - call returned (1, '/etc/hadoop/2.5.2.0-67/0 exist 
> already', '')
> 2016-11-02 18:38:15,690 - checked_call[('ambari-python-wrap', 
> '/usr/bin/conf-select', 'set-conf-dir', '--package', 'hadoop', 
> '--stack-version', '2.5.2.0-67', '--conf-version', '0')] {'logoutput': False, 
> 'sudo': True, 'quiet': False}
> 2016-11-02 18:38:15,716 - checked_call returned (0, '')
> 2016-11-02 18:38:15,716 - Ensuring that hadoop has the correct symlink 
> structure
> 2016-11-02 18:38:15,716 - Using hadoop conf dir: 
> /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,718 - call['ambari-python-wrap /usr/bin/hdp-select status 
> hadoop-hdfs-namenode'] {'timeout': 20}
> 2016-11-02 18:38:15,738 - call returned (0, 'hadoop-hdfs-namenode - 
> 2.5.2.0-67')
> 2016-11-02 18:38:15,739 - hadoop-hdfs-namenode is currently at version 
> 2.5.2.0-67
> 2016-11-02 18:38:15,826 - In the middle of a stack upgrade/downgrade for 
> Stack HDP and destination version 2.5.2.0-67, determining which hadoop conf 
> dir to use.
> 2016-11-02 18:38:15,826 - Hadoop conf dir: /usr/hdp/2.5.2.0-67/hadoop/conf
> 2016-11-02 18:38:15,826 - The hadoop conf dir /usr/hdp/2.5.2.0-67/hadoop/conf 
> exists, will call conf-select on it for version 2.5.2.0-67
> 2016-11-02 18:38:15,826 - Checking if need to create versioned conf dir 
> /etc/hadoop/2.5.2.0-67/0
> 2016-11-02 18:38:15,827 - call[('ambari-python-wrap', '/usr/bin/conf-select', 
> 'create-conf-dir', '--package', 'hadoop', '--stack-version', '2.5.2.0-67', 
> '--conf-version', '0')] {'logoutput': False, 'sudo': True, 'quiet': False, 
> 'stderr': -1}
> 2016-11-02 18:38:15,846 - call returned (1,

Re: Review Request 52455: Ambari UI changes to support PAM authentication

2016-11-04 Thread Di Li 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52455/#review154910
---


Ship it!




Ship It!

- Di Li


On Nov. 4, 2016, 9:23 a.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52455/
> ---
> 
> (Updated Nov. 4, 2016, 9:23 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Alexandr Antonenko, and Di Li.
> 
> 
> Bugs: AMBARI-18476
> https://issues.apache.org/jira/browse/AMBARI-18476
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-12263 adds support for PAM as authentication mechanism for accessing 
> Ambari UI/REST. The changes in this review request cover the corresponding 
> changes in the amabri-admin code. 
> Since a new column groupt_type has been added for groups, the UI will display 
> labels for group type and enable/disable group delete/add member 
> functionality based on the group_type instead of the ldap_group flag.
> Patch includes similar changes for users. The user_type will be used to 
> determine if the user can be deleted or if the user's password can be changed.
> 
> 
> Diffs
> -
> 
>   ambari-admin/src/main/resources/ui/admin-web/app/index.html b3f1bd1 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/groups/GroupsEditCtrl.js
>  2e7830b 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/groups/GroupsListCtrl.js
>  1f86393 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
> af22d7f 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Group.js 
> 660306c 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/GroupConstants.js
>  PRE-CREATION 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/UserConstants.js
>  da63c6d 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
> 8271432 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/groups/list.html 
> b359ede 
>   ambari-admin/src/main/resources/ui/admin-web/app/views/users/show.html 
> e576b28 
> 
> Diff: https://reviews.apache.org/r/52455/diff/
> 
> 
> Testing
> ---
> 
> Manual testing. 
> Tests in ambari admin project ran clean.
> PhantomJS 1.9.7 (Linux): Executed 86 of 86 SUCCESS (0.771 secs / 0.75 secs)
> 
> 
> File Attachments
> 
> 
> New patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/10/03/460d7193-8d55-4ce3-96c6-ad3240502e7b__AMBARI-18476.patch
> Final patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/10/04/3d025c3b-1f60-4a0c-9a49-0802a21c99fd__AMBARI-18476.patch
> Patch based on latest code
>   
> https://reviews.apache.org/media/uploaded/files/2016/11/04/7fa50e96-c66b-4c5e-9321-13b623ab0710__AMBARI-18476.patch
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 53446: AMBARI-18793: Ambari-server: Support stack inheritance for credential store information

2016-11-04 Thread Nate Cole 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53446/#review154907
---


Ship it!




Ship It!

- Nate Cole


On Nov. 3, 2016, 8:30 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53446/
> ---
> 
> (Updated Nov. 3, 2016, 8:30 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Robert Levas, Sumit 
> Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18793
> https://issues.apache.org/jira/browse/AMBARI-18793
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-18793: Ambari-server: Support stack inheritance for credential store 
> information
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stack/ServiceModule.java 
> b6a0da53dfd796dea739da97c994e1fc37a3d155 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/CredentialStoreInfo.java
>  605de999bb1a87dcd74708a14b6cca8ff330a853 
>   ambari-server/src/main/java/org/apache/ambari/server/state/ServiceInfo.java 
> 16042e12017745a674facbe70940ece80f83e56a 
>   
> ambari-server/src/test/java/org/apache/ambari/server/stack/ServiceModuleTest.java
>  47f5eb9b367e76ef7fd81b8c0ee3c8c5d82ea61f 
> 
> Diff: https://reviews.apache.org/r/53446/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install -DskipTests **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [6.599s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Web  SUCCESS [1:14.514s]
> [INFO] Ambari Views .. SUCCESS [0.983s]
> [INFO] Ambari Admin View . SUCCESS [7.031s]
> [INFO] utility ... SUCCESS [0.309s]
> [INFO] ambari-metrics  SUCCESS [0.594s]
> [INFO] Ambari Metrics Common . SUCCESS [3.898s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.938s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [1.139s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [1.203s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [3.223s]
> [INFO] Ambari Metrics Storm Sink (Legacy)  SUCCESS [1.378s]
> [INFO] Ambari Metrics Collector .. SUCCESS [8.550s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.966s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.243s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:11.798s]
> [INFO] Ambari Server . SUCCESS [3:01.245s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.651s]
> [INFO] Ambari Agent .. SUCCESS [26.446s]
> [INFO] Ambari Client . SUCCESS [0.049s]
> [INFO] Ambari Python Client .. SUCCESS [0.941s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.149s]
> [INFO] Ambari Shell .. SUCCESS [0.045s]
> [INFO] Ambari Python Shell ... SUCCESS [0.695s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.932s]
> [INFO] ambari-logsearch .. SUCCESS [0.394s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.241s]
> [INFO] Ambari Logsearch Solr Client .. SUCCESS [1.363s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [6.995s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [4.183s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.083s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 6:57.041s
> [INFO] Finished at: Thu Nov 03 17:29:33 PDT 2016
> [INFO] Final Memory: 305M/1110M
> [INFO] 
> 
> 
> *** 2. mvn -DskipPythonTests -Dtest=*Provider*,*Service*,StackModuleTest **
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 2:41.191s
> [INFO] Finished at: Thu Nov 03 17:14:26 PDT 2016
>

Re: Review Request 52455: Ambari UI changes to support PAM authentication

2016-11-04 Thread Sangeeta Ravindran 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52455/
---

(Updated Nov. 4, 2016, 9:23 a.m.)


Review request for Ambari, Alejandro Fernandez, Alexandr Antonenko, and Di Li.


Bugs: AMBARI-18476
https://issues.apache.org/jira/browse/AMBARI-18476


Repository: ambari


Description
---

AMBARI-12263 adds support for PAM as authentication mechanism for accessing 
Ambari UI/REST. The changes in this review request cover the corresponding 
changes in the amabri-admin code. 
Since a new column groupt_type has been added for groups, the UI will display 
labels for group type and enable/disable group delete/add member functionality 
based on the group_type instead of the ldap_group flag.
Patch includes similar changes for users. The user_type will be used to 
determine if the user can be deleted or if the user's password can be changed.


Diffs
-

  ambari-admin/src/main/resources/ui/admin-web/app/index.html b3f1bd1 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/groups/GroupsEditCtrl.js
 2e7830b 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/groups/GroupsListCtrl.js
 1f86393 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
af22d7f 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Group.js 
660306c 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/GroupConstants.js
 PRE-CREATION 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/UserConstants.js
 da63c6d 
  ambari-admin/src/main/resources/ui/admin-web/app/views/groups/edit.html 
8271432 
  ambari-admin/src/main/resources/ui/admin-web/app/views/groups/list.html 
b359ede 
  ambari-admin/src/main/resources/ui/admin-web/app/views/users/show.html 
e576b28 

Diff: https://reviews.apache.org/r/52455/diff/


Testing
---

Manual testing. 
Tests in ambari admin project ran clean.
PhantomJS 1.9.7 (Linux): Executed 86 of 86 SUCCESS (0.771 secs / 0.75 secs)


File Attachments (updated)


New patch
  
https://reviews.apache.org/media/uploaded/files/2016/10/03/460d7193-8d55-4ce3-96c6-ad3240502e7b__AMBARI-18476.patch
Final patch
  
https://reviews.apache.org/media/uploaded/files/2016/10/04/3d025c3b-1f60-4a0c-9a49-0802a21c99fd__AMBARI-18476.patch
Patch based on latest code
  
https://reviews.apache.org/media/uploaded/files/2016/11/04/7fa50e96-c66b-4c5e-9321-13b623ab0710__AMBARI-18476.patch


Thanks,

Sangeeta Ravindran

Re: Review Request 53433: Add orchestration for HOST_ORDERED upgrades

2016-11-04 Thread Dmitro Lisnichenko 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53433/#review154873
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Nov. 3, 2016, 8:53 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53433/
> ---
> 
> (Updated Nov. 3, 2016, 8:53 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmitro Lisnichenko, and 
> Jonathan Hurley.
> 
> 
> Bugs: AMBARI-18680
> https://issues.apache.org/jira/browse/AMBARI-18680
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Adds orchestration for HOST_ORDERED.  Added unit test and uses code that was 
> checked in recently.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClustersImpl.java
>  706a476 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/HostOrderGrouping.java
>  f3dca62 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/stack/upgrade/StageWrapperBuilder.java
>  267133b 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/host-ordered-upgrade.xml
>  d59be50 
>   ambari-server/src/main/resources/upgrade-pack.xsd cfccb29 
>   
> ambari-server/src/test/java/org/apache/ambari/server/state/UpgradeHelperTest.java
>  52e6513 
>   
> ambari-server/src/test/resources/stacks/HDP/2.1.1/upgrades/upgrade_test_host_ordered.xml
>  b7ffb89 
>   
> ambari-server/src/test/resources/stacks/HDP/2.2.0/upgrades/upgrade_test_host_ordered.xml
>  6a49472 
> 
> Diff: https://reviews.apache.org/r/53433/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated:
> 
> (Failed test has nothing to do with current patch - the test is flaky)
> 
> Failed tests:
>   TestHeartbeatHandler.testRegistrationRecoveryConfig:432 null
> 
> Tests run: 4723, Failures: 1, Errors: 0, Skipped: 42
> 
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> [INFO] Total time: 34:27.732s
> [INFO] Finished at: Thu Nov 03 12:06:26 EDT 2016
> [INFO] Final Memory: 37M/651M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 53334: AMBARI-18758. Livy configuration change for upgrade

2016-11-04 Thread Jeff Zhang 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53334/
---

(Updated Nov. 4, 2016, 6:04 a.m.)


Review request for Ambari, Alejandro Fernandez, Robert Levas, and Sumit Mohanty.


Bugs: AMBARI-18758
https://issues.apache.org/jira/browse/AMBARI-18758


Repository: ambari


Description
---

2 configuration key are changed.

livy.server.kerberos.keytab  --> livy.server.launch.kerberos.keytab
livy.server.kerberos.principal  --> livy.server.launch.kerberos.principal


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/package/scripts/params.py
 3a1c482 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/SPARK/kerberos.json 
PRE-CREATION 

Diff: https://reviews.apache.org/r/53334/diff/


Testing
---

I have verified it in HDP 2.6. But havn't done for the upgrade.

@afernandez, do you know how to change the configuration key for upgrade ? I 
didn't find samples in config-upgrade.xml


Thanks,

Jeff Zhang