Re: Review Request 57582: Create a prototype of ambari-server swagger integration

2017-03-14 Thread Jaimin Jetly

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57582/
---

(Updated March 14, 2017, 6:26 p.m.)


Review request for Ambari, Jayush Luniya, Robert Nettleton, Sumit Mohanty, Sid 
Wagle, and Yusaku Sako.


Bugs: AMBARI-20436
https://issues.apache.org/jira/browse/AMBARI-20436


Repository: ambari


Description (updated)
---

A separate branch is created for this work: ambari-rest-api-explorer

## As part of this task, following changes are done:
1. Users, Groups and Views API are integrated with swagger and exposed from 
ambari rest api explorer ui (swagger ui) on a deployed cluster at path: 
http://c6404.ambari.apache.org:8080/api-docs
2. swagger-maven-plugin is used to generate swagger.json file on compile time. 
This file is published in web resources directory. Note that this file is 
generated build time and will be available on deployed ambari-server host at 
web resources location but it is not yet decided to be committed and maintained 
in Ambari source code
3. swagger2markup-maven-plugin is used to generate asciidoc from swagger.json 
file (that can be shown as markdown in github). More information about this 
format an be found at http://asciidoc.org/ and http://asciidoctor.org/. This 
generates files in docs/api/asciidoc/** location at build time. This directory 
is currently intended to be committed and maintained in ambari source code
4. swagger-ui (version: v2.1.1-M2) compiled code with the different css skin 
(adopted from [link|https://github.com/jensoleg/swagger-ui]) is committed to 
ambari-web/api-docs directory with certain modification to make it work with 
ambari api. Further ui polishing will be done in subsequent tasks. Also there 
is a strong possibility to maintain the fork code of swagger-ui and compile 
(minify and concanate) it during ambari compile time rather than directly using 
swagger-ui dist files. Doing so will help developers when customization done 
over swagger-ui will increase
5. swagger-annotation expects application to define schema of request body and 
response for each endpoint to be encapsulated in a class. While Ambari follows 
this pattern for some of the endpoint, there are many others which does not do 
so. For The ones which do not does so, new request and response classes were 
defined. Going forward at the completion of this epic, either each resource 
type or each resource provider should be coupled with a resource response class 
and a resource request class. As part of this patch, each resourceprovider 
worked upon introduces a new method "getResponse". At completion of this epic 
ResourceProvider interface should also declare methods like "getResponse" and 
"getRequest" that returns response schema and request schema instances for the 
resource API endpoints
6. Currently it seems that swagger has a limitation in supporting  "subresource 
locator methods". This issue is been reported to swagger community and is being 
tracked at [link|https://github.com/swagger-api/swagger-core/issues/2136]. As a 
result of which currently as a temporary workaround, all subresources are 
converted to root resources. Also all root resources on similar path are moved 
under same subpackages (like userApi, groupApi and viewApi).


Diffs
-

  LICENSE.txt f05016f 
  ambari-server/docs/api/asciidoc/definitions.adoc PRE-CREATION 
  ambari-server/docs/api/asciidoc/overview.adoc PRE-CREATION 
  ambari-server/docs/api/asciidoc/paths.adoc PRE-CREATION 
  ambari-server/pom.xml f0c73e4 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ActiveWidgetLayoutService.java
 40cd6e0 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupPrivilegeService.java
 9c1f1a3 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupService.java
 8ec0097 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/MemberService.java
 28e53e6 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/UserAuthorizationService.java
 26a7107 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/UserPrivilegeService.java
 f9c95e7 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/UserService.java
 31f3a8c 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewDataMigrationService.java
 2a9aa64 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewExternalSubResourceService.java
 66ccae7 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewInstanceService.java
 53d4918 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewPermissionService.java
 8f7f4ef 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewPrivilegeService.java
 7393745 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewService.java
 17a9f34 
  

Re: Review Request 57453: Fix up tez view version

2017-03-14 Thread Ashwin Rajeev

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57453/#review168915
---


Ship it!




Ship It!

- Ashwin Rajeev


On March 14, 2017, 1:59 a.m., Nitiraj Rathore wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57453/
> ---
> 
> (Updated March 14, 2017, 1:59 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Pallav 
> Kulshreshtha, Rohit Choudhary, Sreenath Somarajapuram, and Ashwin Rajeev.
> 
> 
> Bugs: AMBARI-20370
> https://issues.apache.org/jira/browse/AMBARI-20370
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> tez view version and build removed and short url of auto instance used for 
> property tez.tez-ui.history-url.base in tez-site
> Removed change of tez version from AbstractUpgradeCatalog.
> UpgardeCatalog250 now contains code to change the previous value of 
> tez.tez-ui.history-url.base to 
> http://HOST:PORT/#/main/view/TEZ/tez_cluster_instance
> changed stack_advisor.py in 2.2 and 2.3 for this change to reflect during 
> installation
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalog.java
>  9be541f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  e3afc33 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> d45cf96 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 6d0ce38 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/AbstractUpgradeCatalogTest.java
>  5fa7cac 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  ad21331 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> 5fce011 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> b1ec05b 
> 
> 
> Diff: https://reviews.apache.org/r/57453/diff/2/
> 
> 
> Testing
> ---
> 
> Tested the upgrade from 2.4.0.2 to 2.5.0.0.
> Tested installation of fresh 2.5.0.0
> Added test cases for changes in UpgardeCatalog250.
> updated test_stack_advisor.py test cases.
> 
> 
> Thanks,
> 
> Nitiraj Rathore
> 
>



Re: Review Request 57612: Some random status command takes more than 5 seconds to execute

2017-03-14 Thread Sid Wagle

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57612/#review168930
---


Ship it!




Ship It!

- Sid Wagle


On March 14, 2017, 5:40 p.m., Eugene Chekanskiy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57612/
> ---
> 
> (Updated March 14, 2017, 5:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai and Andrew Onischuk.
> 
> 
> Bugs: AMBARI-20448
> https://issues.apache.org/jira/browse/AMBARI-20448
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> reimporting subprocess and multiprocessing(this one just in case) modules in 
> StatusCommandsExecutor process to avoid bugs.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 3b23f1c 
> 
> 
> Diff: https://reviews.apache.org/r/57612/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test, manual test
> 
> 
> Thanks,
> 
> Eugene Chekanskiy
> 
>



Re: Review Request 57582: Create a prototype of ambari-server swagger integration

2017-03-14 Thread Sid Wagle

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57582/#review168931
---




ambari-server/pom.xml
Lines 1628 (patched)


Is this specific impl a required dep? We probably should shade some 
specific deps, I know we are not doing that currently in ambari.


- Sid Wagle


On March 14, 2017, 6:27 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57582/
> ---
> 
> (Updated March 14, 2017, 6:27 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
> Nettleton, Sumit Mohanty, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-20436
> https://issues.apache.org/jira/browse/AMBARI-20436
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A separate branch is created for this work: ambari-rest-api-explorer
> 
> ## As part of this task, following changes are done:
> 1. Users, Groups and Views API are integrated with swagger and exposed from 
> ambari rest api explorer ui (swagger ui) on a deployed cluster at path: 
> http://c6404.ambari.apache.org:8080/api-docs
> 2. swagger-maven-plugin is used to generate swagger.json file on compile 
> time. This file is published in web resources directory. Note that this file 
> is generated build time and will be available on deployed ambari-server host 
> at web resources location but it is not yet decided to be committed and 
> maintained in Ambari source code
> 3. swagger2markup-maven-plugin is used to generate asciidoc from swagger.json 
> file (that can be shown as markdown in github). More information about this 
> format an be found at http://asciidoc.org/ and http://asciidoctor.org/. This 
> generates files in docs/api/asciidoc/** location at build time. This 
> directory is currently intended to be committed and maintained in ambari 
> source code
> 4. swagger-ui (version: v2.1.1-M2) compiled code with the different css skin 
> (adopted from [link|https://github.com/jensoleg/swagger-ui]) is committed to 
> ambari-web/api-docs directory with certain modification to make it work with 
> ambari api. Further ui polishing will be done in subsequent tasks. Also there 
> is a strong possibility to maintain the fork code of swagger-ui and compile 
> (minify and concanate) it during ambari compile time rather than directly 
> using swagger-ui dist files. Doing so will help developers when customization 
> done over swagger-ui will increase
> 5. swagger-annotation expects application to define schema of request body 
> and response for each endpoint to be encapsulated in a class. While Ambari 
> follows this pattern for some of the endpoint, there are many others which 
> does not do so. For The ones which do not does so, new request and response 
> classes were defined. Going forward at the completion of this epic, either 
> each resource type or each resource provider should be coupled with a 
> resource response class and a resource request class. As part of this patch, 
> each resourceprovider worked upon introduces a new method "getResponse". At 
> completion of this epic ResourceProvider interface should also declare 
> methods like "getResponse" and "getRequest" that returns response schema and 
> request schema instances for the resource API endpoints
> 6. Currently it seems that swagger has a limitation in supporting  
> "subresource locator methods". This issue is been reported to swagger 
> community and is being tracked at 
> [link|https://github.com/swagger-api/swagger-core/issues/2136]. As a result 
> of which currently as a temporary workaround, all subresources are converted 
> to root resources. Also all root resources on similar path are moved under 
> same subpackages (like userApi, groupApi and viewApi).
> 
> 
> Diffs
> -
> 
>   LICENSE.txt f05016f 
>   ambari-server/docs/api/asciidoc/definitions.adoc PRE-CREATION 
>   ambari-server/docs/api/asciidoc/overview.adoc PRE-CREATION 
>   ambari-server/docs/api/asciidoc/paths.adoc PRE-CREATION 
>   ambari-server/pom.xml f0c73e4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ActiveWidgetLayoutService.java
>  40cd6e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupPrivilegeService.java
>  9c1f1a3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupService.java
>  8ec0097 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/MemberService.java
>  28e53e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/UserAuthorizationService.java
>  26a7107 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/UserPrivilegeService.java
>  

Re: Review Request 57612: Some random status command takes more than 5 seconds to execute

2017-03-14 Thread Eugene Chekanskiy

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57612/
---

(Updated March 14, 2017, 5:40 p.m.)


Review request for Ambari, Attila Doroszlai and Andrew Onischuk.


Bugs: AMBARI-20448
https://issues.apache.org/jira/browse/AMBARI-20448


Repository: ambari


Description
---

reimporting subprocess and multiprocessing(this one just in case) modules in 
StatusCommandsExecutor process to avoid bugs.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 3b23f1c 


Diff: https://reviews.apache.org/r/57612/diff/1/


Testing (updated)
---

mvn clean test, manual test


Thanks,

Eugene Chekanskiy



Re: Review Request 57565: Ambari should allow empty string for hive.metastore.uris when running in embedded mode

2017-03-14 Thread Madhuvanthi Radhakrishnan

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57565/#review168923
---


Ship it!




Ship It!

- Madhuvanthi Radhakrishnan


On March 13, 2017, 6:30 p.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57565/
> ---
> 
> (Updated March 13, 2017, 6:30 p.m.)
> 
> 
> Review request for Ambari, Madhuvanthi Radhakrishnan, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-20427
> https://issues.apache.org/jira/browse/AMBARI-20427
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari UI prevents setting a value of null or empty string for 
> hive.metastore.uris
> This is a valid use-case for embedded mode.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/test/python/resource_management/TestLibraryFunctions.py 
> 4e6b6c3 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/get_port_from_url.py
>  ed4586c 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-site.xml
>  3d08867 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/params.py
>  45a8c07 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  fdc96db 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/HIVE/configuration/hive-site.xml
>  856941f 
> 
> 
> Diff: https://reviews.apache.org/r/57565/diff/1/
> 
> 
> Testing
> ---
> 
> Verified on live cluster.
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>



Re: Review Request 57612: Some random status command takes more than 5 seconds to execute

2017-03-14 Thread Attila Doroszlai

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57612/#review168936
---


Ship it!




Ship It!

- Attila Doroszlai


On March 14, 2017, 6:40 p.m., Eugene Chekanskiy wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57612/
> ---
> 
> (Updated March 14, 2017, 6:40 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai and Andrew Onischuk.
> 
> 
> Bugs: AMBARI-20448
> https://issues.apache.org/jira/browse/AMBARI-20448
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> reimporting subprocess and multiprocessing(this one just in case) modules in 
> StatusCommandsExecutor process to avoid bugs.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 3b23f1c 
> 
> 
> Diff: https://reviews.apache.org/r/57612/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test, manual test
> 
> 
> Thanks,
> 
> Eugene Chekanskiy
> 
>



Re: Review Request 57582: Create a prototype of ambari-server swagger integration

2017-03-14 Thread Sid Wagle

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57582/#review168957
---


Ship it!




Instead of ../api/services/viewApi/*classes*
../resources/views/
etc

- Sid Wagle


On March 14, 2017, 11:35 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57582/
> ---
> 
> (Updated March 14, 2017, 11:35 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
> Nettleton, Sumit Mohanty, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-20436
> https://issues.apache.org/jira/browse/AMBARI-20436
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A separate branch is created for this work: ambari-rest-api-explorer
> 
> ## As part of this task, following changes are done:
> 1. Users, Groups and Views API are integrated with swagger and exposed from 
> ambari rest api explorer ui (swagger ui) on a deployed cluster at path: 
> http://c6404.ambari.apache.org:8080/api-docs
> 2. swagger-maven-plugin is used to generate swagger.json file on compile 
> time. This file is published in web resources directory. Note that this file 
> is generated build time and will be available on deployed ambari-server host 
> at web resources location but it is not yet decided to be committed and 
> maintained in Ambari source code
> 3. swagger2markup-maven-plugin is used to generate asciidoc from swagger.json 
> file (that can be shown as markdown in github). More information about this 
> format an be found at http://asciidoc.org/ and http://asciidoctor.org/. This 
> generates files in docs/api/asciidoc/** location at build time. This 
> directory is currently intended to be committed and maintained in ambari 
> source code
> 4. swagger-ui (version: v2.1.1-M2) compiled code with the different css skin 
> (adopted from [link|https://github.com/jensoleg/swagger-ui]) is committed to 
> ambari-web/api-docs directory with certain modification to make it work with 
> ambari api. Further ui polishing will be done in subsequent tasks. Also there 
> is a strong possibility to maintain the fork code of swagger-ui and compile 
> (minify and concanate) it during ambari compile time rather than directly 
> using swagger-ui dist files. Doing so will help developers when customization 
> done over swagger-ui will increase
> 5. swagger-annotation expects application to define schema of request body 
> and response for each endpoint to be encapsulated in a class. While Ambari 
> follows this pattern for some of the endpoint, there are many others which 
> does not do so. For The ones which do not does so, new request and response 
> classes were defined. Going forward at the completion of this epic, either 
> each resource type or each resource provider should be coupled with a 
> resource response class and a resource request class. As part of this patch, 
> each resourceprovider worked upon introduces a new method "getResponse". At 
> completion of this epic ResourceProvider interface should also declare 
> methods like "getResponse" and "getRequest" that returns response schema and 
> request schema instances for the resource API endpoints
> 6. Currently it seems that swagger has a limitation in supporting  
> "subresource locator methods". This issue is been reported to swagger 
> community and is being tracked at 
> [link|https://github.com/swagger-api/swagger-core/issues/2136]. As a result 
> of which currently as a temporary workaround, all subresources are converted 
> to root resources. Also all root resources on similar path are moved under 
> same subpackages (like userApi, groupApi and viewApi).
> 
> 
> Diffs
> -
> 
>   LICENSE.txt f05016f 
>   ambari-server/docs/api/asciidoc/definitions.adoc PRE-CREATION 
>   ambari-server/docs/api/asciidoc/overview.adoc PRE-CREATION 
>   ambari-server/docs/api/asciidoc/paths.adoc PRE-CREATION 
>   ambari-server/pom.xml f0c73e4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ActiveWidgetLayoutService.java
>  40cd6e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupPrivilegeService.java
>  9c1f1a3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupService.java
>  8ec0097 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/MemberService.java
>  28e53e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/UserAuthorizationService.java
>  26a7107 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/UserPrivilegeService.java
>  f9c95e7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/UserService.java
>  31f3a8c 
>   
> 

Re: Review Request 57582: Create a prototype of ambari-server swagger integration

2017-03-14 Thread Jaimin Jetly


> On March 14, 2017, 7:46 p.m., Sid Wagle wrote:
> > ambari-server/pom.xml
> > Lines 1628 (patched)
> > 
> >
> > Is this specific impl a required dep? We probably should shade some 
> > specific deps, I know we are not doing that currently in ambari.

No. I have removed that dependency in the 2nd revision of the patch


- Jaimin


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57582/#review168931
---


On March 14, 2017, 11:35 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57582/
> ---
> 
> (Updated March 14, 2017, 11:35 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
> Nettleton, Sumit Mohanty, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-20436
> https://issues.apache.org/jira/browse/AMBARI-20436
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> A separate branch is created for this work: ambari-rest-api-explorer
> 
> ## As part of this task, following changes are done:
> 1. Users, Groups and Views API are integrated with swagger and exposed from 
> ambari rest api explorer ui (swagger ui) on a deployed cluster at path: 
> http://c6404.ambari.apache.org:8080/api-docs
> 2. swagger-maven-plugin is used to generate swagger.json file on compile 
> time. This file is published in web resources directory. Note that this file 
> is generated build time and will be available on deployed ambari-server host 
> at web resources location but it is not yet decided to be committed and 
> maintained in Ambari source code
> 3. swagger2markup-maven-plugin is used to generate asciidoc from swagger.json 
> file (that can be shown as markdown in github). More information about this 
> format an be found at http://asciidoc.org/ and http://asciidoctor.org/. This 
> generates files in docs/api/asciidoc/** location at build time. This 
> directory is currently intended to be committed and maintained in ambari 
> source code
> 4. swagger-ui (version: v2.1.1-M2) compiled code with the different css skin 
> (adopted from [link|https://github.com/jensoleg/swagger-ui]) is committed to 
> ambari-web/api-docs directory with certain modification to make it work with 
> ambari api. Further ui polishing will be done in subsequent tasks. Also there 
> is a strong possibility to maintain the fork code of swagger-ui and compile 
> (minify and concanate) it during ambari compile time rather than directly 
> using swagger-ui dist files. Doing so will help developers when customization 
> done over swagger-ui will increase
> 5. swagger-annotation expects application to define schema of request body 
> and response for each endpoint to be encapsulated in a class. While Ambari 
> follows this pattern for some of the endpoint, there are many others which 
> does not do so. For The ones which do not does so, new request and response 
> classes were defined. Going forward at the completion of this epic, either 
> each resource type or each resource provider should be coupled with a 
> resource response class and a resource request class. As part of this patch, 
> each resourceprovider worked upon introduces a new method "getResponse". At 
> completion of this epic ResourceProvider interface should also declare 
> methods like "getResponse" and "getRequest" that returns response schema and 
> request schema instances for the resource API endpoints
> 6. Currently it seems that swagger has a limitation in supporting  
> "subresource locator methods". This issue is been reported to swagger 
> community and is being tracked at 
> [link|https://github.com/swagger-api/swagger-core/issues/2136]. As a result 
> of which currently as a temporary workaround, all subresources are converted 
> to root resources. Also all root resources on similar path are moved under 
> same subpackages (like userApi, groupApi and viewApi).
> 
> 
> Diffs
> -
> 
>   LICENSE.txt f05016f 
>   ambari-server/docs/api/asciidoc/definitions.adoc PRE-CREATION 
>   ambari-server/docs/api/asciidoc/overview.adoc PRE-CREATION 
>   ambari-server/docs/api/asciidoc/paths.adoc PRE-CREATION 
>   ambari-server/pom.xml f0c73e4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/ActiveWidgetLayoutService.java
>  40cd6e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupPrivilegeService.java
>  9c1f1a3 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupService.java
>  8ec0097 
>   
> ambari-server/src/main/java/org/apache/ambari/server/api/services/MemberService.java
>  28e53e6 
>   
> 

Re: Review Request 57582: Create a prototype of ambari-server swagger integration

2017-03-14 Thread Jaimin Jetly

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57582/
---

(Updated March 14, 2017, 11:35 p.m.)


Review request for Ambari, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert 
Nettleton, Sumit Mohanty, Sid Wagle, and Yusaku Sako.


Changes
---

Removed a extra jersey dependency from pom.xml


Bugs: AMBARI-20436
https://issues.apache.org/jira/browse/AMBARI-20436


Repository: ambari


Description
---

A separate branch is created for this work: ambari-rest-api-explorer

## As part of this task, following changes are done:
1. Users, Groups and Views API are integrated with swagger and exposed from 
ambari rest api explorer ui (swagger ui) on a deployed cluster at path: 
http://c6404.ambari.apache.org:8080/api-docs
2. swagger-maven-plugin is used to generate swagger.json file on compile time. 
This file is published in web resources directory. Note that this file is 
generated build time and will be available on deployed ambari-server host at 
web resources location but it is not yet decided to be committed and maintained 
in Ambari source code
3. swagger2markup-maven-plugin is used to generate asciidoc from swagger.json 
file (that can be shown as markdown in github). More information about this 
format an be found at http://asciidoc.org/ and http://asciidoctor.org/. This 
generates files in docs/api/asciidoc/** location at build time. This directory 
is currently intended to be committed and maintained in ambari source code
4. swagger-ui (version: v2.1.1-M2) compiled code with the different css skin 
(adopted from [link|https://github.com/jensoleg/swagger-ui]) is committed to 
ambari-web/api-docs directory with certain modification to make it work with 
ambari api. Further ui polishing will be done in subsequent tasks. Also there 
is a strong possibility to maintain the fork code of swagger-ui and compile 
(minify and concanate) it during ambari compile time rather than directly using 
swagger-ui dist files. Doing so will help developers when customization done 
over swagger-ui will increase
5. swagger-annotation expects application to define schema of request body and 
response for each endpoint to be encapsulated in a class. While Ambari follows 
this pattern for some of the endpoint, there are many others which does not do 
so. For The ones which do not does so, new request and response classes were 
defined. Going forward at the completion of this epic, either each resource 
type or each resource provider should be coupled with a resource response class 
and a resource request class. As part of this patch, each resourceprovider 
worked upon introduces a new method "getResponse". At completion of this epic 
ResourceProvider interface should also declare methods like "getResponse" and 
"getRequest" that returns response schema and request schema instances for the 
resource API endpoints
6. Currently it seems that swagger has a limitation in supporting  "subresource 
locator methods". This issue is been reported to swagger community and is being 
tracked at [link|https://github.com/swagger-api/swagger-core/issues/2136]. As a 
result of which currently as a temporary workaround, all subresources are 
converted to root resources. Also all root resources on similar path are moved 
under same subpackages (like userApi, groupApi and viewApi).


Diffs (updated)
-

  LICENSE.txt f05016f 
  ambari-server/docs/api/asciidoc/definitions.adoc PRE-CREATION 
  ambari-server/docs/api/asciidoc/overview.adoc PRE-CREATION 
  ambari-server/docs/api/asciidoc/paths.adoc PRE-CREATION 
  ambari-server/pom.xml f0c73e4 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ActiveWidgetLayoutService.java
 40cd6e0 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupPrivilegeService.java
 9c1f1a3 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/GroupService.java
 8ec0097 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/MemberService.java
 28e53e6 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/UserAuthorizationService.java
 26a7107 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/UserPrivilegeService.java
 f9c95e7 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/UserService.java
 31f3a8c 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewDataMigrationService.java
 2a9aa64 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewExternalSubResourceService.java
 66ccae7 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewInstanceService.java
 53d4918 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewPermissionService.java
 8f7f4ef 
  
ambari-server/src/main/java/org/apache/ambari/server/api/services/ViewPrivilegeService.java
 7393745 
  

Review Request 57625: Minor refactoring and clean up in ambari-server

2017-03-14 Thread Vitalyi Brodetskyi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57625/
---

Review request for Ambari, Robert Levas, Sumit Mohanty, Sid Wagle, and Yusaku 
Sako.


Bugs: AMBARI-20453
https://issues.apache.org/jira/browse/AMBARI-20453


Repository: ambari


Description
---

Minor refactoring and clean up in ambari-server


Diffs
-

  ambari-server/src/main/assemblies/server.xml 768ba68 
  
ambari-server/src/main/java/org/apache/ambari/server/security/CertificateManager.java
 8d54acb 
  ambari-server/src/main/package/rpm/postinstall.sh 1e8e0f0 
  ambari-server/src/main/python/ambari_server/resourceFilesKeeper.py 188f3ff 
  ambari-server/src/main/python/ambari_server/serverConfiguration.py 3dd165b 
  ambari-server/src/main/resources/scripts/check_ambari_permissions.py 
PRE-CREATION 


Diff: https://reviews.apache.org/r/57625/diff/1/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi



Re: Review Request 57451: The Installation of hbase-client failed

2017-03-14 Thread zhangxiaolu zhangxiaolu


> On 三月 9, 2017, 8:43 p.m., Alejandro Fernandez wrote:
> > +1. I agree, especially if HBase client is added on this host, and at a 
> > later time HDFS client is added.
> 
> zhangxiaolu zhangxiaolu wrote:
> Thank you so much for reviewing my patch on reviewboard, and could you 
> commit the patch ? Thanks a lot.

Yes, you're right.Thank you so much for reviewing my patch on reviewboard, and 
could you commit the patch ? Thanks a lot.


- zhangxiaolu


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57451/#review168511
---


On 三月 9, 2017, 8 a.m., zhangxiaolu zhangxiaolu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57451/
> ---
> 
> (Updated 三月 9, 2017, 8 a.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Jaimin Jetly, Srimanth Gunturi, Sid 
> Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-20368
> https://issues.apache.org/jira/browse/AMBARI-20368
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> The problem is caused by the wrong installation order, which means 
> hbase-client will be installed before the installation of hdfs-client. Under 
> such circumstance, installation of hdfs-client will fail because the 
> hadoop-conf-dir does not exist. 
> Two solutions can be provided to solve this problem:
> 1. Creating the hadoop-conf-dir in hbase.py;
> 2. Setting correct installation order in role_command_order.json;
> Considering that hdfs-client may not beinstalled on the node especially when 
> adding new host, therefore, in my opinion, the second choice would be more 
> appropriate.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/hbase.py
>  20894ec 
> 
> 
> Diff: https://reviews.apache.org/r/57451/diff/1/
> 
> 
> Testing
> ---
> 
> done it
> 
> 
> Thanks,
> 
> zhangxiaolu zhangxiaolu
> 
>



Re: Review Request 57580: Enhance Kafka Service Check

2017-03-14 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57580/#review168877
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On March 14, 2017, 3:23 a.m., Wang Jie wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57580/
> ---
> 
> (Updated March 14, 2017, 3:23 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Andrii Tkach, Dmitro 
> Lisnichenko, Jaimin Jetly, Jayush Luniya, and Srimanth Gunturi.
> 
> 
> Bugs: AMBARI-20434
> https://issues.apache.org/jira/browse/AMBARI-20434
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> Kafka Service Check  will create the kafka topic only if the topic doesn't 
> exist.
> if it exists, Kafka Service Check will return success directly.
> After topic is being created, we sense that each time running service check 
> will not reflect the kafka actual state correctly
> 
> Suggestion:
> Every running kafka service check, delete the existed topic and recreate it.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.8.1/package/scripts/service_check.py
>  0f3a417 
>   
> ambari-server/src/main/resources/common-services/KAFKA/0.9.0/configuration/kafka-broker.xml
>  0275358 
> 
> 
> Diff: https://reviews.apache.org/r/57580/diff/1/
> 
> 
> Testing
> ---
> 
> Done
> 
> 
> Thanks,
> 
> Wang Jie
> 
>



Re: Review Request 57563: Kerberos principal creation fails during blueprint install when kdc_hosts is not specified in blueprint

2017-03-14 Thread Robert Levas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57563/#review168881
---




ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
Line 135 (original), 135 (patched)


We cannot force `kerberos-env/kdc-hosts` to require input.  This value is 
only required when when the `kerberos-env/kdc-type` is "mit-kdc", "ipa", or 
"active-directory".  If `kerberos-env/kdc-type` is "none", which indicates that 
the user will manually set up the Kerbeos infrastructure, then 
`kerberos-env/kdc-hosts` is not needed. 

Also, no mater what the value of `kerberos-env/kdc-type` is, if the user 
opts to manaully manage the krb5.conf file (`krb5-conf/manage_krb5_conf` is set 
to "false"), then `kerberos-env/kdc-hosts` is not required.


- Robert Levas


On March 13, 2017, 1:11 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57563/
> ---
> 
> (Updated March 13, 2017, 1:11 p.m.)
> 
> 
> Review request for Ambari, Di Li and Robert Levas.
> 
> 
> Bugs: AMBARI-20386
> https://issues.apache.org/jira/browse/AMBARI-20386
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Blueprint validation does not prompt if kdc_hosts is not specified in the 
> blueprint. After service installation, deployment fails during Create 
> Principal stage. As /etc/krb5.conf is overwritten with blank values.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/KERBEROS/1.10.3-10/configuration/kerberos-env.xml
>  965ee13 
> 
> 
> Diff: https://reviews.apache.org/r/57563/diff/1/
> 
> 
> Testing
> ---
> 
> Manual testing done, if no kdc_hosts is mentioned then validation exception 
> will be thrown during blueprint registration.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>



Review Request 57591: Alert Definition "Last Changed" field is shown incorrect if alert wan't triggered any time

2017-03-14 Thread Oleg Nechiporenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57591/
---

Review request for Ambari and Denys Buzhor.


Bugs: AMBARI-20441
https://issues.apache.org/jira/browse/AMBARI-20441


Repository: ambari


Description
---

* Go to Alert Definition details page for Definition with NONE status (e.g. 
"NameNode Heap Usage (Daily)")

*AR* "Last Changed" is shown as "Jan 01 1970" 
*ER* This field should be empty


Diffs
-

  ambari-web/app/models/alerts/alert_definition.js b55ef1b 
  
ambari-web/test/mixins/main/service/configs/component_actions_by_configs_test.js
 f987c52 
  ambari-web/test/models/alerts/alert_definition_test.js 3d8b62d 


Diff: https://reviews.apache.org/r/57591/diff/1/


Testing
---

20562 passing (21s)
  153 pending


Thanks,

Oleg Nechiporenko



Re: Review Request 57591: Alert Definition "Last Changed" field is shown incorrect if alert wan't triggered any time

2017-03-14 Thread Denys Buzhor

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57591/#review168876
---


Ship it!




Ship It!

- Denys Buzhor


On March 14, 2017, 9:34 a.m., Oleg Nechiporenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57591/
> ---
> 
> (Updated March 14, 2017, 9:34 a.m.)
> 
> 
> Review request for Ambari and Denys Buzhor.
> 
> 
> Bugs: AMBARI-20441
> https://issues.apache.org/jira/browse/AMBARI-20441
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Go to Alert Definition details page for Definition with NONE status (e.g. 
> "NameNode Heap Usage (Daily)")
> 
> *AR* "Last Changed" is shown as "Jan 01 1970" 
> *ER* This field should be empty
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/alerts/alert_definition.js b55ef1b 
>   
> ambari-web/test/mixins/main/service/configs/component_actions_by_configs_test.js
>  f987c52 
>   ambari-web/test/models/alerts/alert_definition_test.js 3d8b62d 
> 
> 
> Diff: https://reviews.apache.org/r/57591/diff/1/
> 
> 
> Testing
> ---
> 
> 20562 passing (21s)
>   153 pending
> 
> 
> Thanks,
> 
> Oleg Nechiporenko
> 
>



Re: Review Request 57555: Need some way to know a plan is vectorized

2017-03-14 Thread DIPAYAN BHOWMICK

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57555/#review168889
---


Ship it!




Ship It!

- DIPAYAN BHOWMICK


On March 13, 2017, noon, Pallav Kulshreshtha wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57555/
> ---
> 
> (Updated March 13, 2017, noon)
> 
> 
> Review request for Ambari, Abhishek Kumar, DIPAYAN BHOWMICK, Gaurav Nagar, 
> Nitiraj Rathore, Rohit Choudhary, and venkat sairam.
> 
> 
> Bugs: AMBARI-20416
> https://issues.apache.org/jira/browse/AMBARI-20416
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> added property for determining vectorization.
> 
> 
> Diffs
> -
> 
>   contrib/views/hive20/src/main/resources/ui/app/components/visual-explain.js 
> fd450cc 
>   contrib/views/hive20/src/main/resources/ui/app/routes/databases.js 02dbcac 
>   
> contrib/views/hive20/src/main/resources/ui/app/routes/databases/database/tables.js
>  f5940e3 
>   
> contrib/views/hive20/src/main/resources/ui/app/templates/components/visual-explain-detail.hbs
>  3df8e94 
>   
> contrib/views/hive20/src/main/resources/ui/app/templates/components/visual-explain.hbs
>  30f9405 
>   
> contrib/views/hive20/src/main/resources/ui/app/utils/hive-explainer/renderer.js
>  b36d7b4 
>   
> contrib/views/hive20/src/main/resources/ui/app/utils/hive-explainer/transformer.js
>  d88de98 
> 
> 
> Diff: https://reviews.apache.org/r/57555/diff/1/
> 
> 
> Testing
> ---
> 
> manually tested.
> 
> 
> Thanks,
> 
> Pallav Kulshreshtha
> 
>



Re: Review Request 57544: Atlas MetaData server start fails while granting permissions to HBase tables after unkerberizing the cluster

2017-03-14 Thread Laszlo Puskas

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57544/#review168895
---


Ship it!




Ship It!

- Laszlo Puskas


On March 12, 2017, 3:37 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57544/
> ---
> 
> (Updated March 12, 2017, 3:37 p.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Laszlo Puskas, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-20408
> https://issues.apache.org/jira/browse/AMBARI-20408
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR
> 1. Deploy HDP-2.5.0.0 with Ambari-2.5.0.0 (secure MIT cluster installed via 
> blueprint)
> 2. Express Upgrade the cluster to 2.6.0.0
> 3. Disable Kerberos
> 4. Observed that Atlas Metadata server start failed with below errors:
> 
> ```
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py",
>  line 249, in 
> MetadataServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 282, in execute
> method(env)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 720, in restart
> self.start(env, upgrade_type=upgrade_type)
>   File 
> "/var/lib/ambari-agent/cache/common-services/ATLAS/0.1.0.2.3/package/scripts/metadata_server.py",
>  line 102, in start
> user=params.hbase_user
>   File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
> line 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 262, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 72, in inner
> result = function(command, **kwargs)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 102, in checked_call
> tries=tries, try_sleep=try_sleep, 
> timeout_kill_strategy=timeout_kill_strategy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 150, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", 
> line 303, in _call
> raise ExecutionFailed(err_msg, code, out, err)
> resource_management.core.exceptions.ExecutionFailed: Execution of 'cat 
> /var/lib/ambari-agent/tmp/atlas_hbase_setup.rb | hbase shell -n' returned 1. 
>  Hortonworks #
> This is MOTD message, added for testing in qe infra
> atlas_titan
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas
> TABLE
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas_titan
> 2 row(s) in 0.2000 seconds
> 
> nil
> TABLE
> ATLAS_ENTITY_AUDIT_EVENTS
> atlas_titan
> 2 row(s) in 0.0030 seconds
> 
> nil
> java exception
> ERROR Java::OrgApacheHadoopHbaseIpc::RemoteWithExtrasException: 
> org.apache.hadoop.hbase.exceptions.UnknownProtocolException: No registered 
> coprocessor service found for name AccessControlService in region 
> hbase:acl,,1480905643891.19e697cf0c4be8a99c54e39aea069b29.
>   at 
> org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:7692)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execServiceOnRegion(RSRpcServices.java:1897)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1879)
>   at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32299)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:187)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:167)
> ```
> # Cause
> When disabling Kerberos, the stack advisor recommendations are not properly 
> applied due to the order of operations and various conditionals.
> 
> # Solution
> Ensure that the stack advisor recommendations are properly applied when 
> disabling Kerberos.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/KerberosHelper.java
>  0e27d03 
>   
> 

Review Request 57600: Services actions menu is not fully displayed under some circumstances

2017-03-14 Thread Andrii Babiichuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57600/
---

Review request for Ambari and Oleg Nechiporenko.


Bugs: AMBARI-20444
https://issues.apache.org/jira/browse/AMBARI-20444


Repository: ambari


Description
---

**STR**
1. Go to service configs page.
2. Before configs are rendered, switch to another service tab (Summary, 
Heatmaps).
3. Expand services actions dropdown.

**Result**
Dropdown is partially hidden.

Also, dropdown isn't displayed at all in IE and Safari.


Diffs
-

  ambari-web/app/styles/theme/bootstrap-ambari.css a0b24e5 
  ambari-web/vendor/scripts/theme/bootstrap-ambari.js e75bd6b 


Diff: https://reviews.apache.org/r/57600/diff/1/


Testing
---

20562 passing (23s)
  153 pending


Thanks,

Andrii Babiichuk



Re: Review Request 57600: Services actions menu is not fully displayed under some circumstances

2017-03-14 Thread Oleg Nechiporenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57600/#review168900
---


Ship it!




Ship It!

- Oleg Nechiporenko


On March 14, 2017, 2:44 p.m., Andrii Babiichuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57600/
> ---
> 
> (Updated March 14, 2017, 2:44 p.m.)
> 
> 
> Review request for Ambari and Oleg Nechiporenko.
> 
> 
> Bugs: AMBARI-20444
> https://issues.apache.org/jira/browse/AMBARI-20444
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **STR**
> 1. Go to service configs page.
> 2. Before configs are rendered, switch to another service tab (Summary, 
> Heatmaps).
> 3. Expand services actions dropdown.
> 
> **Result**
> Dropdown is partially hidden.
> 
> Also, dropdown isn't displayed at all in IE and Safari.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/styles/theme/bootstrap-ambari.css a0b24e5 
>   ambari-web/vendor/scripts/theme/bootstrap-ambari.js e75bd6b 
> 
> 
> Diff: https://reviews.apache.org/r/57600/diff/1/
> 
> 
> Testing
> ---
> 
> 20562 passing (23s)
>   153 pending
> 
> 
> Thanks,
> 
> Andrii Babiichuk
> 
>



Re: Review Request 57590: Zeppelin: service def not using the correct value from hive.server2.support.dynamic.service.discovery for Hive interpreter config

2017-03-14 Thread Prabhjyot Singh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57590/#review168901
---


Ship it!




Ship It!

- Prabhjyot Singh


On March 14, 2017, 2:28 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57590/
> ---
> 
> (Updated March 14, 2017, 2:28 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-20439
> https://issues.apache.org/jira/browse/AMBARI-20439
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Zeppelin: service def not using the correct value from 
> hive.server2.support.dynamic.service.discovery for Hive interpreter config
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  fdc96db17b 
> 
> 
> Diff: https://reviews.apache.org/r/57590/diff/1/
> 
> 
> Testing
> ---
> 
> Manual testing on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>



Re: Review Request 57557: AMBARI-20417. Spark2 Thrift Server cannot be started after upgrade due to incompletely uploaded tar.gz

2017-03-14 Thread Attila Doroszlai


> On March 13, 2017, 6:43 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
> > Line 66 (original), 66 (patched)
> > 
> >
> > Ok, question still applies. This will then make starting slower.

`copy_to_hdfs` will skip copying the file if its identical to the new source 
file.

```
2017-03-14 15:00:52,265 - DFS file 
/hdp/apps/2.5.3.0-37/spark2/spark2-hdp-yarn-archive.tar.gz is identical to 
/tmp/spark2/spark2-hdp-yarn-archive.tar.gz, skipping the copying
```


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57557/#review168791
---


On March 13, 2017, 1:47 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57557/
> ---
> 
> (Updated March 13, 2017, 1:47 p.m.)
> 
> 
> Review request for Ambari, Bikas Saha, Laszlo Puskas, and Sandor Magyari.
> 
> 
> Bugs: AMBARI-20417
> https://issues.apache.org/jira/browse/AMBARI-20417
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix directory permissions even if already present.  Force upload to HDFS.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c8a2b33ff158fe781926bf88c03be471fa9de1ca 
> 
> 
> Diff: https://reviews.apache.org/r/57557/diff/1/
> 
> 
> Testing
> ---
> 
> Manual test according to steps to reproduce.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>



Re: Review Request 57075: hive-site.xml, hbase-site.xml, etc. are not found in class path for Zeppelin

2017-03-14 Thread Prabhjyot Singh

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57075/
---

(Updated March 14, 2017, 12:38 p.m.)


Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and 
Sumit Mohanty.


Changes
---

adding condition for stack upgrade.


Bugs: AMBARI-20200
https://issues.apache.org/jira/browse/AMBARI-20200


Repository: ambari


Description
---

hive-site.xml, hbase-site.xml, etc. are not found in class path for Zeppelin's 
interpreter.

As a result of which JDBC:phoenix  on kerberos mode doesn't work.


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.6/services/ZEPPELIN/configuration/zeppelin-env.xml
 1f65fac 


Diff: https://reviews.apache.org/r/57075/diff/7/

Changes: https://reviews.apache.org/r/57075/diff/6-7/


Testing
---

Manually on CentOS6


Thanks,

Prabhjyot Singh



Re: Review Request 57515: Zeppelin: Spark2 and Livy2 interpreter config disappear during stack upgrade

2017-03-14 Thread Pallav Kulshreshtha

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57515/#review168869
---


Ship it!




Ship It!

- Pallav Kulshreshtha


On March 10, 2017, 5:47 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57515/
> ---
> 
> (Updated March 10, 2017, 5:47 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-20397
> https://issues.apache.org/jira/browse/AMBARI-20397
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During stack upgrade spark2 and livy2 interpreters are not present in 
> zeppelin by default. Typically for a fresh install- when spark2 service is 
> detected , those 2 interpreters are by default created for user out-of-the box
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  a460aecad0 
> 
> 
> Diff: https://reviews.apache.org/r/57515/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested the stack upgrade on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>



Review Request 57589: Duplicate only global notifications of origin alert group when duplicating alert group in the the dialog of Manage Alert Groups

2017-03-14 Thread wang jianfei

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57589/
---

Review request for Ambari, Abhishek Kumar, Aleksandr Kovalenko, Gaurav Nagar, 
Nate Cole, and Rohit Choudhary.


Bugs: AMBARI-20438
https://issues.apache.org/jira/browse/AMBARI-20438


Repository: ambari


Description
---

PROBLEM:
  When you duplicate alert group in the dialog of Manage Alert Groups,the new 
copied alert group just shows the global notifications rather than all 
notifications of the origin alert group.
  
BUG:
  Make it duplicate all the alert notifications of the origin alert group 
rather than duplicate only the global notifications.


Diffs
-

  ambari-web/app/controllers/main/alerts/manage_alert_groups_controller.js 
db65b2a 


Diff: https://reviews.apache.org/r/57589/diff/1/


Testing
---

DONE.


Thanks,

wang jianfei



Review Request 57590: Zeppelin: service def not using the correct value from hive.server2.support.dynamic.service.discovery for Hive interpreter config

2017-03-14 Thread Renjith Kamath

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57590/
---

Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, and 
Sumit Mohanty.


Bugs: AMBARI-20439
https://issues.apache.org/jira/browse/AMBARI-20439


Repository: ambari


Description
---

Zeppelin: service def not using the correct value from 
hive.server2.support.dynamic.service.discovery for Hive interpreter config


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
 fdc96db17b 


Diff: https://reviews.apache.org/r/57590/diff/1/


Testing
---

Manual testing on CentOS


Thanks,

Renjith Kamath



Review Request 57612: Some random status command takes more than 5 seconds to execute

2017-03-14 Thread Eugene Chekanskiy

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57612/
---

Review request for Ambari, Attila Doroszlai and Andrew Onischuk.


Bugs: AMBARI-20448
https://issues.apache.org/jira/browse/AMBARI-20448


Repository: ambari


Description
---

reimporting subprocess and multiprocessing(this one just in case) modules in 
StatusCommandsExecutor process to avoid bugs.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 3b23f1c 


Diff: https://reviews.apache.org/r/57612/diff/1/


Testing
---


Thanks,

Eugene Chekanskiy



Review Request 57602: Clock widget is not displayed

2017-03-14 Thread Andrii Babiichuk

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57602/
---

Review request for Ambari and Denys Buzhor.


Bugs: AMBARI-20446
https://issues.apache.org/jira/browse/AMBARI-20446


Repository: ambari


Description
---

Optional digital clock widget attached to page is hidden underneath the 
navigation bar.


Diffs
-

  ambari-web/app/styles/application.less 151e404 


Diff: https://reviews.apache.org/r/57602/diff/1/


Testing
---

20569 passing (22s)
  153 pending


Thanks,

Andrii Babiichuk



Review Request 57604: YARN service check failed during HDP 2.4-2.6 rolling upgrade with YARN HA enabled

2017-03-14 Thread Dmitro Lisnichenko

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57604/
---

Review request for Ambari, Jonathan Hurley, Nate Cole, and Vinod Kumar 
Vavilapalli.


Bugs: AMBARI-20447
https://issues.apache.org/jira/browse/AMBARI-20447


Repository: ambari


Description
---

The problem with YARN service check failure is that during Rolling upgrade from 
HDP-2.4 to HDP-2.6 (with YARN HA turned on):
# After "core master restart" step, yarn client uses new (HDP-2.6) config and 
fails with Class 
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found . 
Forcing yarn client to use old (HDP-2.4) config until client binary is updated 
helps here
# After "core slave restart" step, using old YARN client config with old YARN 
client binary does not help. NM/RM classpath points to HDP-2.6. App job gets 
scheduled, but then fails with log:

{code}17/03/06 16:39:27 INFO service.AbstractService: Service 
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl failed in state STARTED; 
cause: java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: Class 
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: Class 
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2240)
at 
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:160)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
at 
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at 
org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.serviceStart(AMRMClientImpl.java:186)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at 
org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.serviceStart(AMRMClientAsyncImpl.java:96)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
at 
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.run(ApplicationMaster.java:559)
at 
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:299)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class 
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2208)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2232)
... 9 more
Caused by: java.lang.ClassNotFoundException: Class 
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2114)
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2206)
... 10 more
17/03/06 16:39:27 INFO service.AbstractService: Service 
org.apache.hadoop.yarn.client.api.async.AMRMClientAsync failed in state 
STARTED; cause: java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: Class 
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
java.lang.RuntimeException: java.lang.RuntimeException: 
java.lang.ClassNotFoundException: Class 
org.apache.hadoop.yarn.client.RequestHedgingRMFailoverProxyProvider not found
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:2240)
at 
org.apache.hadoop.yarn.client.RMProxy.createRMFailoverProxyProvider(RMProxy.java:160)
at org.apache.hadoop.yarn.client.RMProxy.createRMProxy(RMProxy.java:93)
at 
org.apache.hadoop.yarn.client.ClientRMProxy.createRMProxy(ClientRMProxy.java:72)
at
{code}
# After yarn client is updated to a new binary, service check works fine.


Bottom line, this is a known problem with DistributedShell - it was never fixed 
to not rely on cluster's configuration. What this means is that client 
configuration changes like this can break DistributedShell apps over upgrades.
Unfortunately nothing we do now can fix this broken upgrade for 
DistributedShell - as to ideally fix it, we have to go back in time and provide 
changes.

We have to do two things
# Disable DistributedShell based service-check when we go from 2.4 > 2.6. The 
RequestHedgingRMFailoverProxyProvider is added in 2.5, so 2.5 > 2.6 is fine.
# Also fix yarn-site.xml starting 2.6 with the following change to avoid this 
in the future. The change is from using $HADOOP_CONF_DIR which is inherited 
from the NodeManager to /etc/hadoop/conf/ which is always tied to the client 
version.
{code}

yarn.application.classpath

Re: Review Request 57557: AMBARI-20417. Spark2 Thrift Server cannot be started after upgrade due to incompletely uploaded tar.gz

2017-03-14 Thread Alejandro Fernandez

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57557/#review168913
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 13, 2017, 12:47 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57557/
> ---
> 
> (Updated March 13, 2017, 12:47 p.m.)
> 
> 
> Review request for Ambari, Bikas Saha, Laszlo Puskas, and Sandor Magyari.
> 
> 
> Bugs: AMBARI-20417
> https://issues.apache.org/jira/browse/AMBARI-20417
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix directory permissions even if already present.  Force upload to HDFS.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK2/2.0.0/package/scripts/spark_service.py
>  c8a2b33ff158fe781926bf88c03be471fa9de1ca 
> 
> 
> Diff: https://reviews.apache.org/r/57557/diff/1/
> 
> 
> Testing
> ---
> 
> Manual test according to steps to reproduce.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>



Review Request 57610: Tokenize kerberos principal name appearing in kerberos rules in exported blueprint

2017-03-14 Thread Amruta Borkar

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57610/
---

Review request for Ambari, Di Li, Robert Nettleton, and Sandor Magyari.


Bugs: AMBARI-20366
https://issues.apache.org/jira/browse/AMBARI-20366


Repository: ambari


Description
---

If blueprint is exported from a kerberos enabled cluster Kerberos rules export 
principal names which contain cluster name and Realm, this exports existing 
cluster name and realm name as tokens and replaces those tokens with new values 
of cluster name and realm during successive cluster deployments.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRenderer.java
 5e19a6c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
 5732a1c 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/ClusterConfigurationRequest.java
 e29417b 
  
ambari-server/src/test/java/org/apache/ambari/server/api/query/render/ClusterBlueprintRendererTest.java
 75ffd31 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
 d160050 
  
ambari-server/src/test/java/org/apache/ambari/server/topology/ClusterConfigurationRequestTest.java
 c97c568 


Diff: https://reviews.apache.org/r/57610/diff/1/


Testing
---

Tested manually.
Modified test cases.


Thanks,

Amruta Borkar



Re: Review Request 57602: Clock widget is not displayed

2017-03-14 Thread Denys Buzhor

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57602/#review168908
---


Ship it!




Ship It!

- Denys Buzhor


On March 14, 2017, 4:50 p.m., Andrii Babiichuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57602/
> ---
> 
> (Updated March 14, 2017, 4:50 p.m.)
> 
> 
> Review request for Ambari and Denys Buzhor.
> 
> 
> Bugs: AMBARI-20446
> https://issues.apache.org/jira/browse/AMBARI-20446
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Optional digital clock widget attached to page is hidden underneath the 
> navigation bar.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/styles/application.less 151e404 
> 
> 
> Diff: https://reviews.apache.org/r/57602/diff/1/
> 
> 
> Testing
> ---
> 
> 20569 passing (22s)
>   153 pending
> 
> 
> Thanks,
> 
> Andrii Babiichuk
> 
>