date:20161020


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53050/#review153375
---


Ship it!




Ship It!

- Dmytro Grinenko


On Oct. 20, 2016, 9:33 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53050/
> ---
> 
> (Updated Oct. 20, 2016, 9:33 a.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18643
> https://issues.apache.org/jira/browse/AMBARI-18643
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Breakout role command order for ZEPPELIN from stacks/HDP and put it into 
> common-services/ZEPPELIN.
>   * Verify and document any issues observed with breaking down stack level 
> role command order to service level.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/role_command_order.json
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 0e1319a 
> 
> Diff: https://reviews.apache.org/r/53050/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53052: Move ATLAS role command order to common-services/ATLAS


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53052/#review153377
---


Ship it!




Ship It!

- Dmytro Grinenko


On Oct. 20, 2016, 9:35 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53052/
> ---
> 
> (Updated Oct. 20, 2016, 9:35 a.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18645
> https://issues.apache.org/jira/browse/AMBARI-18645
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/role_command_order.json
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json 
> edd6d64 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 0e1319a 
> 
> Diff: https://reviews.apache.org/r/53052/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53054: Move AMBARI_INFRA role command order to common-services/AMBARI_INFRA


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53054/#review153378
---


Ship it!




Ship It!

- Dmytro Grinenko


On Oct. 20, 2016, 9:36 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53054/
> ---
> 
> (Updated Oct. 20, 2016, 9:36 a.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18646
> https://issues.apache.org/jira/browse/AMBARI-18646
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Move stacks/HDP/2.0.6/services/AMBARI_INFRA/role_command_order.json to common-
> services/AMBARI-INFRA
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_INFRA/role_command_order.json
>  7246e9e 
> 
> Diff: https://reviews.apache.org/r/53054/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53051: Move LOGSEARCH role command order to common-services/LOGSEARCH


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53051/#review153376
---


Ship it!




Ship It!

- Dmytro Grinenko


On Oct. 20, 2016, 9:34 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53051/
> ---
> 
> (Updated Oct. 20, 2016, 9:34 a.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18644
> https://issues.apache.org/jira/browse/AMBARI-18644
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Move stacks/HDP/2.2/services/LOGSEARCH/role_command_order.json to common-
> services/LOGSEARCH
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/LOGSEARCH/role_command_order.json
>  e294e1b 
> 
> Diff: https://reviews.apache.org/r/53051/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 50803: [Ambari-17981] Integrate Druid With Ambari

2016-10-20 Thread Nishant Bangarwa


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50803/
---

(Updated Oct. 20, 2016, 7:47 a.m.)


Review request for Ambari, Renjith Kamath and Swapan Shridhar.


Summary (updated)
-

[Ambari-17981] Integrate Druid With Ambari


Bugs: Ambari-19781
https://issues.apache.org/jira/browse/Ambari-19781


Repository: ambari


Description
---

Defines Druid as a new service in HDP 2.6


Diffs
-

  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-broker.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-coordinator.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-historical.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-log4j.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-logrotate.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-middlemanager.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-overlord.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-router.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/DRUID/0.9.2/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/broker.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/coordinator.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid_node.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/historical.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/middlemanager.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/overlord.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/params.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/router.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/service_check.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/status_params.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/quicklinks/quicklinks.json
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/themes/theme.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/role_command_order.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/DRUID/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/DRUID/metainfo.xml 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 
1f722dc 
  ambari-server/src/test/python/stacks/2.6/DRUID/test_druid_historical.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.6/configs/default.json PRE-CREATION 

Diff: https://reviews.apache.org/r/50803/diff/


Testing
---

Tested it locally by installing ambari and adding the newly added resources. 
Was able to install druid and start all newly added services.


Thanks,

Nishant Bangarwa

Re: Review Request 50803: [Ambari-17981] Integrate Druid With Ambari

2016-10-20 Thread Nishant Bangarwa


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50803/
---

(Updated Oct. 20, 2016, 8:08 a.m.)


Review request for Ambari, Renjith Kamath and Swapan Shridhar.


Bugs: Ambari-19781
https://issues.apache.org/jira/browse/Ambari-19781


Repository: ambari


Description
---

Defines Druid as a new service in HDP 2.6


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-broker.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-common.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-coordinator.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-historical.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-log4j.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-logrotate.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-middlemanager.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-overlord.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/configuration/druid-router.xml
 PRE-CREATION 
  ambari-server/src/main/resources/common-services/DRUID/0.9.2/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/broker.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/coordinator.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/druid_node.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/historical.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/middlemanager.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/overlord.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/params.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/router.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/service_check.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/package/scripts/status_params.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/quicklinks/quicklinks.json
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/DRUID/0.9.2/themes/theme.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/role_command_order.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/DRUID/kerberos.json 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/DRUID/metainfo.xml 
PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.6/services/stack_advisor.py 
1f722dc 
  ambari-server/src/test/python/stacks/2.6/DRUID/test_druid.py PRE-CREATION 
  ambari-server/src/test/python/stacks/2.6/common/test_stack_advisor.py 
PRE-CREATION 
  ambari-server/src/test/python/stacks/2.6/configs/default.json PRE-CREATION 

Diff: https://reviews.apache.org/r/50803/diff/


Testing
---

Tested it locally by installing ambari and adding the newly added resources. 
Was able to install druid and start all newly added services.


Thanks,

Nishant Bangarwa

Review Request 53051: Move LOGSEARCH role command order to common-services/LOGSEARCH


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53051/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18644
https://issues.apache.org/jira/browse/AMBARI-18644


Repository: ambari


Description
---

Move stacks/HDP/2.2/services/LOGSEARCH/role_command_order.json to common-
services/LOGSEARCH


Diffs
-

  
ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/role_command_order.json
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.2/services/LOGSEARCH/role_command_order.json
 e294e1b 

Diff: https://reviews.apache.org/r/53051/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 53054: Move AMBARI_INFRA role command order to common-services/AMBARI_INFRA


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53054/#review153366
---


Ship it!




Ship It!

- Dmytro Sen


On Окт. 20, 2016, 9:36 д.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53054/
> ---
> 
> (Updated Окт. 20, 2016, 9:36 д.п.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18646
> https://issues.apache.org/jira/browse/AMBARI-18646
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Move stacks/HDP/2.0.6/services/AMBARI_INFRA/role_command_order.json to common-
> services/AMBARI-INFRA
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_INFRA/role_command_order.json
>  7246e9e 
> 
> Diff: https://reviews.apache.org/r/53054/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53050: Move ZEPPELIN role command order to common-services/ZEPPELIN


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53050/#review153363
---


Ship it!




Ship It!

- Dmytro Sen


On Окт. 20, 2016, 9:33 д.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53050/
> ---
> 
> (Updated Окт. 20, 2016, 9:33 д.п.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18643
> https://issues.apache.org/jira/browse/AMBARI-18643
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Breakout role command order for ZEPPELIN from stacks/HDP and put it into 
> common-services/ZEPPELIN.
>   * Verify and document any issues observed with breaking down stack level 
> role command order to service level.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/role_command_order.json
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 0e1319a 
> 
> Diff: https://reviews.apache.org/r/53050/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53052: Move ATLAS role command order to common-services/ATLAS


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53052/#review153365
---


Ship it!




Ship It!

- Dmytro Sen


On Окт. 20, 2016, 9:35 д.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53052/
> ---
> 
> (Updated Окт. 20, 2016, 9:35 д.п.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18645
> https://issues.apache.org/jira/browse/AMBARI-18645
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/role_command_order.json
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json 
> edd6d64 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 0e1319a 
> 
> Diff: https://reviews.apache.org/r/53052/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53051: Move LOGSEARCH role command order to common-services/LOGSEARCH


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53051/#review153364
---


Ship it!




Ship It!

- Dmytro Sen


On Окт. 20, 2016, 9:34 д.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53051/
> ---
> 
> (Updated Окт. 20, 2016, 9:34 д.п.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18644
> https://issues.apache.org/jira/browse/AMBARI-18644
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Move stacks/HDP/2.2/services/LOGSEARCH/role_command_order.json to common-
> services/LOGSEARCH
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/LOGSEARCH/role_command_order.json
>  e294e1b 
> 
> Diff: https://reviews.apache.org/r/53051/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53050: Move ZEPPELIN role command order to common-services/ZEPPELIN


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53050/#review153367
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Oct. 20, 2016, 12:33 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53050/
> ---
> 
> (Updated Oct. 20, 2016, 12:33 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18643
> https://issues.apache.org/jira/browse/AMBARI-18643
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> * Breakout role command order for ZEPPELIN from stacks/HDP and put it into 
> common-services/ZEPPELIN.
>   * Verify and document any issues observed with breaking down stack level 
> role command order to service level.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/role_command_order.json
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 0e1319a 
> 
> Diff: https://reviews.apache.org/r/53050/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 53054: Move AMBARI_INFRA role command order to common-services/AMBARI_INFRA


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53054/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18646
https://issues.apache.org/jira/browse/AMBARI-18646


Repository: ambari


Description
---

Move stacks/HDP/2.0.6/services/AMBARI_INFRA/role_command_order.json to common-
services/AMBARI-INFRA


Diffs
-

  
ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/role_command_order.json
 PRE-CREATION 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_INFRA/role_command_order.json
 7246e9e 

Diff: https://reviews.apache.org/r/53054/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 53051: Move LOGSEARCH role command order to common-services/LOGSEARCH


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53051/#review153368
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Oct. 20, 2016, 12:34 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53051/
> ---
> 
> (Updated Oct. 20, 2016, 12:34 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18644
> https://issues.apache.org/jira/browse/AMBARI-18644
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Move stacks/HDP/2.2/services/LOGSEARCH/role_command_order.json to common-
> services/LOGSEARCH
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/LOGSEARCH/0.5.0/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.2/services/LOGSEARCH/role_command_order.json
>  e294e1b 
> 
> Diff: https://reviews.apache.org/r/53051/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53052: Move ATLAS role command order to common-services/ATLAS


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53052/#review153369
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Oct. 20, 2016, 12:35 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53052/
> ---
> 
> (Updated Oct. 20, 2016, 12:35 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18645
> https://issues.apache.org/jira/browse/AMBARI-18645
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/role_command_order.json
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json 
> edd6d64 
>   ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
> 0e1319a 
> 
> Diff: https://reviews.apache.org/r/53052/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53054: Move AMBARI_INFRA role command order to common-services/AMBARI_INFRA


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53054/#review153370
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Oct. 20, 2016, 12:36 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53054/
> ---
> 
> (Updated Oct. 20, 2016, 12:36 p.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18646
> https://issues.apache.org/jira/browse/AMBARI-18646
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Move stacks/HDP/2.0.6/services/AMBARI_INFRA/role_command_order.json to common-
> services/AMBARI-INFRA
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/AMBARI_INFRA/0.1.0/role_command_order.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/services/AMBARI_INFRA/role_command_order.json
>  7246e9e 
> 
> Diff: https://reviews.apache.org/r/53054/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 53052: Move ATLAS role command order to common-services/ATLAS


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53052/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18645
https://issues.apache.org/jira/browse/AMBARI-18645


Repository: ambari


Description
---


Diffs
-

  
ambari-server/src/main/resources/common-services/ATLAS/0.1.0.2.3/role_command_order.json
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/ATLAS/0.7.0.2.5/role_command_order.json
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.3/role_command_order.json 
edd6d64 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
0e1319a 

Diff: https://reviews.apache.org/r/53052/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 53050: Move ZEPPELIN role command order to common-services/ZEPPELIN


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53050/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-18643
https://issues.apache.org/jira/browse/AMBARI-18643


Repository: ambari


Description
---

* Breakout role command order for ZEPPELIN from stacks/HDP and put it into 
common-services/ZEPPELIN.
  * Verify and document any issues observed with breaking down stack level role 
command order to service level.


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/role_command_order.json
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/role_command_order.json 
0e1319a 

Diff: https://reviews.apache.org/r/53050/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/#review153387
---




ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
 (line 161)


`AmbariPamAuthorization` should be `ambariPamAuthorization` - incorrect 
name due to Ambari naming conventions.


- Robert Levas


On Oct. 17, 2016, 4:50 p.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52369/
> ---
> 
> (Updated Oct. 17, 2016, 4:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.
> 
> 
> Bugs: AMBARI-12263
> https://issues.apache.org/jira/browse/AMBARI-12263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hello Robert,
> 
> How are you doing? 
> 
> We have been working on PAM support into Ambari and have something ready for 
> review. Can you please take a look at the patch and documentation and provide 
> your feedback.
> 
> Please let me know if you have any questions.
> 
> Note: I have added you as a reviewer as i see some authentication related 
> commits under your name.
> 
> Thanks,
> -Vishal
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml d507b82 
>   ambari-server/sbin/ambari-server 762ae19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1fc9dbf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  5e498f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
>  ef28f61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
>  e1aa5ac 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
>  bdd73a6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java 
> e4ed9c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
>  00e233e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
>  26d4da7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
>  b20df8d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
>  aa9f3e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
>  e547f05 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  185bd58 
>   ambari-server/src/main/python/ambari-server.py bb6bc0e 
>   ambari-server/src/main/python/ambari_server/setupActions.py 697bc1d 
>   ambari-server/src/main/python/ambari_server/setupSecurity.py 119a7d8 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 1d55515 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
>   ambari-server/src/main/resources/properties.json eb27878 
>   ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 500c0bf 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java
>  a80cd03 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7b6c3ad 
> 
> Diff: https://reviews.apache.org/r/52369/diff/
> 
> 
> Testing
> ---
> 
> No test cases added at this point.
> 
> 
> File Attachments
> 
> 
> AMBARI-12263.patch_base
>   
> https://reviews.apache.org/media/uploaded/files/2016/10/17/5107a016-3a83-478c-b98c-2f35ecf6cbc5__AMBARI-12263.patch_base
> 
> 
> Thanks,
> 
> Vishal Ghugare
> 
>

Review Request 53060: Authorizations given to roles, should use generic role-based principals rather than hard-coded pseudo-role-based principals


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53060/
---

Review request for Ambari, Aleksandr Kovalenko, DIPAYAN BHOWMICK, Jonathan 
Hurley, Nate Cole, and Sebastian Toader.


Bugs: AMBARI-18635
https://issues.apache.org/jira/browse/AMBARI-18635


Repository: ambari


Description
---

Authorizations given to roles, should use generic role-based principals rather 
than hard-coded resource types.  

Access to views can be assigned to all users with a given role.  The 
implementation for this lead to the creation of hard-coded principals that 
represent the current set of roles. This is not dynamic enough for possibly 
future enhancements where new roles may be created by administrators. 

This needs to be changed such that rather that using the hard-coded 
pseudo-role-principals, the dynamically generated role-principals are to be 
used.

The hard-coded pseudo-role-principals have the following `adminprincipaltype` 
values as opposed to "ROLE":

* ALL.CLUSTER.ADMINISTRATOR
* ALL.CLUSTER.OPERATOR
* ALL.SERVICE.ADMINISTRATOR
* ALL.SERVICE.OPERATOR
* ALL.CLUSTER.USER

These should be removed along with the associated `adminprincipal` records. 

Also, the FE should be updated to set permissions using the dynamic 
role-principals.

Finally, code should be cleaned up to remove unneeded code in 

- 
org.apache.ambari.server.security.authorization.ClusterInheritedPermissionHelper
- 
org.apache.ambari.server.controller.internal.GroupPrivilegeResourceProvider#getResources
- 
org.apache.ambari.server.controller.internal.PrivilegeResourceProvider#toEntity
- 
org.apache.ambari.server.controller.internal.UserPrivilegeResourceProvider#getResources
- 
org.apache.ambari.server.security.authorization.AuthorizationHelper#isAuthorized
- org.apache.ambari.server.view.ViewRegistry#addClusterInheritedPermissions
- ...


Diffs
-

  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/ViewsEditCtrl.js
 bd74b16 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
af22d7f 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionLoader.js
 988986b 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionsSaver.js
 c7b9295 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/View.js 
5bc0509 
  ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/edit.html 
69eb1c1 
  
ambari-admin/src/main/resources/ui/admin-web/test/unit/services/PermissionSaver_test.js
 fa36d98 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ClusterPrivilegeChangeRequestAuditEvent.java
 b28bb2a 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ViewPrivilegeChangeRequestAuditEvent.java
 11c558c 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/PrivilegeEventCreator.java
 5c476c6 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ViewPrivilegeEventCreator.java
 56d35c0 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 56e2398 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariPrivilegeResourceProvider.java
 e5c95cb 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterPrivilegeResourceProvider.java
 8f37764 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupPrivilegeResourceProvider.java
 94d1cad 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PrivilegeResourceProvider.java
 34111df 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
 bdd73a6 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ViewPrivilegeResourceProvider.java
 e5bd224 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PermissionDAO.java 
88d9775 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PrincipalDAO.java 
efbdfab 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PrincipalTypeDAO.java
 7823d56 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/PermissionEntity.java
 f091bab 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/PrincipalTypeEntity.java
 716d4f7 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AuthorizationHelper.java
 8639a2f 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/ClusterInheritedPermissionHelper.java
 9922bb2 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
 a4f0031 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog242.java
 a5276c2

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/
---

(Updated Oct. 20, 2016, 3:50 p.m.)


Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-18505
https://issues.apache.org/jira/browse/AMBARI-18505


Repository: ambari


Description
---

SmartSense status command does an http call without timeout in 1.2 version of
SS.

Ambari agent queues STATUS commands until we end up with HB lost state with
ton of commands queued up.

This should just never happen is commands are getting queued by by the agent
every HB.


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini 914e09a 
  ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
  ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
PRE-CREATION 

Diff: https://reviews.apache.org/r/52420/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 53066: Preview - HDP-2.5 installation allows ZKFC to advertise version



> On Oct. 20, 2016, 7:25 p.m., Dmitro Lisnichenko wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java,
> >  line 127
> > 
> >
> > ComponentInfo refers to previous stack version during upgrade (and was 
> > never reloaded before this patch)

that's why I have to extract it from stack metainfo


- Dmitro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53066/#review153410
---


On Oct. 20, 2016, 7:26 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53066/
> ---
> 
> (Updated Oct. 20, 2016, 7:26 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18651
> https://issues.apache.org/jira/browse/AMBARI-18651
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Previous to HDP-2.5.0.0, ZKFC was not included with hdp-select.  Therefore, 
> the metainfo for ZKFC used {{false}}.  
> It was recently discovered that 2.5.0.0-1154+ has added this capability.
> 
> * Change versionAdvertised
> * Add code to ZKFC python to hdp-select.  This code MUST check the 
> stack_feature to ensure that it is executed ONLY for HDP-2.5+
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
>  87247eb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
>  a07d0e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java
>  983cbdf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
>  3e805a0 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
>  aa0ab0f 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/metainfo.xml 
> a3e4a64 
> 
> Diff: https://reviews.apache.org/r/53066/diff/
> 
> 
> Testing
> ---
> 
> few runs on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/
---

(Updated Oct. 20, 2016, 3:54 p.m.)


Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.


Bugs: AMBARI-18505
https://issues.apache.org/jira/browse/AMBARI-18505


Repository: ambari


Description
---

SmartSense status command does an http call without timeout in 1.2 version of
SS.

Ambari agent queues STATUS commands until we end up with HB lost state with
ton of commands queued up.

This should just never happen is commands are getting queued by by the agent
every HB.


Diffs (updated)
-

  ambari-agent/conf/unix/ambari-agent.ini 914e09a 
  ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
  ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
PRE-CREATION 

Diff: https://reviews.apache.org/r/52420/diff/


Testing
---

unit tests are pending to be added. Patch is not final


Thanks,

Andrew Onischuk

Re: Review Request 53060: Authorizations given to roles, should use generic role-based principals rather than hard-coded pseudo-role-based principals


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53060/#review153412
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PrivilegeResourceProvider.java
 (line 336)


containsKey() could have a null bound to it, right? Should this just try to 
retrieve the entity and null check it, or are you confident that the key won't 
be bound to null.



ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PrivilegeResourceProvider.java
 (line 374)


I like StringUtils() here - I find it more readable.



ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PrincipalDAO.java 
(line 132)


No need to merge() here



ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PrincipalTypeDAO.java
 (line 109)


No need for merge()


- Jonathan Hurley


On Oct. 20, 2016, 10:20 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53060/
> ---
> 
> (Updated Oct. 20, 2016, 10:20 a.m.)
> 
> 
> Review request for Ambari, Aleksandr Kovalenko, DIPAYAN BHOWMICK, Jonathan 
> Hurley, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18635
> https://issues.apache.org/jira/browse/AMBARI-18635
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Authorizations given to roles, should use generic role-based principals 
> rather than hard-coded resource types.  
> 
> Access to views can be assigned to all users with a given role.  The 
> implementation for this lead to the creation of hard-coded principals that 
> represent the current set of roles. This is not dynamic enough for possibly 
> future enhancements where new roles may be created by administrators. 
> 
> This needs to be changed such that rather that using the hard-coded 
> pseudo-role-principals, the dynamically generated role-principals are to be 
> used.
> 
> The hard-coded pseudo-role-principals have the following `adminprincipaltype` 
> values as opposed to "ROLE":
> 
> * ALL.CLUSTER.ADMINISTRATOR
> * ALL.CLUSTER.OPERATOR
> * ALL.SERVICE.ADMINISTRATOR
> * ALL.SERVICE.OPERATOR
> * ALL.CLUSTER.USER
> 
> These should be removed along with the associated `adminprincipal` records. 
> 
> Also, the FE should be updated to set permissions using the dynamic 
> role-principals.
> 
> Finally, code should be cleaned up to remove unneeded code in 
> 
> - 
> org.apache.ambari.server.security.authorization.ClusterInheritedPermissionHelper
> - 
> org.apache.ambari.server.controller.internal.GroupPrivilegeResourceProvider#getResources
> - 
> org.apache.ambari.server.controller.internal.PrivilegeResourceProvider#toEntity
> - 
> org.apache.ambari.server.controller.internal.UserPrivilegeResourceProvider#getResources
> - 
> org.apache.ambari.server.security.authorization.AuthorizationHelper#isAuthorized
> - org.apache.ambari.server.view.ViewRegistry#addClusterInheritedPermissions
> - ...
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/ViewsEditCtrl.js
>  bd74b16 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
> af22d7f 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionLoader.js
>  988986b 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionsSaver.js
>  c7b9295 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/View.js 
> 5bc0509 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/edit.html 
> 69eb1c1 
>   
> ambari-admin/src/main/resources/ui/admin-web/test/unit/services/PermissionSaver_test.js
>  fa36d98 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ClusterPrivilegeChangeRequestAuditEvent.java
>  b28bb2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ViewPrivilegeChangeRequestAuditEvent.java
>  11c558c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/PrivilegeEventCreator.java
>  5c476c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ViewPrivilegeEventCreator.java
>  56d35c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  56e2398 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariPrivilegeResourceProvider.java
>  e5c95cb 
>   
>

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN



> On Oct. 20, 2016, 2:01 a.m., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml,
> >  line 686
> > 
> >
> > 1. Why %.user=% vs op=% ?
> > 2. I was expecting op=% vs op=_%, is that special syntx?
> 
> Aravindan Vijayan wrote:
> 1. We don't want to include a metric like 
> dfs.NNTopUserOpCounts.windowMs=6.op=getFileInfo.user=yarn and exclude a 
> summed up metric like dfs.NNTopUserOpCounts.windowMs=6.op=* in our Top N 
> calculation. 
> 2. op=% allows op=* , whereas op=__* does not. The small yet reasonable 
> hitch is that an op name should have at least 2 characters in length.

In the first point, I meant "We want to include...", not "We don't want to 
include".


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153330
---


On Oct. 18, 2016, 6:35 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 18, 2016, 6:35 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  b6e5da9 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52979: NPE when a non-existent host is provided as part of the host filter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52979/#review153403
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On Oct. 20, 2016, 6:42 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52979/
> ---
> 
> (Updated Oct. 20, 2016, 6:42 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18623
> https://issues.apache.org/jira/browse/AMBARI-18623
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a non-existent host is provided as part of the host filter then server
> throws an NPE.
> 
> API call
> 
> 
> 
> 
> curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
> '{"RequestInfo":{"context":"YARN Service 
> Check","command":"YARN_SERVICE_CHECK"},"Requests/resource_filters":[{"service_name":"YARN","hosts":"hn0-d7d2bf.hdinsight.net"}]}'
>  http://localhost:8080/api/v1/clusters/jeezraspark2zeppelin18/requests
> {
>   "status": 500,
>   "message": "Server Error"
> }
> 
> 
> 
> 
> 15 Oct 2016 01:10:28,655  INFO [ambari-client-thread-1378] 
> AmbariManagementControllerImpl:3762 - Received action execution request, 
> clusterName=jeezraspark2zeppelin18, request=isCommand :true, action :null, 
> command :YARN_SERVICE_CHECK, inputs :{}, resourceFilters: 
> [RequestResourceFilter{serviceName='YARN', componentName='null', 
> hostNames=[hn0-d7d2bf.hdinsight.net]}], exclusive: false, clusterName 
> :jeezraspark2zeppelin18
> 15 Oct 2016 01:10:28,658 ERROR [ambari-client-thread-1378] 
> BaseManagementHandler:71 - Caught a runtime exception while attempting to 
> create a resource: null
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.findHostAndAddServiceCheckAction(AmbariCustomCommandExecutionHelper.java:557)
> at 
> org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.addExecutionCommandsToStage(AmbariCustomCommandExecutionHelper.java:997)
> at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.createAction(AmbariManagementControllerImpl.java:3818)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  faa21cb 
> 
> Diff: https://reviews.apache.org/r/52979/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 53068: Alert Targets Cannot Be Updated Due To Transaction / Cache Timing Issues


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53068/
---

Review request for Ambari, Nate Cole and Robert Levas.


Bugs: AMBARI-18652
https://issues.apache.org/jira/browse/AMBARI-18652


Repository: ambari


Description
---

There is a JPA timing issue WRT to updating alert targets where lazy loaded 
lists outside of a transaction cause the data set on the {{AlertTargetEnity}} 
to be refreshed from the database.

While maintaining the bi-directional relationship between {{AlertTargetEntity}} 
and {{AlertGroupEntity}}, and {{IndirectSet}} from JPA causes JPA to detect a 
stale entity from another transaction. 

STR:
It's more likely to happen when there are active alerts.
- Create a new alert target for _just_ HDFS
- Shut down HDFS
- Try to edit the name of the alert target

It will return a 200 return code, but the data will not have changed.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProvider.java
 48a5183 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertGroupEntity.java
 0d6d151 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertTargetEntity.java
 de21f2d 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProviderTest.java
 bcdb9ce 

Diff: https://reviews.apache.org/r/53068/diff/


Testing
---

Tests run: 4689, Failures: 0, Errors: 0, Skipped: 42

[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 29:00 min
[INFO] Finished at: 2016-10-20T12:37:48-04:00
[INFO] Final Memory: 57M/702M
[INFO] 


Thanks,

Jonathan Hurley

Re: Review Request 53066: Preview - HDP-2.5 installation allows ZKFC to advertise version


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53066/
---

(Updated Oct. 20, 2016, 7:24 p.m.)


Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-18651
https://issues.apache.org/jira/browse/AMBARI-18651


Repository: ambari


Description
---

Previous to HDP-2.5.0.0, ZKFC was not included with hdp-select.  Therefore, the 
metainfo for ZKFC used {{false}}.  It 
was recently discovered that 2.5.0.0-1154+ has added this capability.

* Change versionAdvertised
* Add code to ZKFC python to hdp-select.  This code MUST check the 
stack_feature to ensure that it is executed ONLY for HDP-2.5+


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
 87247eb 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
 a07d0e6 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java
 983cbdf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
 3e805a0 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 aa0ab0f 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/metainfo.xml 
a3e4a64 

Diff: https://reviews.apache.org/r/53066/diff/


Testing (updated)
---

few runs on live cluster


Thanks,

Dmitro Lisnichenko

Review Request 53066: Preview - HDP-2.5 installation allows ZKFC to advertise version


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53066/
---

Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-18651
https://issues.apache.org/jira/browse/AMBARI-18651


Repository: ambari


Description
---

Previous to HDP-2.5.0.0, ZKFC was not included with hdp-select.  Therefore, the 
metainfo for ZKFC used {{false}}.  It 
was recently discovered that 2.5.0.0-1154+ has added this capability.

* Change versionAdvertised
* Add code to ZKFC python to hdp-select.  This code MUST check the 
stack_feature to ensure that it is executed ONLY for HDP-2.5+


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
 87247eb 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
 a07d0e6 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java
 983cbdf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
 3e805a0 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 aa0ab0f 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/metainfo.xml 
a3e4a64 

Diff: https://reviews.apache.org/r/53066/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 52691: Provision actions to happen based only on specified dependencies


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52691/#review153414
---


Ship it!




Ship It!

- Jonathan Hurley


On Oct. 18, 2016, 2:17 p.m., Sandor Magyari wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52691/
> ---
> 
> (Updated Oct. 18, 2016, 2:17 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Laszlo 
> Puskas, Nate Cole, Sumit Mohanty, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18553
> https://issues.apache.org/jira/browse/AMBARI-18553
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Today, the START commands stored in role_command_orders table are bound to 
> multiple stages created based on dependencies between components - a 
> component in the second stage can only begin its START after the entire first 
> stage is done as opposed to just its dependencies in the first stage. This 
> eventually increases the overall blueprint deployment time.
> The goal is to be able to configure a direct dependency based execution model 
> of commands, for now only for Blueprint based deployment commands.
> 
> Implementation:
> ---
> When creating stages we set the commandExecutionType to RoleGraph. In case 
> commandExecutionType is set to DEPENDENCY_ORDERED there's only one stage 
> created. commandExecutionType is persisted into Stage object / entity as 
> well, so ActionScheduler can decide based on commandExecutionType how to 
> execute the stage. In case commandExecutionType is set to DEPENDENCY_ORDERED 
> it will filter out commands having dependencies on other commands 
> IN_PROGRESS. By default commandExecutionType is STAGE_BASED which works as 
> before, creating one or more stages dependening on dependecies.
> DEPENDENCY_ORDERED commandExecutionType is set only in case of START commands 
> initiated by Blueprint deployment and if Ambari property 
> server.stage.command.execution_type = DEPENDENCY_ORDERED.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/ActionScheduler.java
>  8cbfb1e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/CommandExecutionType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/actionmanager/Stage.java 
> f03d8ea 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  5afaba8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/CachedRoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrder.java
>  bbdb808 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandOrderProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/metadata/RoleCommandPair.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/StageEntity.java
>  eaea913 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraph.java
>  c9ab6f9 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactory.java
>  625b168 
>   
> ambari-server/src/main/java/org/apache/ambari/server/stageplanner/RoleGraphFactoryImpl.java
>  5ca4d88 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  34331ee 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 4b64955 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 15a84cd 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 6c3c036 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 570b684 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 170e430 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 1501143 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/KerberosHelperTest.java
>  bd23e00 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
>  d38108f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/StackUpgradeConfigurationMergeTest.java
>  a49fc09 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
>  2ccbcda 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
>  d96e7b5 
>   
>

Re: Review Request 53068: Alert Targets Cannot Be Updated Due To Transaction / Cache Timing Issues

2016-10-20 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53068/#review153421
---


Ship it!




Ship It!

- Nate Cole


On Oct. 20, 2016, 1:02 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53068/
> ---
> 
> (Updated Oct. 20, 2016, 1:02 p.m.)
> 
> 
> Review request for Ambari, Nate Cole and Robert Levas.
> 
> 
> Bugs: AMBARI-18652
> https://issues.apache.org/jira/browse/AMBARI-18652
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> There is a JPA timing issue WRT to updating alert targets where lazy loaded 
> lists outside of a transaction cause the data set on the {{AlertTargetEnity}} 
> to be refreshed from the database.
> 
> While maintaining the bi-directional relationship between 
> {{AlertTargetEntity}} and {{AlertGroupEntity}}, and {{IndirectSet}} from JPA 
> causes JPA to detect a stale entity from another transaction. 
> 
> STR:
> It's more likely to happen when there are active alerts.
> - Create a new alert target for _just_ HDFS
> - Shut down HDFS
> - Try to edit the name of the alert target
> 
> It will return a 200 return code, but the data will not have changed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProvider.java
>  48a5183 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertGroupEntity.java
>  0d6d151 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/AlertTargetEntity.java
>  de21f2d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/AlertTargetResourceProviderTest.java
>  bcdb9ce 
> 
> Diff: https://reviews.apache.org/r/53068/diff/
> 
> 
> Testing
> ---
> 
> Tests run: 4689, Failures: 0, Errors: 0, Skipped: 42
> 
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 29:00 min
> [INFO] Finished at: 2016-10-20T12:37:48-04:00
> [INFO] Final Memory: 57M/702M
> [INFO] 
> 
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN



> On Oct. 20, 2016, 2:01 a.m., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml,
> >  line 686
> > 
> >
> > 1. Why %.user=% vs op=% ?
> > 2. I was expecting op=% vs op=_%, is that special syntx?
> 
> Aravindan Vijayan wrote:
> 1. We don't want to include a metric like 
> dfs.NNTopUserOpCounts.windowMs=6.op=getFileInfo.user=yarn and exclude a 
> summed up metric like dfs.NNTopUserOpCounts.windowMs=6.op=* in our Top N 
> calculation. 
> 2. op=% allows op=* , whereas op=__* does not. The small yet reasonable 
> hitch is that an op name should have at least 2 characters in length.
> 
> Aravindan Vijayan wrote:
> In the first point, I meant "We want to include...", not "We don't want 
> to include".

Got it !


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153330
---


On Oct. 18, 2016, 6:35 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 18, 2016, 6:35 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  b6e5da9 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/#review153406
---



This is preliminary version of the patch. Basically to have the concept 
reviewed. Still have to add unit tests and do more testing.

- Andrew Onischuk


On Oct. 20, 2016, 3:51 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52420/
> ---
> 
> (Updated Oct. 20, 2016, 3:51 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18505
> https://issues.apache.org/jira/browse/AMBARI-18505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SmartSense status command does an http call without timeout in 1.2 version of
> SS.
> 
> Ambari agent queues STATUS commands until we end up with HB lost state with
> ton of commands queued up.
> 
> This should just never happen is commands are getting queued by by the agent
> every HB.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 914e09a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52420/diff/
> 
> 
> Testing
> ---
> 
> unit tests are pending to be added. Patch is not final
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 53070: Add ability to configure sink configs for external AMS


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53070/
---

Review request for Ambari, Andrew Onischuk, Aravindan Vijayan, Sid Wagle, and 
Vitalyi Brodetskyi.


Bugs: AMBARI-18649
https://issues.apache.org/jira/browse/AMBARI-18649


Repository: ambari


Description
---

Since hadoop-metrics2.properties is not configurable for a cluster we need to 
find how we could make this like any other config that can be changed to point 
to External AMS through a blueprint.


Diffs
-

  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/metainfo.xml 
65d166a 

Diff: https://reviews.apache.org/r/53070/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 53066: Preview - HDP-2.5 installation allows ZKFC to advertise version

2016-10-20 Thread Nate Cole


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53066/#review153418
---


Ship it!





ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 (line 200)


No need to check params.version again.


- Nate Cole


On Oct. 20, 2016, 12:26 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53066/
> ---
> 
> (Updated Oct. 20, 2016, 12:26 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18651
> https://issues.apache.org/jira/browse/AMBARI-18651
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Previous to HDP-2.5.0.0, ZKFC was not included with hdp-select.  Therefore, 
> the metainfo for ZKFC used {{false}}.  
> It was recently discovered that 2.5.0.0-1154+ has added this capability.
> 
> * Change versionAdvertised
> * Add code to ZKFC python to hdp-select.  This code MUST check the 
> stack_feature to ensure that it is executed ONLY for HDP-2.5+
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
>  87247eb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
>  a07d0e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java
>  983cbdf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
>  3e805a0 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
>  aa0ab0f 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/metainfo.xml 
> a3e4a64 
> 
> Diff: https://reviews.apache.org/r/53066/diff/
> 
> 
> Testing
> ---
> 
> few runs on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 52979: NPE when a non-existent host is provided as part of the host filter


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52979/
---

(Updated Oct. 20, 2016, 3:42 p.m.)


Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-18623
https://issues.apache.org/jira/browse/AMBARI-18623


Repository: ambari


Description
---

When a non-existent host is provided as part of the host filter then server
throws an NPE.

API call




curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
'{"RequestInfo":{"context":"YARN Service 
Check","command":"YARN_SERVICE_CHECK"},"Requests/resource_filters":[{"service_name":"YARN","hosts":"hn0-d7d2bf.hdinsight.net"}]}'
 http://localhost:8080/api/v1/clusters/jeezraspark2zeppelin18/requests
{
  "status": 500,
  "message": "Server Error"
}




15 Oct 2016 01:10:28,655  INFO [ambari-client-thread-1378] 
AmbariManagementControllerImpl:3762 - Received action execution request, 
clusterName=jeezraspark2zeppelin18, request=isCommand :true, action :null, 
command :YARN_SERVICE_CHECK, inputs :{}, resourceFilters: 
[RequestResourceFilter{serviceName='YARN', componentName='null', 
hostNames=[hn0-d7d2bf.hdinsight.net]}], exclusive: false, clusterName 
:jeezraspark2zeppelin18
15 Oct 2016 01:10:28,658 ERROR [ambari-client-thread-1378] 
BaseManagementHandler:71 - Caught a runtime exception while attempting to 
create a resource: null
java.lang.NullPointerException
at 
org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.findHostAndAddServiceCheckAction(AmbariCustomCommandExecutionHelper.java:557)
at 
org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.addExecutionCommandsToStage(AmbariCustomCommandExecutionHelper.java:997)
at 
org.apache.ambari.server.controller.AmbariManagementControllerImpl.createAction(AmbariManagementControllerImpl.java:3818)


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 faa21cb 

Diff: https://reviews.apache.org/r/52979/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 53066: Preview - HDP-2.5 installation allows ZKFC to advertise version


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53066/#review153424
---




ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
 (lines 72 - 75)


Don't do this; instead use a Provider



ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
 (line 152)


Make this a little easier and just return if the string is blank - no need 
to indent the entire method.



ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
 (lines 304 - 308)


Should we have an event for this to make it more de-coupled? A 
StackUpgradeCompletedEvent ?

This way, the code here and down below gets centralized in a listener and 
you only need to fire the event.



ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java
 (line 108)


JavaDoc.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 (lines 46 - 47)


I don't think this is needed



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 (line 48)


no need to check params.version - the compare_versions does that for you.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 (line 49)


Remove.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 (line 199)


No need to check params.version - compare_versions does that already


- Jonathan Hurley


On Oct. 20, 2016, 12:26 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53066/
> ---
> 
> (Updated Oct. 20, 2016, 12:26 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18651
> https://issues.apache.org/jira/browse/AMBARI-18651
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Previous to HDP-2.5.0.0, ZKFC was not included with hdp-select.  Therefore, 
> the metainfo for ZKFC used {{false}}.  
> It was recently discovered that 2.5.0.0-1154+ has added this capability.
> 
> * Change versionAdvertised
> * Add code to ZKFC python to hdp-select.  This code MUST check the 
> stack_feature to ensure that it is executed ONLY for HDP-2.5+
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
>  87247eb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
>  a07d0e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java
>  983cbdf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
>  3e805a0 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
>  aa0ab0f 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/metainfo.xml 
> a3e4a64 
> 
> Diff: https://reviews.apache.org/r/53066/diff/
> 
> 
> Testing
> ---
> 
> few runs on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 53060: Authorizations given to roles, should use generic role-based principals rather than hard-coded pseudo-role-based principals



> On Oct. 20, 2016, 12:44 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PrivilegeResourceProvider.java,
> >  line 337
> > 
> >
> > containsKey() could have a null bound to it, right? Should this just 
> > try to retrieve the entity and null check it, or are you confident that the 
> > key won't be bound to null.

This seems to be the pattern for similar tasks in the method... but I think I 
like your accessment of this and protect against `null`.  Thanks.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53060/#review153412
---


On Oct. 20, 2016, 10:20 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53060/
> ---
> 
> (Updated Oct. 20, 2016, 10:20 a.m.)
> 
> 
> Review request for Ambari, Aleksandr Kovalenko, DIPAYAN BHOWMICK, Jonathan 
> Hurley, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18635
> https://issues.apache.org/jira/browse/AMBARI-18635
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Authorizations given to roles, should use generic role-based principals 
> rather than hard-coded resource types.  
> 
> Access to views can be assigned to all users with a given role.  The 
> implementation for this lead to the creation of hard-coded principals that 
> represent the current set of roles. This is not dynamic enough for possibly 
> future enhancements where new roles may be created by administrators. 
> 
> This needs to be changed such that rather that using the hard-coded 
> pseudo-role-principals, the dynamically generated role-principals are to be 
> used.
> 
> The hard-coded pseudo-role-principals have the following `adminprincipaltype` 
> values as opposed to "ROLE":
> 
> * ALL.CLUSTER.ADMINISTRATOR
> * ALL.CLUSTER.OPERATOR
> * ALL.SERVICE.ADMINISTRATOR
> * ALL.SERVICE.OPERATOR
> * ALL.CLUSTER.USER
> 
> These should be removed along with the associated `adminprincipal` records. 
> 
> Also, the FE should be updated to set permissions using the dynamic 
> role-principals.
> 
> Finally, code should be cleaned up to remove unneeded code in 
> 
> - 
> org.apache.ambari.server.security.authorization.ClusterInheritedPermissionHelper
> - 
> org.apache.ambari.server.controller.internal.GroupPrivilegeResourceProvider#getResources
> - 
> org.apache.ambari.server.controller.internal.PrivilegeResourceProvider#toEntity
> - 
> org.apache.ambari.server.controller.internal.UserPrivilegeResourceProvider#getResources
> - 
> org.apache.ambari.server.security.authorization.AuthorizationHelper#isAuthorized
> - org.apache.ambari.server.view.ViewRegistry#addClusterInheritedPermissions
> - ...
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/ViewsEditCtrl.js
>  bd74b16 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
> af22d7f 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionLoader.js
>  988986b 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionsSaver.js
>  c7b9295 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/View.js 
> 5bc0509 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/edit.html 
> 69eb1c1 
>   
> ambari-admin/src/main/resources/ui/admin-web/test/unit/services/PermissionSaver_test.js
>  fa36d98 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ClusterPrivilegeChangeRequestAuditEvent.java
>  b28bb2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ViewPrivilegeChangeRequestAuditEvent.java
>  11c558c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/PrivilegeEventCreator.java
>  5c476c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ViewPrivilegeEventCreator.java
>  56d35c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  56e2398 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariPrivilegeResourceProvider.java
>  e5c95cb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterPrivilegeResourceProvider.java
>  8f37764 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupPrivilegeResourceProvider.java
>  94d1cad 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PrivilegeResourceProvider.java
>  34111df 
>   
>

Re: Review Request 53066: Preview - HDP-2.5 installation allows ZKFC to advertise version


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53066/
---

(Updated Oct. 20, 2016, 7:26 p.m.)


Review request for Ambari, Jonathan Hurley and Nate Cole.


Bugs: AMBARI-18651
https://issues.apache.org/jira/browse/AMBARI-18651


Repository: ambari


Description
---

Previous to HDP-2.5.0.0, ZKFC was not included with hdp-select.  Therefore, the 
metainfo for ZKFC used {{false}}.  It 
was recently discovered that 2.5.0.0-1154+ has added this capability.

* Change versionAdvertised
* Add code to ZKFC python to hdp-select.  This code MUST check the 
stack_feature to ensure that it is executed ONLY for HDP-2.5+


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
 87247eb 
  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
 a07d0e6 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java
 983cbdf 
  
ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
 3e805a0 
  
ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 aa0ab0f 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/metainfo.xml 
a3e4a64 

Diff: https://reviews.apache.org/r/53066/diff/


Testing
---

few runs on live cluster


Thanks,

Dmitro Lisnichenko

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/#review153416
---




ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py (line 56)


This looks weird I would think you would need a self.statusCommandsExecutor 
reference to point to this new executor, would this not get lost after 1st 
re-spawn?


- Sid Wagle


On Oct. 20, 2016, 3:54 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52420/
> ---
> 
> (Updated Oct. 20, 2016, 3:54 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18505
> https://issues.apache.org/jira/browse/AMBARI-18505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SmartSense status command does an http call without timeout in 1.2 version of
> SS.
> 
> Ambari agent queues STATUS commands until we end up with HB lost state with
> ton of commands queued up.
> 
> This should just never happen is commands are getting queued by by the agent
> every HB.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 914e09a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52420/diff/
> 
> 
> Testing
> ---
> 
> unit tests are pending to be added. Patch is not final
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN



> On Oct. 20, 2016, 2:01 a.m., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml,
> >  line 686
> > 
> >
> > 1. Why %.user=% vs op=% ?
> > 2. I was expecting op=% vs op=_%, is that special syntx?

1. We don't want to include a metric like 
dfs.NNTopUserOpCounts.windowMs=6.op=getFileInfo.user=yarn and exclude a 
summed up metric like dfs.NNTopUserOpCounts.windowMs=6.op=* in our Top N 
calculation. 
2. op=% allows op=* , whereas op=__* does not. The small yet reasonable hitch 
is that an op name should have at least 2 characters in length.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153330
---


On Oct. 18, 2016, 6:35 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 18, 2016, 6:35 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  b6e5da9 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval



> On Oct. 20, 2016, 6:11 p.m., Andrew Onischuk wrote:
> > The blacklisting logic we previously were implementing is not needed now 
> > (as disscussed with Sid).
> > 
> > Reasons for that:
> > 1. If command takes more than 5 seconds, the executor will just restart 
> > itself and continue to execute other status commands. 
> > 2. Also since status commands run parallel to the agent as of now, we won't 
> > get into situations where some timeouted commands, can make heartbeat hang, 
> > the results will just come back with the next heartbeat.

3. Status commands queue cannot get full due to timeouted commands. Because 
with every new pack of status commands, old commands get removed from queue.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/#review153434
---


On Oct. 20, 2016, 3:54 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52420/
> ---
> 
> (Updated Oct. 20, 2016, 3:54 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18505
> https://issues.apache.org/jira/browse/AMBARI-18505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SmartSense status command does an http call without timeout in 1.2 version of
> SS.
> 
> Ambari agent queues STATUS commands until we end up with HB lost state with
> ton of commands queued up.
> 
> This should just never happen is commands are getting queued by by the agent
> every HB.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 914e09a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52420/diff/
> 
> 
> Testing
> ---
> 
> unit tests are pending to be added. Patch is not final
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 53066: Preview - HDP-2.5 installation allows ZKFC to advertise version

2016-10-20 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53066/#review153439
---




ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
 (line 112)


Please add some javadoc for how this is supposed to be used.



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
 (line 48)


This should use the stack_features.json file
since this python file is shared with older stacks.


- Alejandro Fernandez


On Oct. 20, 2016, 4:26 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53066/
> ---
> 
> (Updated Oct. 20, 2016, 4:26 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley and Nate Cole.
> 
> 
> Bugs: AMBARI-18651
> https://issues.apache.org/jira/browse/AMBARI-18651
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Previous to HDP-2.5.0.0, ZKFC was not included with hdp-select.  Therefore, 
> the metainfo for ZKFC used {{false}}.  
> It was recently discovered that 2.5.0.0-1154+ has added this capability.
> 
> * Change versionAdvertised
> * Add code to ZKFC python to hdp-select.  This code MUST check the 
> stack_feature to ensure that it is executed ONLY for HDP-2.5+
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/events/listeners/upgrade/StackVersionListener.java
>  87247eb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/upgrades/FinalizeUpgradeAction.java
>  a07d0e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponent.java
>  983cbdf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/ServiceComponentImpl.java
>  3e805a0 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/zkfc_slave.py
>  aa0ab0f 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/HDFS/metainfo.xml 
> a3e4a64 
> 
> Diff: https://reviews.apache.org/r/53066/diff/
> 
> 
> Testing
> ---
> 
> few runs on live cluster
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/#review153434
---



The blacklisting logic we previously were implementing is not needed now (as 
disscussed with Sid).

Reasons for that:
1. If command takes more than 5 seconds, the executor will just restart itself 
and continue to execute other status commands. 
2. Also since status commands run parallel to the agent as of now, we won't get 
into situations where some timeouted commands, can make heartbeat hang, the 
results will just come back with the next heartbeat.

- Andrew Onischuk


On Oct. 20, 2016, 3:54 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52420/
> ---
> 
> (Updated Oct. 20, 2016, 3:54 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18505
> https://issues.apache.org/jira/browse/AMBARI-18505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SmartSense status command does an http call without timeout in 1.2 version of
> SS.
> 
> Ambari agent queues STATUS commands until we end up with HB lost state with
> ton of commands queued up.
> 
> This should just never happen is commands are getting queued by by the agent
> every HB.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 914e09a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52420/diff/
> 
> 
> Testing
> ---
> 
> unit tests are pending to be added. Patch is not final
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52636: Implement instrumented Lock for profiling/logging



> On Oct. 20, 2016, 1:34 p.m., Jonathan Hurley wrote:
> > Ship It!

branch-feature-AMBARI-18456 and trunk are in-sync - feel free to commit to 
trunk and branch-2.5.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52636/#review153428
---


On Oct. 7, 2016, 4:21 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52636/
> ---
> 
> (Updated Oct. 7, 2016, 4:21 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Sandor Magyari, 
> and Sebastian Toader.
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Create `Lock` and `ReadWriteLock` implementation for profiling and logging 
> lock usage
> 2. Introduce `LockFactory` that creates regular or instrumented locks based 
> on Ambari Server configuration
> 3. As sample usage, replace locks in `ClusterImpl` with the ones created by 
> `LockFactory`
> 4. Allows on-demand dump of lock profiler summary stats as part of the debug 
> log for `/api/v1/clusters` requests (This is a temporary solution.  Once the 
> factory is used in more places, the dump should be triggered by some other 
> request unrelated to the clusters.)
> 
> Configuration:
> 
>  * `server.locks.profiling=true` enables usage of instrumented locks instead 
> of regular ones (goes in `ambari.properties`)
>  * 
> `log4j.logger.org.apache.ambari.server.controller.AmbariManagementControllerImpl=DEBUG`
>  required for the on-demand dump (goes in `log4j.properties`)
>  * `log4j.logger.org.apache.ambari.server.logging=DEBUG` enables logging of 
> individual "requested", "acquired", "released" events. Only the outermost 
> lock/unlock call is logged, reentrant calls are ignored (goes in 
> `log4j.properties`)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/LockFactory.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/LockProfileDelegate.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  2f7d6b9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
>  a56ace5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/logging/LockFactoryTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLockTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52636/diff/
> 
> 
> Testing
> ---
> 
> * Manual testing: cluster creation via blueprint both with and without 
> `server.locks.profiling=true` setting
>  * Unit tests:
>Tests run: 56, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.968 sec 
> - in org.apache.ambari.server.configuration.ConfigurationTest
>Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 sec 
> - in org.apache.ambari.server.logging.DelegatingLockTest
>Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.279 sec 
> - in org.apache.ambari.server.logging.LockFactoryTest
>Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec 
> - in org.apache.ambari.server.logging.ProfiledReentrantReadWriteLockTest
>  * Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
> approved: 5012 licence.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval



> On Oct. 20, 2016, 4:52 p.m., Sid Wagle wrote:
> > ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py, line 56
> > 
> >
> > This looks weird I would think you would need a 
> > self.statusCommandsExecutor reference to point to this new executor, would 
> > this not get lost after 1st re-spawn?

Dropped my previous comment since it is not exactly valid but there is no 
reference to the newly respawned executor right? I think the controller should 
always have a handle on the executor.


- Sid


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/#review153416
---


On Oct. 20, 2016, 3:54 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52420/
> ---
> 
> (Updated Oct. 20, 2016, 3:54 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18505
> https://issues.apache.org/jira/browse/AMBARI-18505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SmartSense status command does an http call without timeout in 1.2 version of
> SS.
> 
> Ambari agent queues STATUS commands until we end up with HB lost state with
> ton of commands queued up.
> 
> This should just never happen is commands are getting queued by by the agent
> every HB.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 914e09a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52420/diff/
> 
> 
> Testing
> ---
> 
> unit tests are pending to be added. Patch is not final
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval

2016-10-20 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/#review153444
---




ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py (line 38)


Please add a comment that this is in seconds.


- Alejandro Fernandez


On Oct. 20, 2016, 3:54 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52420/
> ---
> 
> (Updated Oct. 20, 2016, 3:54 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18505
> https://issues.apache.org/jira/browse/AMBARI-18505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SmartSense status command does an http call without timeout in 1.2 version of
> SS.
> 
> Ambari agent queues STATUS commands until we end up with HB lost state with
> ton of commands queued up.
> 
> This should just never happen is commands are getting queued by by the agent
> every HB.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 914e09a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52420/diff/
> 
> 
> Testing
> ---
> 
> unit tests are pending to be added. Patch is not final
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52636: Implement instrumented Lock for profiling/logging


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52636/#review153428
---


Ship it!




Ship It!

- Jonathan Hurley


On Oct. 7, 2016, 4:21 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52636/
> ---
> 
> (Updated Oct. 7, 2016, 4:21 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Sandor Magyari, 
> and Sebastian Toader.
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Create `Lock` and `ReadWriteLock` implementation for profiling and logging 
> lock usage
> 2. Introduce `LockFactory` that creates regular or instrumented locks based 
> on Ambari Server configuration
> 3. As sample usage, replace locks in `ClusterImpl` with the ones created by 
> `LockFactory`
> 4. Allows on-demand dump of lock profiler summary stats as part of the debug 
> log for `/api/v1/clusters` requests (This is a temporary solution.  Once the 
> factory is used in more places, the dump should be triggered by some other 
> request unrelated to the clusters.)
> 
> Configuration:
> 
>  * `server.locks.profiling=true` enables usage of instrumented locks instead 
> of regular ones (goes in `ambari.properties`)
>  * 
> `log4j.logger.org.apache.ambari.server.controller.AmbariManagementControllerImpl=DEBUG`
>  required for the on-demand dump (goes in `log4j.properties`)
>  * `log4j.logger.org.apache.ambari.server.logging=DEBUG` enables logging of 
> individual "requested", "acquired", "released" events. Only the outermost 
> lock/unlock call is logged, reentrant calls are ignored (goes in 
> `log4j.properties`)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/LockFactory.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/LockProfileDelegate.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  2f7d6b9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
>  a56ace5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/logging/LockFactoryTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLockTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52636/diff/
> 
> 
> Testing
> ---
> 
> * Manual testing: cluster creation via blueprint both with and without 
> `server.locks.profiling=true` setting
>  * Unit tests:
>Tests run: 56, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.968 sec 
> - in org.apache.ambari.server.configuration.ConfigurationTest
>Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 sec 
> - in org.apache.ambari.server.logging.DelegatingLockTest
>Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.279 sec 
> - in org.apache.ambari.server.logging.LockFactoryTest
>Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec 
> - in org.apache.ambari.server.logging.ProfiledReentrantReadWriteLockTest
>  * Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
> approved: 5012 licence.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 52979: NPE when a non-existent host is provided as part of the host filter

2016-10-20 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52979/#review153440
---


Ship it!




Ship It!

- Alejandro Fernandez


On Oct. 20, 2016, 3:42 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52979/
> ---
> 
> (Updated Oct. 20, 2016, 3:42 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-18623
> https://issues.apache.org/jira/browse/AMBARI-18623
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a non-existent host is provided as part of the host filter then server
> throws an NPE.
> 
> API call
> 
> 
> 
> 
> curl -u admin:admin -H "X-Requested-By: ambari" -X POST -d 
> '{"RequestInfo":{"context":"YARN Service 
> Check","command":"YARN_SERVICE_CHECK"},"Requests/resource_filters":[{"service_name":"YARN","hosts":"hn0-d7d2bf.hdinsight.net"}]}'
>  http://localhost:8080/api/v1/clusters/jeezraspark2zeppelin18/requests
> {
>   "status": 500,
>   "message": "Server Error"
> }
> 
> 
> 
> 
> 15 Oct 2016 01:10:28,655  INFO [ambari-client-thread-1378] 
> AmbariManagementControllerImpl:3762 - Received action execution request, 
> clusterName=jeezraspark2zeppelin18, request=isCommand :true, action :null, 
> command :YARN_SERVICE_CHECK, inputs :{}, resourceFilters: 
> [RequestResourceFilter{serviceName='YARN', componentName='null', 
> hostNames=[hn0-d7d2bf.hdinsight.net]}], exclusive: false, clusterName 
> :jeezraspark2zeppelin18
> 15 Oct 2016 01:10:28,658 ERROR [ambari-client-thread-1378] 
> BaseManagementHandler:71 - Caught a runtime exception while attempting to 
> create a resource: null
> java.lang.NullPointerException
> at 
> org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.findHostAndAddServiceCheckAction(AmbariCustomCommandExecutionHelper.java:557)
> at 
> org.apache.ambari.server.controller.AmbariCustomCommandExecutionHelper.addExecutionCommandsToStage(AmbariCustomCommandExecutionHelper.java:997)
> at 
> org.apache.ambari.server.controller.AmbariManagementControllerImpl.createAction(AmbariManagementControllerImpl.java:3818)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  faa21cb 
> 
> Diff: https://reviews.apache.org/r/52979/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52636: Implement instrumented Lock for profiling/logging

2016-10-20 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52636/
---

(Updated Oct. 20, 2016, 10:08 p.m.)


Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Sandor Magyari, and 
Sebastian Toader.


Changes
---

Added default label


Repository: ambari


Description
---

1. Create `Lock` and `ReadWriteLock` implementation for profiling and logging 
lock usage
2. Introduce `LockFactory` that creates regular or instrumented locks based on 
Ambari Server configuration
3. As sample usage, replace locks in `ClusterImpl` with the ones created by 
`LockFactory`
4. Allows on-demand dump of lock profiler summary stats as part of the debug 
log for `/api/v1/clusters` requests (This is a temporary solution.  Once the 
factory is used in more places, the dump should be triggered by some other 
request unrelated to the clusters.)

Configuration:

 * `server.locks.profiling=true` enables usage of instrumented locks instead of 
regular ones (goes in `ambari.properties`)
 * 
`log4j.logger.org.apache.ambari.server.controller.AmbariManagementControllerImpl=DEBUG`
 required for the on-demand dump (goes in `log4j.properties`)
 * `log4j.logger.org.apache.ambari.server.logging=DEBUG` enables logging of 
individual "requested", "acquired", "released" events. Only the outermost 
lock/unlock call is logged, reentrant calls are ignored (goes in 
`log4j.properties`)


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 1759dc4 
  ambari-server/src/main/java/org/apache/ambari/server/logging/LockFactory.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/logging/LockProfileDelegate.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledLock.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantLock.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLock.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 84697b8 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
 a56ace5 
  
ambari-server/src/test/java/org/apache/ambari/server/logging/LockFactoryTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLockTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/52636/diff/


Testing
---

* Manual testing: cluster creation via blueprint both with and without 
`server.locks.profiling=true` setting
 * Unit tests:
   Tests run: 56, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.968 sec - 
in org.apache.ambari.server.configuration.ConfigurationTest
   Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 sec - 
in org.apache.ambari.server.logging.DelegatingLockTest
   Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.279 sec - 
in org.apache.ambari.server.logging.LockFactoryTest
   Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec - 
in org.apache.ambari.server.logging.ProfiledReentrantReadWriteLockTest
 * Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 
5012 licence.


Thanks,

Attila Doroszlai

Re: Review Request 52999: Incorporate database consistency check into main Ambari process

2016-10-20 Thread Vitalyi Brodetskyi

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52999/
---

(Updated Жов. 20, 2016, 8:42 після полудня)

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.

Bugs: AMBARI-18631
https://issues.apache.org/jira/browse/AMBARI-18631

Repository: ambari

Description
---

Currently the database consistency check runs as a separate process prior
Ambari process. The database consistency checker load various modules needed
for performing the validations. (e.g. load stack definitions to be able to
compare service configs from stack with configs from db).
Once database consistency checker completed Ambari server is started. Ambari
server beside others loads the same modules as database consistency checker.
This double initialisation adds time to the ambari server startup time which
could be reduced if the database consistency check is moved into the ambari
server process.
Moreover the database consistency check may check at the beginning if the
database is empty and perform the checks only if there is data in the database.

Diffs (updated)
-

ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
2d91eca

ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyChecker.java
535d74f

ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
4030ef9
ambari-server/src/main/python/ambari_server_main.py 57ec58d

ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
4663310

ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariServerTest.java
54f6147

Diff: https://reviews.apache.org/r/52999/diff/

Testing
---

UT will be added and checked after patch draft review

Thanks,

Vitalyi Brodetskyi

Re: Review Request 52999: Incorporate database consistency check into main Ambari process

2016-10-20 Thread Vitalyi Brodetskyi



> On Жов. 18, 2016, 10:43 після полудня, Sid Wagle wrote:
> > ambari-server/src/main/python/ambari_server_main.py, line 105
> > 
> >
> > This should be configurable if it isn't.

I don't think we need to make this time configurable. This is the time during 
which we monitoring server process. For 3.0.0 we changed it to 40-50 seconds, 
and we stop monitoring when port 8080 become busy. I set this time to 30 
seconds because i think it's enough to finish db check.


- Vitalyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52999/#review153199
---


On Жов. 20, 2016, 8:42 після полудня, Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52999/
> ---
> 
> (Updated Жов. 20, 2016, 8:42 після полудня)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18631
> https://issues.apache.org/jira/browse/AMBARI-18631
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the database consistency check runs as a separate process prior 
> Ambari process. The database consistency checker load various modules needed 
> for performing the validations. (e.g. load stack definitions to be able to 
> compare service configs from stack with configs from db).
> Once database consistency checker completed Ambari server is started. Ambari 
> server beside others loads the same modules as database consistency checker.
> This double initialisation adds time to the ambari server startup time which 
> could be reduced if the database consistency check is moved into the ambari 
> server process.
> Moreover the database consistency check may check at the beginning if the 
> database is empty and perform the checks only if there is data in the 
> database.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  2d91eca 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyChecker.java
>  535d74f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  4030ef9 
>   ambari-server/src/main/python/ambari_server_main.py 57ec58d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
>  4663310 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariServerTest.java
>  54f6147 
> 
> Diff: https://reviews.apache.org/r/52999/diff/
> 
> 
> Testing
> ---
> 
> UT will be added and checked after patch draft review
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 53060: Authorizations given to roles, should use generic role-based principals rather than hard-coded pseudo-role-based principals


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53060/
---

(Updated Oct. 20, 2016, 4:06 p.m.)


Review request for Ambari, Aleksandr Kovalenko, DIPAYAN BHOWMICK, Jonathan 
Hurley, Nate Cole, and Sebastian Toader.


Bugs: AMBARI-18635
https://issues.apache.org/jira/browse/AMBARI-18635


Repository: ambari


Description
---

Authorizations given to roles, should use generic role-based principals rather 
than hard-coded resource types.  

Access to views can be assigned to all users with a given role.  The 
implementation for this lead to the creation of hard-coded principals that 
represent the current set of roles. This is not dynamic enough for possibly 
future enhancements where new roles may be created by administrators. 

This needs to be changed such that rather that using the hard-coded 
pseudo-role-principals, the dynamically generated role-principals are to be 
used.

The hard-coded pseudo-role-principals have the following `adminprincipaltype` 
values as opposed to "ROLE":

* ALL.CLUSTER.ADMINISTRATOR
* ALL.CLUSTER.OPERATOR
* ALL.SERVICE.ADMINISTRATOR
* ALL.SERVICE.OPERATOR
* ALL.CLUSTER.USER

These should be removed along with the associated `adminprincipal` records. 

Also, the FE should be updated to set permissions using the dynamic 
role-principals.

Finally, code should be cleaned up to remove unneeded code in 

- 
org.apache.ambari.server.security.authorization.ClusterInheritedPermissionHelper
- 
org.apache.ambari.server.controller.internal.GroupPrivilegeResourceProvider#getResources
- 
org.apache.ambari.server.controller.internal.PrivilegeResourceProvider#toEntity
- 
org.apache.ambari.server.controller.internal.UserPrivilegeResourceProvider#getResources
- 
org.apache.ambari.server.security.authorization.AuthorizationHelper#isAuthorized
- org.apache.ambari.server.view.ViewRegistry#addClusterInheritedPermissions
- ...


Diffs (updated)
-

  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/ViewsEditCtrl.js
 bd74b16 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
af22d7f 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionLoader.js
 988986b 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionsSaver.js
 c7b9295 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/View.js 
5bc0509 
  ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/edit.html 
69eb1c1 
  
ambari-admin/src/main/resources/ui/admin-web/test/unit/services/PermissionSaver_test.js
 fa36d98 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ClusterPrivilegeChangeRequestAuditEvent.java
 b28bb2a 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ViewPrivilegeChangeRequestAuditEvent.java
 11c558c 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/PrivilegeEventCreator.java
 5c476c6 
  
ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ViewPrivilegeEventCreator.java
 56d35c0 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 56e2398 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariPrivilegeResourceProvider.java
 e5c95cb 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterPrivilegeResourceProvider.java
 8f37764 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupPrivilegeResourceProvider.java
 94d1cad 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PrivilegeResourceProvider.java
 34111df 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
 bdd73a6 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ViewPrivilegeResourceProvider.java
 e5bd224 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PermissionDAO.java 
88d9775 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PrincipalDAO.java 
efbdfab 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PrincipalTypeDAO.java
 7823d56 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/PermissionEntity.java
 f091bab 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/PrincipalTypeEntity.java
 716d4f7 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AuthorizationHelper.java
 8639a2f 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/ClusterInheritedPermissionHelper.java
 9922bb2 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
 a4f0031 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog242.java
 a5276c2

Re: Review Request 53060: Authorizations given to roles, should use generic role-based principals rather than hard-coded pseudo-role-based principals



> On Oct. 20, 2016, 12:44 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PrivilegeResourceProvider.java,
> >  line 375
> > 
> >
> > I like StringUtils() here - I find it more readable.

fixed.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53060/#review153412
---


On Oct. 20, 2016, 4:06 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53060/
> ---
> 
> (Updated Oct. 20, 2016, 4:06 p.m.)
> 
> 
> Review request for Ambari, Aleksandr Kovalenko, DIPAYAN BHOWMICK, Jonathan 
> Hurley, Nate Cole, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-18635
> https://issues.apache.org/jira/browse/AMBARI-18635
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Authorizations given to roles, should use generic role-based principals 
> rather than hard-coded resource types.  
> 
> Access to views can be assigned to all users with a given role.  The 
> implementation for this lead to the creation of hard-coded principals that 
> represent the current set of roles. This is not dynamic enough for possibly 
> future enhancements where new roles may be created by administrators. 
> 
> This needs to be changed such that rather that using the hard-coded 
> pseudo-role-principals, the dynamically generated role-principals are to be 
> used.
> 
> The hard-coded pseudo-role-principals have the following `adminprincipaltype` 
> values as opposed to "ROLE":
> 
> * ALL.CLUSTER.ADMINISTRATOR
> * ALL.CLUSTER.OPERATOR
> * ALL.SERVICE.ADMINISTRATOR
> * ALL.SERVICE.OPERATOR
> * ALL.CLUSTER.USER
> 
> These should be removed along with the associated `adminprincipal` records. 
> 
> Also, the FE should be updated to set permissions using the dynamic 
> role-principals.
> 
> Finally, code should be cleaned up to remove unneeded code in 
> 
> - 
> org.apache.ambari.server.security.authorization.ClusterInheritedPermissionHelper
> - 
> org.apache.ambari.server.controller.internal.GroupPrivilegeResourceProvider#getResources
> - 
> org.apache.ambari.server.controller.internal.PrivilegeResourceProvider#toEntity
> - 
> org.apache.ambari.server.controller.internal.UserPrivilegeResourceProvider#getResources
> - 
> org.apache.ambari.server.security.authorization.AuthorizationHelper#isAuthorized
> - org.apache.ambari.server.view.ViewRegistry#addClusterInheritedPermissions
> - ...
> 
> 
> Diffs
> -
> 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/ambariViews/ViewsEditCtrl.js
>  bd74b16 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
> af22d7f 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionLoader.js
>  988986b 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/PermissionsSaver.js
>  c7b9295 
>   ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/View.js 
> 5bc0509 
>   
> ambari-admin/src/main/resources/ui/admin-web/app/views/ambariViews/edit.html 
> 69eb1c1 
>   
> ambari-admin/src/main/resources/ui/admin-web/test/unit/services/PermissionSaver_test.js
>  fa36d98 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ClusterPrivilegeChangeRequestAuditEvent.java
>  b28bb2a 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/event/request/ViewPrivilegeChangeRequestAuditEvent.java
>  11c558c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/PrivilegeEventCreator.java
>  5c476c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/audit/request/eventcreator/ViewPrivilegeEventCreator.java
>  56d35c0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  56e2398 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AmbariPrivilegeResourceProvider.java
>  e5c95cb 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ClusterPrivilegeResourceProvider.java
>  8f37764 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupPrivilegeResourceProvider.java
>  94d1cad 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/PrivilegeResourceProvider.java
>  34111df 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
>  bdd73a6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/ViewPrivilegeResourceProvider.java
>  e5bd224 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/PermissionDAO.java
>

Review Request 53075: Fix JSHint errors in Workflow Manager view

2016-10-20 Thread Sangeeta Ravindran


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53075/
---

Review request for Ambari, Di Li and Yusaku Sako.


Bugs: AMBARI-18615
https://issues.apache.org/jira/browse/AMBARI-18615


Repository: ambari


Description
---

When you compile the Workflow Manager view, you see 94 JSHint errors. 

Fix includes:

1) Adding a .jshintrc file for applying /*jshint esversion: 6 */ to the whole 
project to address errors similar to 'import' is only available in ES6 (use 
'esversion: 6').
2) Adding semi-colons where they are missing.
3) Using dot notation instead of array

The patch also fixes the error due to incorrect version of 
org.apache.ambari.contrib.views.

The test failure for this patch is caused by an existing issue unrelated to 
this patch.

Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.4:war 
(default-war) on project oozie-ui: The specified web.xml file 
'/home/jenkins/jenkins-slave/workspace/Ambari-trunk-test-patch/ambari/contrib/views/wfmanager/src/main/resources/ui/oozie-ambari-view/src/main/resources/WEB-INF/web.xml'
 does not exist -> [Help 1]


Diffs
-

  contrib/views/wfmanager/.jshintrc PRE-CREATION 
  contrib/views/wfmanager/pom.xml 1e910e2 
  contrib/views/wfmanager/src/main/resources/ui/README.md dcac346 
  contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
bd2944b 
  
contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-parameters.js
 1f75e64 
  contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-sla.js 
dac325f 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/actionjob_handler.js 
PRE-CREATION 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/actionjob_hanlder.js 
33204ea 
  
contrib/views/wfmanager/src/main/resources/ui/app/domain/default-layout-manager.js
 e208f83 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/layout-manager1.js 
0cd306a 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/layout-manager2.js 
d82b89e 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/mapping-utils.js 
7cb82e1 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/node-handler.js 
49347d8 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/schema-versions.js 
9562ae8 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/sla-info.js 76dffbd 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow-importer.js 
f29adb6 
  
contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow-xml-generator.js
 9fc791c 
  contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow.js 5908de5 
  
contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow_xml_mapper.js 
d5dc4da 
  contrib/views/wfmanager/src/main/resources/ui/app/routes/job.js d849609 
  
contrib/views/wfmanager/src/main/resources/ui/app/services/property-extractor.js
 17ff9aa 

Diff: https://reviews.apache.org/r/53075/diff/


Testing
---

Manual testing


File Attachments


JSHint errors
  
https://reviews.apache.org/media/uploaded/files/2016/10/20/c6a88e6d-d789-4f80-a7ca-9ec7b319331e__buildOutputJSHint_error.txt


Thanks,

Sangeeta Ravindran

Re: Review Request 53075: Fix JSHint errors in Workflow Manager view

2016-10-20 Thread Di Li


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53075/#review153457
---




contrib/views/wfmanager/src/main/resources/ui/app/domain/actionjob_handler.js 
(line 1)


Why a new file ? The changes are just the different way to referencing 
values in the dictionary, yes ?



contrib/views/wfmanager/src/main/resources/ui/app/domain/default-layout-manager.js
 (line 30)


not really a functional issue though, more of a coding practice.



contrib/views/wfmanager/src/main/resources/ui/app/routes/job.js (line 70)


Should this one be checking for falsy instead of an exact null ?



contrib/views/wfmanager/src/main/resources/ui/app/services/property-extractor.js
 (line 30)


Should this one be checking for falsy instead of an exact null ?


- Di Li


On Oct. 20, 2016, 8:41 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53075/
> ---
> 
> (Updated Oct. 20, 2016, 8:41 p.m.)
> 
> 
> Review request for Ambari, Di Li and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18615
> https://issues.apache.org/jira/browse/AMBARI-18615
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When you compile the Workflow Manager view, you see 94 JSHint errors. 
> 
> Fix includes:
> 
> 1) Adding a .jshintrc file for applying /*jshint esversion: 6 */ to the whole 
> project to address errors similar to 'import' is only available in ES6 (use 
> 'esversion: 6').
> 2) Adding semi-colons where they are missing.
> 3) Using dot notation instead of array
> 
> The patch also fixes the error due to incorrect version of 
> org.apache.ambari.contrib.views.
> 
> The test failure for this patch is caused by an existing issue unrelated to 
> this patch.
> 
> Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.4:war 
> (default-war) on project oozie-ui: The specified web.xml file 
> '/home/jenkins/jenkins-slave/workspace/Ambari-trunk-test-patch/ambari/contrib/views/wfmanager/src/main/resources/ui/oozie-ambari-view/src/main/resources/WEB-INF/web.xml'
>  does not exist -> [Help 1]
> 
> 
> Diffs
> -
> 
>   contrib/views/wfmanager/.jshintrc PRE-CREATION 
>   contrib/views/wfmanager/pom.xml 1e910e2 
>   contrib/views/wfmanager/src/main/resources/ui/README.md dcac346 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> bd2944b 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-parameters.js
>  1f75e64 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-sla.js 
> dac325f 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/actionjob_handler.js 
> PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/actionjob_hanlder.js 
> 33204ea 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/default-layout-manager.js
>  e208f83 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/layout-manager1.js 
> 0cd306a 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/layout-manager2.js 
> d82b89e 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/mapping-utils.js 
> 7cb82e1 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/node-handler.js 
> 49347d8 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/schema-versions.js 
> 9562ae8 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/sla-info.js 
> 76dffbd 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow-importer.js 
> f29adb6 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow-xml-generator.js
>  9fc791c 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow.js 
> 5908de5 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow_xml_mapper.js
>  d5dc4da 
>   contrib/views/wfmanager/src/main/resources/ui/app/routes/job.js d849609 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/services/property-extractor.js
>  17ff9aa 
> 
> Diff: https://reviews.apache.org/r/53075/diff/
> 
> 
> Testing
> ---
> 
> Manual testing
> 
> 
> File Attachments
> 
> 
> JSHint errors
>   
> https://reviews.apache.org/media/uploaded/files/2016/10/20/c6a88e6d-d789-4f80-a7ca-9ec7b319331e__buildOutputJSHint_error.txt
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 52420: Ambari Status commands should enforce a timeout < heartbeat interval


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52420/#review153451
---


Fix it, then Ship it!




Pending unit tests for the patch, +1 for the impl.


ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py (line 57)


Reference to the executor should not be lost.


- Sid Wagle


On Oct. 20, 2016, 3:54 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52420/
> ---
> 
> (Updated Oct. 20, 2016, 3:54 p.m.)
> 
> 
> Review request for Ambari, Sumit Mohanty, Sid Wagle, and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-18505
> https://issues.apache.org/jira/browse/AMBARI-18505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> SmartSense status command does an http call without timeout in 1.2 version of
> SS.
> 
> Ambari agent queues STATUS commands until we end up with HB lost state with
> ton of commands queued up.
> 
> This should just never happen is commands are getting queued by by the agent
> every HB.
> 
> 
> Diffs
> -
> 
>   ambari-agent/conf/unix/ambari-agent.ini 914e09a 
>   ambari-agent/src/main/python/ambari_agent/Controller.py 2a4d384 
>   ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 
> PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52420/diff/
> 
> 
> Testing
> ---
> 
> unit tests are pending to be added. Patch is not final
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 52636: Implement instrumented Lock for profiling/logging

2016-10-20 Thread Attila Doroszlai



> On Oct. 7, 2016, 8:53 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/logging/LockFactory.java,
> >  line 50
> > 
> >
> > There should be a default prefix
> 
> Attila Doroszlai wrote:
> What should be the default prefix?
> 
> Jonathan Hurley wrote:
> Could be the simple name of the invoking class?

Set to filename + line number (stack trace does not have simple class name).


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52636/#review151839
---


On Oct. 20, 2016, 10:08 p.m., Attila Doroszlai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52636/
> ---
> 
> (Updated Oct. 20, 2016, 10:08 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Sandor Magyari, 
> and Sebastian Toader.
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> 1. Create `Lock` and `ReadWriteLock` implementation for profiling and logging 
> lock usage
> 2. Introduce `LockFactory` that creates regular or instrumented locks based 
> on Ambari Server configuration
> 3. As sample usage, replace locks in `ClusterImpl` with the ones created by 
> `LockFactory`
> 4. Allows on-demand dump of lock profiler summary stats as part of the debug 
> log for `/api/v1/clusters` requests (This is a temporary solution.  Once the 
> factory is used in more places, the dump should be triggered by some other 
> request unrelated to the clusters.)
> 
> Configuration:
> 
>  * `server.locks.profiling=true` enables usage of instrumented locks instead 
> of regular ones (goes in `ambari.properties`)
>  * 
> `log4j.logger.org.apache.ambari.server.controller.AmbariManagementControllerImpl=DEBUG`
>  required for the on-demand dump (goes in `log4j.properties`)
>  * `log4j.logger.org.apache.ambari.server.logging=DEBUG` enables logging of 
> individual "requested", "acquired", "released" events. Only the outermost 
> lock/unlock call is logged, reentrant calls are ignored (goes in 
> `log4j.properties`)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  1759dc4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/LockFactory.java 
> PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/LockProfileDelegate.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLock.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
>  84697b8 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
>  a56ace5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/logging/LockFactoryTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLockTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/52636/diff/
> 
> 
> Testing
> ---
> 
> * Manual testing: cluster creation via blueprint both with and without 
> `server.locks.profiling=true` setting
>  * Unit tests:
>Tests run: 56, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.968 sec 
> - in org.apache.ambari.server.configuration.ConfigurationTest
>Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 sec 
> - in org.apache.ambari.server.logging.DelegatingLockTest
>Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.279 sec 
> - in org.apache.ambari.server.logging.LockFactoryTest
>Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec 
> - in org.apache.ambari.server.logging.ProfiledReentrantReadWriteLockTest
>  * Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 
> approved: 5012 licence.
> 
> 
> Thanks,
> 
> Attila Doroszlai
> 
>

Re: Review Request 52636: Implement instrumented Lock for profiling/logging

2016-10-20 Thread Attila Doroszlai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52636/
---

(Updated Oct. 20, 2016, 10:20 p.m.)


Review request for Ambari, Jonathan Hurley, Laszlo Puskas, Sandor Magyari, and 
Sebastian Toader.


Bugs: AMBARI-18654
https://issues.apache.org/jira/browse/AMBARI-18654


Repository: ambari


Description
---

1. Create `Lock` and `ReadWriteLock` implementation for profiling and logging 
lock usage
2. Introduce `LockFactory` that creates regular or instrumented locks based on 
Ambari Server configuration
3. As sample usage, replace locks in `ClusterImpl` with the ones created by 
`LockFactory`
4. Allows on-demand dump of lock profiler summary stats as part of the debug 
log for `/api/v1/clusters` requests (This is a temporary solution.  Once the 
factory is used in more places, the dump should be triggered by some other 
request unrelated to the clusters.)

Configuration:

 * `server.locks.profiling=true` enables usage of instrumented locks instead of 
regular ones (goes in `ambari.properties`)
 * 
`log4j.logger.org.apache.ambari.server.controller.AmbariManagementControllerImpl=DEBUG`
 required for the on-demand dump (goes in `log4j.properties`)
 * `log4j.logger.org.apache.ambari.server.logging=DEBUG` enables logging of 
individual "requested", "acquired", "released" events. Only the outermost 
lock/unlock call is logged, reentrant calls are ignored (goes in 
`log4j.properties`)


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 1759dc4 
  ambari-server/src/main/java/org/apache/ambari/server/logging/LockFactory.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/logging/LockProfileDelegate.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledLock.java 
PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantLock.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLock.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/state/cluster/ClusterImpl.java
 84697b8 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
 a56ace5 
  
ambari-server/src/test/java/org/apache/ambari/server/logging/LockFactoryTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/logging/ProfiledReentrantReadWriteLockTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/52636/diff/


Testing
---

* Manual testing: cluster creation via blueprint both with and without 
`server.locks.profiling=true` setting
 * Unit tests:
   Tests run: 56, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.968 sec - 
in org.apache.ambari.server.configuration.ConfigurationTest
   Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.044 sec - 
in org.apache.ambari.server.logging.DelegatingLockTest
   Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.279 sec - 
in org.apache.ambari.server.logging.LockFactoryTest
   Tests run: 10, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.168 sec - 
in org.apache.ambari.server.logging.ProfiledReentrantReadWriteLockTest
 * Rat check: Summary of files. Unapproved: 0 unknown: 0 generated: 0 approved: 
5012 licence.


Thanks,

Attila Doroszlai

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/
---

(Updated Oct. 20, 2016, 8:41 p.m.)


Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, and 
Sid Wagle.


Changes
---

Addressed review comments.


Bugs: AMBARI-18593
https://issues.apache.org/jira/browse/AMBARI-18593


Repository: ambari


Description
---

HDFS exposes top user activity broken down by operations in jmx (nntop).
These metrics should be captured in AMS and exposed in Grafana's HDFS 
dashboards.

Downsampling should likely be a function like MIN, MAX, AVG, SUM of underlying 
timeseries specified from the client.


Diffs (updated)
-

  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
 ba7807b 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
 c056d79 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
 118c695 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
 177e444 
  
ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
 f7060e0 
  
ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 b6e5da9 

Diff: https://reviews.apache.org/r/52842/diff/


Testing
---

Manually tested.
Unit tests added.
mvn clean test on ambari-metrics pending.


Thanks,

Aravindan Vijayan

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153452
---


Ship it!




Ship It!

- Sid Wagle


On Oct. 20, 2016, 8:41 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 20, 2016, 8:41 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  b6e5da9 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52999: Incorporate database consistency check into main Ambari process


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52999/#review153454
---


Ship it!




Ship It!

- Sid Wagle


On Oct. 20, 2016, 8:42 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52999/
> ---
> 
> (Updated Oct. 20, 2016, 8:42 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18631
> https://issues.apache.org/jira/browse/AMBARI-18631
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently the database consistency check runs as a separate process prior 
> Ambari process. The database consistency checker load various modules needed 
> for performing the validations. (e.g. load stack definitions to be able to 
> compare service configs from stack with configs from db).
> Once database consistency checker completed Ambari server is started. Ambari 
> server beside others loads the same modules as database consistency checker.
> This double initialisation adds time to the ambari server startup time which 
> could be reduced if the database consistency check is moved into the ambari 
> server process.
> Moreover the database consistency check may check at the beginning if the 
> database is empty and perform the checks only if there is data in the 
> database.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelper.java
>  2d91eca 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/DatabaseConsistencyChecker.java
>  535d74f 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  4030ef9 
>   ambari-server/src/main/python/ambari_server_main.py 57ec58d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/DatabaseConsistencyCheckHelperTest.java
>  4663310 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariServerTest.java
>  54f6147 
> 
> Diff: https://reviews.apache.org/r/52999/diff/
> 
> 
> Testing
> ---
> 
> UT will be added and checked after patch draft review
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 53075: Fix JSHint errors in Workflow Manager view

2016-10-20 Thread Sangeeta Ravindran



> On Oct. 20, 2016, 9:11 p.m., Di Li wrote:
> > contrib/views/wfmanager/src/main/resources/ui/app/domain/actionjob_handler.js,
> >  line 1
> > 
> >
> > Why a new file ? The changes are just the different way to referencing 
> > values in the dictionary, yes ?

I created a new file to rename it actionjob_hanlder to actionjob_handler. I had 
updated 
contrib/views/wfmanager/src/main/resources/ui/app/domain/node-handler.js where 
the file is being imported.


> On Oct. 20, 2016, 9:11 p.m., Di Li wrote:
> > contrib/views/wfmanager/src/main/resources/ui/app/domain/default-layout-manager.js,
> >  line 30
> > 
> >
> > not really a functional issue though, more of a coding practice.

Yes I agree, however JSHint flags it as an error because same variable is 
reused.


> On Oct. 20, 2016, 9:11 p.m., Di Li wrote:
> > contrib/views/wfmanager/src/main/resources/ui/app/routes/job.js, line 70
> > 
> >
> > Should this one be checking for falsy instead of an exact null ?

params would contain an array of search parameters which is set based on the 
last search. If no search was previously done, params would be null.


> On Oct. 20, 2016, 9:11 p.m., Di Li wrote:
> > contrib/views/wfmanager/src/main/resources/ui/app/services/property-extractor.js,
> >  line 30
> > 
> >
> > Should this one be checking for falsy instead of an exact null ?

same as previous comment. matches is a array.


- Sangeeta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53075/#review153457
---


On Oct. 20, 2016, 8:41 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53075/
> ---
> 
> (Updated Oct. 20, 2016, 8:41 p.m.)
> 
> 
> Review request for Ambari, Di Li and Yusaku Sako.
> 
> 
> Bugs: AMBARI-18615
> https://issues.apache.org/jira/browse/AMBARI-18615
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When you compile the Workflow Manager view, you see 94 JSHint errors. 
> 
> Fix includes:
> 
> 1) Adding a .jshintrc file for applying /*jshint esversion: 6 */ to the whole 
> project to address errors similar to 'import' is only available in ES6 (use 
> 'esversion: 6').
> 2) Adding semi-colons where they are missing.
> 3) Using dot notation instead of array
> 
> The patch also fixes the error due to incorrect version of 
> org.apache.ambari.contrib.views.
> 
> The test failure for this patch is caused by an existing issue unrelated to 
> this patch.
> 
> Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.4:war 
> (default-war) on project oozie-ui: The specified web.xml file 
> '/home/jenkins/jenkins-slave/workspace/Ambari-trunk-test-patch/ambari/contrib/views/wfmanager/src/main/resources/ui/oozie-ambari-view/src/main/resources/WEB-INF/web.xml'
>  does not exist -> [Help 1]
> 
> 
> Diffs
> -
> 
>   contrib/views/wfmanager/.jshintrc PRE-CREATION 
>   contrib/views/wfmanager/pom.xml 1e910e2 
>   contrib/views/wfmanager/src/main/resources/ui/README.md dcac346 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/flow-designer.js 
> bd2944b 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-parameters.js
>  1f75e64 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/components/workflow-sla.js 
> dac325f 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/actionjob_handler.js 
> PRE-CREATION 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/actionjob_hanlder.js 
> 33204ea 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/default-layout-manager.js
>  e208f83 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/layout-manager1.js 
> 0cd306a 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/layout-manager2.js 
> d82b89e 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/mapping-utils.js 
> 7cb82e1 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/node-handler.js 
> 49347d8 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/schema-versions.js 
> 9562ae8 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/sla-info.js 
> 76dffbd 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow-importer.js 
> f29adb6 
>   
> contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow-xml-generator.js
>  9fc791c 
>   contrib/views/wfmanager/src/main/resources/ui/app/domain/workflow.js 
> 5908de5 
>   
>

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST



> On Oct. 20, 2016, 6:36 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java,
> >  lines 171-189
> > 
> >
> > There should be a configuration option to allow the user to choose 
> > whether groups should be automatically created or not.

What if the user says no. For external users, the user and groups have to be 
created together just like when we sync with LDAP. We can not have orphan users 
with no group to associate with. 
Also, the rationale of creating these groups is to have no latency syncing. For 
LDAP until the admin does LDAP-sync, changes to user-group memebership are not 
reflected in  ambari. This is a security hole and we need immediate syncing of 
user-group membership to reflect the backend.


> On Oct. 20, 2016, 6:36 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java,
> >  lines 165-167
> > 
> >
> > What happends on username collision.  Maybe there needs to be an option 
> > to determine whether to change the existing user to a PAM user or fail - 
> > depending on the configuration.
> > 
> > There is a similar effort going on now dealing with what to do with 
> > username collisions during LDAP sync.

I have fixed the code to avoid username & group collision which was missing in 
my previous patch. With this patch there should not be any collision.


- Vishal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/#review153389
---


On Oct. 20, 2016, 6:01 p.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52369/
> ---
> 
> (Updated Oct. 20, 2016, 6:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.
> 
> 
> Bugs: AMBARI-12263
> https://issues.apache.org/jira/browse/AMBARI-12263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hello Robert,
> 
> How are you doing? 
> 
> We have been working on PAM support into Ambari and have something ready for 
> review. Can you please take a look at the patch and documentation and provide 
> your feedback.
> 
> Please let me know if you have any questions.
> 
> Note: I have added you as a reviewer as i see some authentication related 
> commits under your name.
> 
> Thanks,
> -Vishal
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml d507b82 
>   ambari-server/sbin/ambari-server 762ae19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1fc9dbf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  5e498f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
>  ef28f61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
>  e1aa5ac 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
>  bdd73a6 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java 
> 255c5e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java 
> e4ed9c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
>  00e233e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
>  26d4da7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
>  b20df8d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
>  aa9f3e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
>  e547f05 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  185bd58 
>   ambari-server/src/main/python/ambari-server.py bb6bc0e 
>   ambari-server/src/main/python/ambari_server/setupActions.py 697bc1d 
>

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST



> On Oct. 7, 2016, 10:55 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java,
> >  line 817
> > 
> >
> > Since this is called each time Ambari starts up new groups can be 
> > _magically_ created each time Ambari is started.  This seems like a 
> > possible issue since it gives a non-Ambari-administrator the ability to 
> > create groups and assign roles to them. In many cases, the user that has 
> > write access to the ambari.properties file does not have admin access to 
> > Ambari. So being able to change something like this becomes a security 
> > hole. 
> > 
> > If we do find a way to do this securely, the solution should be more 
> > generic since it may not apply only to PAM.
> 
> Vishal Ghugare wrote:
> we could possibly do the PAM group creation securely & in a generic way 
> by invoking a rest api (a new api).
> 
> Robert Levas wrote:
> Can this feature be dropped from this patch?  We can then create a JIRA 
> and discuss a more generic and secure way to handle setting roles on imported 
> or manaully created groups. This will apply to the exists LDAP integration as 
> well as any other authentication source we may add in the future.

Currently, the only way to create a LDAP group in amabri is by LDAP-sync. 
Ambari do not have control over LDAP user-group membership. 

I will open a new JIRA for this work and take out the predefined group creation 
from this patch for now.


- Vishal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/#review151829
---


On Oct. 20, 2016, 6:01 p.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52369/
> ---
> 
> (Updated Oct. 20, 2016, 6:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.
> 
> 
> Bugs: AMBARI-12263
> https://issues.apache.org/jira/browse/AMBARI-12263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hello Robert,
> 
> How are you doing? 
> 
> We have been working on PAM support into Ambari and have something ready for 
> review. Can you please take a look at the patch and documentation and provide 
> your feedback.
> 
> Please let me know if you have any questions.
> 
> Note: I have added you as a reviewer as i see some authentication related 
> commits under your name.
> 
> Thanks,
> -Vishal
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml d507b82 
>   ambari-server/sbin/ambari-server 762ae19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1fc9dbf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  5e498f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
>  ef28f61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
>  e1aa5ac 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
>  bdd73a6 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java 
> 255c5e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java 
> e4ed9c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
>  00e233e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
>  26d4da7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
>  b20df8d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
>  aa9f3e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
>  e547f05 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  185bd58 
>   ambari-server/src/main/python/ambari-server.py bb6bc0e 
>   ambari-server/src/main/python/ambari_server/setupActions.py 697bc1d 
>   ambari-server/src/main/python/ambari_server/setupSecurity.py 119a7d8 
>

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST



> On Oct. 7, 2016, 10:55 a.m., Robert Levas wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java,
> >  line 817
> > 
> >
> > Since this is called each time Ambari starts up new groups can be 
> > _magically_ created each time Ambari is started.  This seems like a 
> > possible issue since it gives a non-Ambari-administrator the ability to 
> > create groups and assign roles to them. In many cases, the user that has 
> > write access to the ambari.properties file does not have admin access to 
> > Ambari. So being able to change something like this becomes a security 
> > hole. 
> > 
> > If we do find a way to do this securely, the solution should be more 
> > generic since it may not apply only to PAM.
> 
> Vishal Ghugare wrote:
> we could possibly do the PAM group creation securely & in a generic way 
> by invoking a rest api (a new api).
> 
> Robert Levas wrote:
> Can this feature be dropped from this patch?  We can then create a JIRA 
> and discuss a more generic and secure way to handle setting roles on imported 
> or manaully created groups. This will apply to the exists LDAP integration as 
> well as any other authentication source we may add in the future.
> 
> Vishal Ghugare wrote:
> Currently, the only way to create a LDAP group in amabri is by LDAP-sync. 
> Ambari do not have control over LDAP user-group membership. 
> 
> I will open a new JIRA for this work and take out the predefined group 
> creation from this patch for now.

Created a new JIRA for this work: AMBARI-18656.


- Vishal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/#review151829
---


On Oct. 20, 2016, 6:01 p.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52369/
> ---
> 
> (Updated Oct. 20, 2016, 6:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.
> 
> 
> Bugs: AMBARI-12263
> https://issues.apache.org/jira/browse/AMBARI-12263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hello Robert,
> 
> How are you doing? 
> 
> We have been working on PAM support into Ambari and have something ready for 
> review. Can you please take a look at the patch and documentation and provide 
> your feedback.
> 
> Please let me know if you have any questions.
> 
> Note: I have added you as a reviewer as i see some authentication related 
> commits under your name.
> 
> Thanks,
> -Vishal
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml d507b82 
>   ambari-server/sbin/ambari-server 762ae19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1fc9dbf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  5e498f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
>  ef28f61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
>  e1aa5ac 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
>  bdd73a6 
>   ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java 
> 255c5e6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java 
> e4ed9c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
>  00e233e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
>  26d4da7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
>  b20df8d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
>  aa9f3e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
>  e547f05 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  185bd58 
>   ambari-server/src/main/python/ambari-server.py bb6bc0e 
>   ambari-server/src/main/python/ambari_server/setupActions.py 697bc1d 
>

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/
---

(Updated Oct. 20, 2016, 6:01 p.m.)


Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.


Bugs: AMBARI-12263
https://issues.apache.org/jira/browse/AMBARI-12263


Repository: ambari


Description
---

Hello Robert,

How are you doing? 

We have been working on PAM support into Ambari and have something ready for 
review. Can you please take a look at the patch and documentation and provide 
your feedback.

Please let me know if you have any questions.

Note: I have added you as a reviewer as i see some authentication related 
commits under your name.

Thanks,
-Vishal


Diffs (updated)
-

  ambari-server/pom.xml d507b82 
  ambari-server/sbin/ambari-server 762ae19 
  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 2e850ef 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 1fc9dbf 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 5e498f0 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
 ef28f61 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
 e1aa5ac 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
 bdd73a6 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/GroupDAO.java 
255c5e6 
  ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java 
e4ed9c6 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
 00e233e 
  
ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
 26d4da7 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
 b20df8d 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
 PRE-CREATION 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
 aa9f3e0 
  
ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
 e547f05 
  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 185bd58 
  ambari-server/src/main/python/ambari-server.py bb6bc0e 
  ambari-server/src/main/python/ambari_server/setupActions.py 697bc1d 
  ambari-server/src/main/python/ambari_server/setupSecurity.py 119a7d8 
  ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 1d55515 
  ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
  ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
  ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
  ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
  ambari-server/src/main/resources/properties.json eb27878 
  ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 500c0bf 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
 PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java
 a80cd03 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 7b6c3ad 

Diff: https://reviews.apache.org/r/52369/diff/


Testing
---

No test cases added at this point.


File Attachments


AMBARI-12263.patch_base
  
https://reviews.apache.org/media/uploaded/files/2016/10/17/5107a016-3a83-478c-b98c-2f35ecf6cbc5__AMBARI-12263.patch_base


Thanks,

Vishal Ghugare

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN



> On Oct. 20, 2016, 2:01 a.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java,
> >  line 96
> > 
> >
> > The only problem I see here is we loose context of what topN funtion 
> > was applied if we copy same value everywhere. What about only for the 
> > column = function and rest as NULL?

I am ok with setting "NULL" values for columns that are not computed through 
the downsampling. But, I feel that the downsampled metric value should be set 
in the AVG column, which is returned by default when the metric is requested. 
Even though we do a Top N by Max value for the downsampling function, the 
computed value will be in the default value column, and not the MAX column. 

The user's query will be like "Get me the Top 3 op-user combination for every 5 
min slice the last 6 hours". The query need not explicitly request "MAX" of the 
metric, since that is basically what the topn downsampling configs dictated.


- Aravindan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153330
---


On Oct. 18, 2016, 6:35 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 18, 2016, 6:35 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  b6e5da9 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52369: AMBARI-12263: Support PAM as authentication mechanism for accessing Ambari UI/REST


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52369/#review153389
---




ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
 (lines 165 - 167)


What happends on username collision.  Maybe there needs to be an option to 
determine whether to change the existing user to a PAM user or fail - depending 
on the configuration.

There is a similar effort going on now dealing with what to do with 
username collisions during LDAP sync.



ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
 (lines 171 - 189)


There should be a configuration option to allow the user to choose whether 
groups should be automatically created or not.


- Robert Levas


On Oct. 17, 2016, 4:50 p.m., Vishal Ghugare wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52369/
> ---
> 
> (Updated Oct. 17, 2016, 4:50 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Di Li, and Robert Levas.
> 
> 
> Bugs: AMBARI-12263
> https://issues.apache.org/jira/browse/AMBARI-12263
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Hello Robert,
> 
> How are you doing? 
> 
> We have been working on PAM support into Ambari and have something ready for 
> review. Can you please take a look at the patch and documentation and provide 
> your feedback.
> 
> Please let me know if you have any questions.
> 
> Note: I have added you as a reviewer as i see some authentication related 
> commits under your name.
> 
> Thanks,
> -Vishal
> 
> 
> Diffs
> -
> 
>   ambari-server/pom.xml d507b82 
>   ambari-server/sbin/ambari-server 762ae19 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  2e850ef 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  1fc9dbf 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  5e498f0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/GroupResponse.java
>  ef28f61 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/GroupResourceProvider.java
>  e1aa5ac 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/UserPrivilegeResourceProvider.java
>  bdd73a6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/ResourceDAO.java 
> e4ed9c6 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/entities/GroupEntity.java
>  00e233e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/ClientSecurityType.java
>  26d4da7 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProvider.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Group.java
>  b20df8d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/GroupType.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/PamAuthenticationException.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/UserType.java
>  aa9f3e0 
>   
> ambari-server/src/main/java/org/apache/ambari/server/security/authorization/Users.java
>  e547f05 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  185bd58 
>   ambari-server/src/main/python/ambari-server.py bb6bc0e 
>   ambari-server/src/main/python/ambari_server/setupActions.py 697bc1d 
>   ambari-server/src/main/python/ambari_server/setupSecurity.py 119a7d8 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 1d55515 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 49f3e2f 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 7aa52ef 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 0c95471 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 631b5c4 
>   ambari-server/src/main/resources/properties.json eb27878 
>   ambari-server/src/main/resources/webapp/WEB-INF/spring-security.xml 500c0bf 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/AmbariPamAuthenticationProviderTest.java
>  PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/security/authorization/TestUsers.java
>  a80cd03 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7b6c3ad 
> 
> Diff:

Re: Review Request 52456: Modify HTTP headers to follow best security practices