date:20160421

Please use the following document to collaborate:

https://docs.google.com/a/hortonworks.com/document/d/1RcAth8M1oEDOjIZ0qMaCAtatCyz6qdLWuEcZLC4Ejbs/edit?disco=Aq7NM04=comment_email_document

Thank you.




On Thu, Apr 21, 2016 at 7:41 PM -0700, "Ajit Kumar" 
> wrote:

This is an automatically generated e-mail. To reply, visit: 
https://reviews.apache.org/r/46545/


On April 22nd, 2016, 12:01 a.m. UTC, Ajit Kumar wrote:

Can you explain what from_version and to_version represent here?

On April 22nd, 2016, 12:45 a.m. UTC, Nahappan Somasundaram wrote:

from_version is the current version; to_version is the version ambari is moving 
to.
In case of a clean install, from_version is 0.0.0.0

Lets discuss about DB modeling over skype or in person. There are different 
approaches like having two columns from_version_id and current_version instead 
of from_version and to_version.


- Ajit


On April 21st, 2016, 11:15 p.m. UTC, Nahappan Somasundaram wrote:

Review request for Ambari, Alejandro Fernandez, Ajit Kumar, Jonathan Hurley, 
Jayush Luniya, and Sumit Mohanty.
By Nahappan Somasundaram.

Updated April 21, 2016, 11:15 p.m.

Bugs: AMBARI-16029
Repository: ambari
Description

AMBARI-16029: Ambari version history - Create DB table, constraints and 
sequence id.




 Issue
In order to support tracking of ambari version, a DB table is required.




 Fix
Create ambari_version_history table to store from version, to version, install 
type, start and stop timestamps.


Testing

 1. mvn clean install




[INFO] 
[INFO] Reactor Summary:
[INFO]
[INFO] Ambari Main ... SUCCESS [5.113s]
[INFO] Apache Ambari Project POM . SUCCESS [0.038s]
[INFO] Ambari Web  SUCCESS [26.897s]
[INFO] Ambari Views .. SUCCESS [1.400s]
[INFO] Ambari Admin View . SUCCESS [9.242s]
[INFO] ambari-metrics  SUCCESS [0.346s]
[INFO] Ambari Metrics Common . SUCCESS [0.528s]
[INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.237s]
[INFO] Ambari Metrics Flume Sink . SUCCESS [0.566s]
[INFO] Ambari Metrics Kafka Sink . SUCCESS [0.600s]
[INFO] Ambari Metrics Storm Sink . SUCCESS [1.531s]
[INFO] Ambari Metrics Collector .. SUCCESS [7.183s]
[INFO] Ambari Metrics Monitor  SUCCESS [1.853s]
[INFO] Ambari Metrics Grafana  SUCCESS [1.031s]
[INFO] Ambari Metrics Assembly ... SUCCESS [1:18.252s]
[INFO] Ambari Server . SUCCESS [2:58.522s]
[INFO] Ambari Functional Tests ... SUCCESS [2.194s]
[INFO] Ambari Agent .. SUCCESS [25.339s]
[INFO] Ambari Client . SUCCESS [0.047s]
[INFO] Ambari Python Client .. SUCCESS [0.947s]
[INFO] Ambari Groovy Client .. SUCCESS [2.196s]
[INFO] Ambari Shell .. SUCCESS [0.038s]
[INFO] Ambari Python Shell ... SUCCESS [0.693s]
[INFO] Ambari Groovy Shell ... SUCCESS [0.907s]
[INFO] ambari-logsearch .. SUCCESS [0.253s]
[INFO] Ambari Logsearch Portal ... SUCCESS [7.069s]
[INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.570s]
[INFO] Ambari Logsearch Appender . SUCCESS [0.190s]
[INFO] Ambari Logsearch Assembly . SUCCESS [0.079s]
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 5:56.545s
[INFO] Finished at: Thu Apr 21 16:14:34 PDT 2016
[INFO] Final Memory: 274M/1094M
[INFO] 




 2. Manual testing
Tested with Ambari-DDL-Postgres-EMBEDDED-CREATE.sql. Copied the SQL script to a 
VM and ran ambari-server reset. Verified the schema:




ambari=> \d+ ambari_version_history
 Table "ambari.ambari_version_history"
 Column  |  Type  | Modifiers | Storage  | Description
-++---+--+-
 id  | bigint | not null  | plain|
 from_version| character varying(255) | not null  | extended |
 to_version  | character varying(255) | not null  | extended |
 start_timestamp | bigint | not null

Re: Review Request 46545: AMBARI-16029: Ambari version history - Create DB table, constraints and sequence id.



> On April 21, 2016, 5:01 p.m., Ajit Kumar wrote:
> > Can you explain what from_version and to_version represent here?
> 
> Nahappan Somasundaram wrote:
> from_version is the current version; to_version is the version ambari is 
> moving to.
> In case of a clean install, from_version is 0.0.0.0
> 
> Ajit Kumar wrote:
> Lets discuss about DB modeling over skype or in person. There are 
> different approaches like having two columns from_version_id and 
> current_version instead of from_version and to_version.

We aleady have two columns here. I suppose your suggestion is to rename the 
columns to be more informative, right?


- Nahappan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46545/#review129988
---


On April 21, 2016, 4:15 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46545/
> ---
> 
> (Updated April 21, 2016, 4:15 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Ajit Kumar, Jonathan Hurley, 
> Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16029
> https://issues.apache.org/jira/browse/AMBARI-16029
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16029: Ambari version history - Create DB table, constraints and 
> sequence id.
> 
> ** Issue **
> In order to support tracking of ambari version, a DB table is required.
> 
> ** Fix **
> Create ambari_version_history table to store from version, to version, 
> install type, start and stop timestamps.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 
> 93576f7498179df1e0b83e65dcd3a1580ce5ba3c 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 
> b0264f2db234d7a69b8089bc3aac321c8d0ce989 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 
> 56a661675b7cb15763bfce5d043b0eba868d9521 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 
> f18cdec49f6bfe300eaec31af3ff33142c8c4012 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4584d5e03c0d3979005ad1474f77f5d58d2519f7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
> d2737d787babff36c52b375884f295cc886002a4 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 
> c85ae46ca221985b0a258c225634af5ace10d880 
> 
> Diff: https://reviews.apache.org/r/46545/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [5.113s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Web  SUCCESS [26.897s]
> [INFO] Ambari Views .. SUCCESS [1.400s]
> [INFO] Ambari Admin View . SUCCESS [9.242s]
> [INFO] ambari-metrics  SUCCESS [0.346s]
> [INFO] Ambari Metrics Common . SUCCESS [0.528s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.237s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [0.566s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.600s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.531s]
> [INFO] Ambari Metrics Collector .. SUCCESS [7.183s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.853s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.031s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:18.252s]
> [INFO] Ambari Server . SUCCESS [2:58.522s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.194s]
> [INFO] Ambari Agent .. SUCCESS [25.339s]
> [INFO] Ambari Client . SUCCESS [0.047s]
> [INFO] Ambari Python Client .. SUCCESS [0.947s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.196s]
> [INFO] Ambari Shell .. SUCCESS [0.038s]
> [INFO] Ambari Python Shell ... SUCCESS [0.693s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.907s]
> [INFO] ambari-logsearch .. SUCCESS [0.253s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [7.069s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.570s]
> [INFO] Ambari Logsearch

Re: Review Request 46545: AMBARI-16029: Ambari version history - Create DB table, constraints and sequence id.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46545/#review130009
---




ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql (line 822)


I suggested some changes in Jira AMBARI-16029 so we can capture more 
information, specifically in cases where errors occurs.


- Alejandro Fernandez


On April 21, 2016, 11:15 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46545/
> ---
> 
> (Updated April 21, 2016, 11:15 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Ajit Kumar, Jonathan Hurley, 
> Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16029
> https://issues.apache.org/jira/browse/AMBARI-16029
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16029: Ambari version history - Create DB table, constraints and 
> sequence id.
> 
> ** Issue **
> In order to support tracking of ambari version, a DB table is required.
> 
> ** Fix **
> Create ambari_version_history table to store from version, to version, 
> install type, start and stop timestamps.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 
> 93576f7498179df1e0b83e65dcd3a1580ce5ba3c 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 
> b0264f2db234d7a69b8089bc3aac321c8d0ce989 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 
> 56a661675b7cb15763bfce5d043b0eba868d9521 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 
> f18cdec49f6bfe300eaec31af3ff33142c8c4012 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4584d5e03c0d3979005ad1474f77f5d58d2519f7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
> d2737d787babff36c52b375884f295cc886002a4 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 
> c85ae46ca221985b0a258c225634af5ace10d880 
> 
> Diff: https://reviews.apache.org/r/46545/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [5.113s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Web  SUCCESS [26.897s]
> [INFO] Ambari Views .. SUCCESS [1.400s]
> [INFO] Ambari Admin View . SUCCESS [9.242s]
> [INFO] ambari-metrics  SUCCESS [0.346s]
> [INFO] Ambari Metrics Common . SUCCESS [0.528s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.237s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [0.566s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.600s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.531s]
> [INFO] Ambari Metrics Collector .. SUCCESS [7.183s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.853s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.031s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:18.252s]
> [INFO] Ambari Server . SUCCESS [2:58.522s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.194s]
> [INFO] Ambari Agent .. SUCCESS [25.339s]
> [INFO] Ambari Client . SUCCESS [0.047s]
> [INFO] Ambari Python Client .. SUCCESS [0.947s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.196s]
> [INFO] Ambari Shell .. SUCCESS [0.038s]
> [INFO] Ambari Python Shell ... SUCCESS [0.693s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.907s]
> [INFO] ambari-logsearch .. SUCCESS [0.253s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [7.069s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.570s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.190s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.079s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 5:56.545s
> [INFO] Finished at: Thu

Review Request 46547: [PREVIEW] Create "/hadoop/llap/local" on each host and disk in Kerberized cluster for LLAP


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46547/
---

Review request for Ambari, Siddharth Seth and Swapan Shridhar.


Bugs: AMBARI-16031
https://issues.apache.org/jira/browse/AMBARI-16031


Repository: ambari


Description
---

- In non-kerberized cluster, hive.llap.daemon.work.dir will point to : 
"${yarn.nodemanager.local-dirs}”
- In kerberized cluster, we need to create "/hadoop/llap/local" on each node 
and disk and have "hive.llap.daemon.work.dirs" point to it.
- It's similar to the way yarn.nodemanager.local-dirs is created.


Diffs
-

  ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/metainfo.xml 
10e6405 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/nodemanager.py
 dd6b130 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py
 327781a 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/package/scripts/yarn.py
 735d8cb 
  ambari-web/app/controllers/main/service/item.js b1ea523 
  ambari-web/app/messages.js 6c0049d 
  ambari-web/app/models/host_component.js b4a369e 
  ambari-web/app/views/main/service/item.js 51f9bdd 
  ambari-web/test/views/main/service/item_test.js 085eaad 

Diff: https://reviews.apache.org/r/46547/diff/


Testing
---

Verified on cluster with HDP 2.4, Hive Server Interactive + LLAP, and then 
kerberized.

Python unit tests passed,
--
Total run:976
Total errors:0
Total failures:0
OK

on ambari-web, mvn test passed,
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 39.952 s
[INFO] Finished at: 2016-04-21T15:55:34-07:00
[INFO] Final Memory: 36M/3926M
[INFO] 


Thanks,

Alejandro Fernandez

Re: Review Request 46545: AMBARI-16029: Ambari version history - Create DB table, constraints and sequence id.



> On April 21, 2016, 5:01 p.m., Ajit Kumar wrote:
> > Can you explain what from_version and to_version represent here?

from_version is the current version; to_version is the version ambari is moving 
to.
In case of a clean install, from_version is 0.0.0.0


- Nahappan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46545/#review129988
---


On April 21, 2016, 4:15 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46545/
> ---
> 
> (Updated April 21, 2016, 4:15 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Ajit Kumar, Jonathan Hurley, 
> Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16029
> https://issues.apache.org/jira/browse/AMBARI-16029
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16029: Ambari version history - Create DB table, constraints and 
> sequence id.
> 
> ** Issue **
> In order to support tracking of ambari version, a DB table is required.
> 
> ** Fix **
> Create ambari_version_history table to store from version, to version, 
> install type, start and stop timestamps.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 
> 93576f7498179df1e0b83e65dcd3a1580ce5ba3c 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 
> b0264f2db234d7a69b8089bc3aac321c8d0ce989 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 
> 56a661675b7cb15763bfce5d043b0eba868d9521 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 
> f18cdec49f6bfe300eaec31af3ff33142c8c4012 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4584d5e03c0d3979005ad1474f77f5d58d2519f7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
> d2737d787babff36c52b375884f295cc886002a4 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 
> c85ae46ca221985b0a258c225634af5ace10d880 
> 
> Diff: https://reviews.apache.org/r/46545/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [5.113s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Web  SUCCESS [26.897s]
> [INFO] Ambari Views .. SUCCESS [1.400s]
> [INFO] Ambari Admin View . SUCCESS [9.242s]
> [INFO] ambari-metrics  SUCCESS [0.346s]
> [INFO] Ambari Metrics Common . SUCCESS [0.528s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.237s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [0.566s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.600s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.531s]
> [INFO] Ambari Metrics Collector .. SUCCESS [7.183s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.853s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.031s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:18.252s]
> [INFO] Ambari Server . SUCCESS [2:58.522s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.194s]
> [INFO] Ambari Agent .. SUCCESS [25.339s]
> [INFO] Ambari Client . SUCCESS [0.047s]
> [INFO] Ambari Python Client .. SUCCESS [0.947s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.196s]
> [INFO] Ambari Shell .. SUCCESS [0.038s]
> [INFO] Ambari Python Shell ... SUCCESS [0.693s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.907s]
> [INFO] ambari-logsearch .. SUCCESS [0.253s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [7.069s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.570s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.190s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.079s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 5:56.545s
> [INFO] Finished at: Thu Apr 21

Re: Review Request 46545: AMBARI-16029: Ambari version history - Create DB table, constraints and sequence id.



> On April 21, 2016, 5:33 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql, line 822
> > 
> >
> > I want to understand the motivation behind this table first.

https://issues.apache.org/jira/browse/AMBARI-16030


- Nahappan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46545/#review129995
---


On April 21, 2016, 4:15 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46545/
> ---
> 
> (Updated April 21, 2016, 4:15 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Ajit Kumar, Jonathan Hurley, 
> Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16029
> https://issues.apache.org/jira/browse/AMBARI-16029
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16029: Ambari version history - Create DB table, constraints and 
> sequence id.
> 
> ** Issue **
> In order to support tracking of ambari version, a DB table is required.
> 
> ** Fix **
> Create ambari_version_history table to store from version, to version, 
> install type, start and stop timestamps.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 
> 93576f7498179df1e0b83e65dcd3a1580ce5ba3c 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 
> b0264f2db234d7a69b8089bc3aac321c8d0ce989 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 
> 56a661675b7cb15763bfce5d043b0eba868d9521 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 
> f18cdec49f6bfe300eaec31af3ff33142c8c4012 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4584d5e03c0d3979005ad1474f77f5d58d2519f7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
> d2737d787babff36c52b375884f295cc886002a4 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 
> c85ae46ca221985b0a258c225634af5ace10d880 
> 
> Diff: https://reviews.apache.org/r/46545/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [5.113s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Web  SUCCESS [26.897s]
> [INFO] Ambari Views .. SUCCESS [1.400s]
> [INFO] Ambari Admin View . SUCCESS [9.242s]
> [INFO] ambari-metrics  SUCCESS [0.346s]
> [INFO] Ambari Metrics Common . SUCCESS [0.528s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.237s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [0.566s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.600s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.531s]
> [INFO] Ambari Metrics Collector .. SUCCESS [7.183s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.853s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.031s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:18.252s]
> [INFO] Ambari Server . SUCCESS [2:58.522s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.194s]
> [INFO] Ambari Agent .. SUCCESS [25.339s]
> [INFO] Ambari Client . SUCCESS [0.047s]
> [INFO] Ambari Python Client .. SUCCESS [0.947s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.196s]
> [INFO] Ambari Shell .. SUCCESS [0.038s]
> [INFO] Ambari Python Shell ... SUCCESS [0.693s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.907s]
> [INFO] ambari-logsearch .. SUCCESS [0.253s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [7.069s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.570s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.190s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.079s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
>

Re: Review Request 46545: AMBARI-16029: Ambari version history - Create DB table, constraints and sequence id.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46545/#review129995
---




ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql (line 822)


I want to understand the motivation behind this table first.


- Alejandro Fernandez


On April 21, 2016, 11:15 p.m., Nahappan Somasundaram wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46545/
> ---
> 
> (Updated April 21, 2016, 11:15 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Ajit Kumar, Jonathan Hurley, 
> Jayush Luniya, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16029
> https://issues.apache.org/jira/browse/AMBARI-16029
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> AMBARI-16029: Ambari version history - Create DB table, constraints and 
> sequence id.
> 
> ** Issue **
> In order to support tracking of ambari version, a DB table is required.
> 
> ** Fix **
> Create ambari_version_history table to store from version, to version, 
> install type, start and stop timestamps.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/Ambari-DDL-Derby-CREATE.sql 
> 93576f7498179df1e0b83e65dcd3a1580ce5ba3c 
>   ambari-server/src/main/resources/Ambari-DDL-MySQL-CREATE.sql 
> b0264f2db234d7a69b8089bc3aac321c8d0ce989 
>   ambari-server/src/main/resources/Ambari-DDL-Oracle-CREATE.sql 
> 56a661675b7cb15763bfce5d043b0eba868d9521 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-CREATE.sql 
> f18cdec49f6bfe300eaec31af3ff33142c8c4012 
>   ambari-server/src/main/resources/Ambari-DDL-Postgres-EMBEDDED-CREATE.sql 
> 4584d5e03c0d3979005ad1474f77f5d58d2519f7 
>   ambari-server/src/main/resources/Ambari-DDL-SQLAnywhere-CREATE.sql 
> d2737d787babff36c52b375884f295cc886002a4 
>   ambari-server/src/main/resources/Ambari-DDL-SQLServer-CREATE.sql 
> c85ae46ca221985b0a258c225634af5ace10d880 
> 
> Diff: https://reviews.apache.org/r/46545/diff/
> 
> 
> Testing
> ---
> 
> ** 1. mvn clean install **
> 
> [INFO] 
> 
> [INFO] Reactor Summary:
> [INFO]
> [INFO] Ambari Main ... SUCCESS [5.113s]
> [INFO] Apache Ambari Project POM . SUCCESS [0.038s]
> [INFO] Ambari Web  SUCCESS [26.897s]
> [INFO] Ambari Views .. SUCCESS [1.400s]
> [INFO] Ambari Admin View . SUCCESS [9.242s]
> [INFO] ambari-metrics  SUCCESS [0.346s]
> [INFO] Ambari Metrics Common . SUCCESS [0.528s]
> [INFO] Ambari Metrics Hadoop Sink  SUCCESS [1.237s]
> [INFO] Ambari Metrics Flume Sink . SUCCESS [0.566s]
> [INFO] Ambari Metrics Kafka Sink . SUCCESS [0.600s]
> [INFO] Ambari Metrics Storm Sink . SUCCESS [1.531s]
> [INFO] Ambari Metrics Collector .. SUCCESS [7.183s]
> [INFO] Ambari Metrics Monitor  SUCCESS [1.853s]
> [INFO] Ambari Metrics Grafana  SUCCESS [1.031s]
> [INFO] Ambari Metrics Assembly ... SUCCESS [1:18.252s]
> [INFO] Ambari Server . SUCCESS [2:58.522s]
> [INFO] Ambari Functional Tests ... SUCCESS [2.194s]
> [INFO] Ambari Agent .. SUCCESS [25.339s]
> [INFO] Ambari Client . SUCCESS [0.047s]
> [INFO] Ambari Python Client .. SUCCESS [0.947s]
> [INFO] Ambari Groovy Client .. SUCCESS [2.196s]
> [INFO] Ambari Shell .. SUCCESS [0.038s]
> [INFO] Ambari Python Shell ... SUCCESS [0.693s]
> [INFO] Ambari Groovy Shell ... SUCCESS [0.907s]
> [INFO] ambari-logsearch .. SUCCESS [0.253s]
> [INFO] Ambari Logsearch Portal ... SUCCESS [7.069s]
> [INFO] Ambari Logsearch Log Feeder ... SUCCESS [1.570s]
> [INFO] Ambari Logsearch Appender . SUCCESS [0.190s]
> [INFO] Ambari Logsearch Assembly . SUCCESS [0.079s]
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 5:56.545s
> [INFO] Finished at: Thu Apr 21 16:14:34 PDT 2016
> [INFO] Final Memory: 274M/1094M
>

Re: Review Request 45792: Integrate Version Registration in Select Stack Page

2016-04-21 Thread Xi Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45792/
---

(Updated April 21, 2016, 5:26 p.m.)


Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, Richard Zang, and 
Yusaku Sako.


Changes
---

Including the new functional Register Version page.


Bugs: AMBARI-15724
https://issues.apache.org/jira/browse/AMBARI-15724


Repository: ambari


Description
---

Implement New "Select Stack Page".
Should have two main options : use local repo & use public repo.


Diffs (updated)
-

  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/stackVersions/StackVersionsCreateCtrl.js
 5596a7f 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/stackVersions/StackVersionsEditCtrl.js
 3da088b 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/i18n.config.js 
e95ebdb 
  ambari-admin/src/main/resources/ui/admin-web/app/scripts/services/Stack.js 
fd3a8ba 
  ambari-admin/src/main/resources/ui/admin-web/app/styles/main.css edf8524 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/stackVersions/stackVersionPage.html
 8e9d3e2 
  ambari-web/app/assets/data/wizard/stack/HDP_version_definitions.json 
PRE-CREATION 
  ambari-web/app/controllers/installer.js 73f9801 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js a056d3b 
  ambari-web/app/controllers/main/service/add_controller.js 73abd59 
  ambari-web/app/controllers/wizard.js 52fb241 
  ambari-web/app/controllers/wizard/step1_controller.js c357bf4 
  ambari-web/app/controllers/wizard/step8_controller.js ad53291 
  ambari-web/app/mappers/repository_version_mapper.js 67cfe64 
  ambari-web/app/mappers/stack_mapper.js 75245e0 
  ambari-web/app/messages.js 6c0049d 
  ambari-web/app/models/stack.js 0b1c9d2 
  ambari-web/app/routes/add_service_routes.js 2aaf1c1 
  ambari-web/app/styles/application.less 890a43c 
  ambari-web/app/templates/main/admin/stack_upgrade/services.hbs a87c50c 
  ambari-web/app/templates/main/admin/stack_upgrade/versions.hbs a9fc440 
  ambari-web/app/templates/wizard/step1.hbs ab9a0dd 
  ambari-web/app/templates/wizard/step4.hbs cc4c68f 
  ambari-web/app/utils/ajax/ajax.js 1cb70c5 
  ambari-web/app/utils/db.js 865839c 
  ambari-web/app/views/main/admin/stack_upgrade/services_view.js 8805c47 
  ambari-web/app/views/wizard/step1_view.js 452e676 
  ambari-web/test/controllers/installer_test.js 353129b 
  ambari-web/test/controllers/wizard/step8_test.js 4974460 
  ambari-web/test/mappers/stack_mapper_test.js 0bc0f3c 
  ambari-web/test/views/main/admin/stack_upgrade/services_view_test.js 9a42c42 
  ambari-web/test/views/wizard/step1_view_test.js 0dacad8 

Diff: https://reviews.apache.org/r/45792/diff/


Testing
---

25612 tests complete (26 seconds)
  154 tests pending


Thanks,

Xi Wang

Re: Review Request 46509: Remove performing service check during "Remove Standby Wizard"

2016-04-21 Thread bhuvnesh chaudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46509/
---

(Updated April 21, 2016, 11:01 p.m.)


Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr 
Diachenko.


Bugs: AMBARI-16024
https://issues.apache.org/jira/browse/AMBARI-16024


Repository: ambari


Description
---

Users will need to remove the HAWQ Standby Master using "Remove Standby Wizard" 
after enabling HDFS HA.
So the service check may fail during the wizard as the HAWQ catalog might be 
still pointing to the old filespace.


Diffs
-

  
ambari-web/app/controllers/main/admin/highAvailability/hawq/removeStandby/step3_controller.js
 5bcf34f 

Diff: https://reviews.apache.org/r/46509/diff/


Testing (updated)
---

yes.
  25660 tests complete (29 seconds)
  154 tests pending
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 46.511 s
[INFO] Finished at: 2016-04-21T16:03:11-07:00
[INFO] Final Memory: 14M/331M


Thanks,

bhuvnesh chaudhary

Re: Review Request 46475: Provide UI validation for widget_name and description fields in Create/Edit Widget pop-up.

2016-04-21 Thread Keta Patel



> On April 21, 2016, 3:23 p.m., Di Li wrote:
> > ambari-web/app/templates/main/service/widgets/create/step3.hbs, line 24
> > 
> >
> > Why are these styled as warnings? Shouldn't they be errors?

Hello Di,
I have corrected these warning messages to erro messages. They now appear in 
red. For this, where the error messages are displayed on top, I have uploaded 
the patch under "AMBARI-15979-topError.patch". This is just for your reference 
to compare the 2 styles (inline error messages vs top error messages).


- Keta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46475/#review129904
---


On April 21, 2016, 10:51 p.m., Keta Patel wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46475/
> ---
> 
> (Updated April 21, 2016, 10:51 p.m.)
> 
> 
> Review request for Ambari, Andrii Tkach and Di Li.
> 
> 
> Bugs: AMBARI-15979
> https://issues.apache.org/jira/browse/AMBARI-15979
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> ISSUE:
> The UI validation at present checks only the length of the user input for 
> widget_name and description fields. All characters are allowed to be stored 
> in the database through them. A more strict UI validation that limits the 
> type of characters entered for these fields will provide a good first line of 
> defense. 
> 
> Steps to reproduce:
> 1. Make sure you have Ambari Metrics service installed on your cluster.
> 2. On the Dashboard, select any service that makes use of Ambari Metrics, say 
> HDFS.
> 3. In the "Metrics" section, click the "Actions" button in the top-right 
> corner, and select "Create a new widget" option from the drop-down. 
> (attachment: create_widget_button_location.tiff)
> 4. Create widget pop-up is displayed. 
> 5. On Step-1, select any type for the widget and click "Next". (attachment: 
> create_widget_step1.tiff)
> 6. On Step-2, select any valid metrics parameter and click "Next". 
> (attachment: create_widget_step2.tiff)
> 7. On Step-3, for widget_name and description fields, you can enter any 
> character. No validation is present to check the contents. The only 
> validation present checks the length of the input text. 
> (attachments:
> create_widget_step3.tiff, 
> original_characters_allowed_for_name_and_description.tiff,
> original_length_validation_for_name.tiff,
> original_length_validation_for_description.tiff )
> 
> 
> FIX:
> The UI validation is enhanced by checking the content of the input for name 
> and description.
> The patch attached allows only alphanumeric, underscore, hyphen, space and 
> percentage symbol to be valid characters for both fields.
> The % symbol is added as part of the white-list as there are existing widgets 
> that contain % symbols in their names. In order to keep the characters in the 
> name consistent, this was decided.
> The description could probably have a little more flexibility in terms of 
> characters allowed. Any suggestions to update this validation would be 
> helpful.
> The warning message template is also updated to conform to the existing norms 
> used in several Ambari pop-ups (e.g. Manage Config Groups, Manage Alert 
> Groups).
> 
> The images attached (starting with "fixed_") show how the fix appears with 
> the patch.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/service/widgets/create/step3_controller.js 
> dd7a93f 
>   ambari-web/app/messages.js 6c0049d 
>   ambari-web/app/templates/main/service/widgets/create/step3.hbs 9f431af 
>   
> ambari-web/test/controllers/main/service/widgets/create/step3_controller_test.js
>  6f92142 
> 
> Diff: https://reviews.apache.org/r/46475/diff/
> 
> 
> Testing
> ---
> 
> Tests are added that check the validate functions added for widget name and 
> description.
> Both widget name and description are tested for:
> 1. all valid characters
> 2. invalid characters
> 3. length of input
> 4. empty string
> 
> Ambari-Web tests with the patch:
> 25671 tests complete (33 seconds)
> 154 tests pending
> 
> 
> File Attachments
> 
> 
> AMBARI-15979-inlineError.patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/21/03bc972a-b076-4520-948b-3a204082eca0__AMBARI-15979-inlineError.patch
> AMBARI-15979-topError.patch
>   
> https://reviews.apache.org/media/uploaded/files/2016/04/21/8b4e9a7b-96cb-4a17-9c9c-56e59b5fd349__AMBARI-15979-topError.patch
> 
> 
> Thanks,
> 
> Keta Patel
> 
>

Re: Review Request 46475: Provide UI validation for widget_name and description fields in Create/Edit Widget pop-up.

2016-04-21 Thread Keta Patel

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46475/
---

(Updated April 21, 2016, 10:51 p.m.)

Review request for Ambari, Andrii Tkach and Di Li.

Bugs: AMBARI-15979
https://issues.apache.org/jira/browse/AMBARI-15979

Repository: ambari

Description
---

ISSUE:
The UI validation at present checks only the length of the user input for
widget_name and description fields. All characters are allowed to be stored in
the database through them. A more strict UI validation that limits the type of
characters entered for these fields will provide a good first line of defense.

Steps to reproduce:
1. Make sure you have Ambari Metrics service installed on your cluster.
2. On the Dashboard, select any service that makes use of Ambari Metrics, say
HDFS.
3. In the "Metrics" section, click the "Actions" button in the top-right
corner, and select "Create a new widget" option from the drop-down.
(attachment: create_widget_button_location.tiff)
4. Create widget pop-up is displayed.
5. On Step-1, select any type for the widget and click "Next". (attachment:
create_widget_step1.tiff)
6. On Step-2, select any valid metrics parameter and click "Next". (attachment:
create_widget_step2.tiff)
7. On Step-3, for widget_name and description fields, you can enter any
character. No validation is present to check the contents. The only validation
present checks the length of the input text.
(attachments:
create_widget_step3.tiff,
original_characters_allowed_for_name_and_description.tiff,
original_length_validation_for_name.tiff,
original_length_validation_for_description.tiff )

FIX:
The UI validation is enhanced by checking the content of the input for name and
description.
The patch attached allows only alphanumeric, underscore, hyphen, space and
percentage symbol to be valid characters for both fields.
The % symbol is added as part of the white-list as there are existing widgets
that contain % symbols in their names. In order to keep the characters in the
name consistent, this was decided.
The description could probably have a little more flexibility in terms of
characters allowed. Any suggestions to update this validation would be helpful.
The warning message template is also updated to conform to the existing norms
used in several Ambari pop-ups (e.g. Manage Config Groups, Manage Alert Groups).

The images attached (starting with "fixed_") show how the fix appears with the
patch.

Diffs (updated)
-

ambari-web/app/controllers/main/service/widgets/create/step3_controller.js
dd7a93f
ambari-web/app/messages.js 6c0049d
ambari-web/app/templates/main/service/widgets/create/step3.hbs 9f431af

ambari-web/test/controllers/main/service/widgets/create/step3_controller_test.js
6f92142

Diff: https://reviews.apache.org/r/46475/diff/

Testing
---

Tests are added that check the validate functions added for widget name and
description.
Both widget name and description are tested for:
1. all valid characters
2. invalid characters
3. length of input
4. empty string

Ambari-Web tests with the patch:
25671 tests complete (33 seconds)
154 tests pending

File Attachments (updated)

AMBARI-15979-inlineError.patch

https://reviews.apache.org/media/uploaded/files/2016/04/21/03bc972a-b076-4520-948b-3a204082eca0__AMBARI-15979-inlineError.patch
AMBARI-15979-topError.patch

https://reviews.apache.org/media/uploaded/files/2016/04/21/8b4e9a7b-96cb-4a17-9c9c-56e59b5fd349__AMBARI-15979-topError.patch

Thanks,

Keta Patel

Re: Review Request 46543: After HSI is enabled in install wizard the validation call does not send HSI in the component list of the host_group resulting in validation errors

2016-04-21 Thread Yusaku Sako


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46543/#review129975
---


Ship it!




Ship It!

- Yusaku Sako


On April 21, 2016, 10:39 p.m., Jaimin Jetly wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46543/
> ---
> 
> (Updated April 21, 2016, 10:39 p.m.)
> 
> 
> Review request for Ambari, Srimanth Gunturi and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15997
> https://issues.apache.org/jira/browse/AMBARI-15997
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Validation API call made on leaving "Configure Services" page needs to 
> include HIVE_INTERACTIVE_SERVER in the host component layout if enable 
> inetarctive query is selected to be true.
> Validation API call made in an installed cluster on saving configuration must 
> add/delete HIVE_INTERACTIVE_SERVER if user has enable/disable interactive 
> query
> 
> 
> Diffs
> -
> 
>   ambari-web/app/app.js 5fa4448 
>   ambari-web/app/controllers/installer.js 9c92282 
>   ambari-web/app/controllers/main/service/info/configs.js b589040 
>   ambari-web/app/controllers/main/service/item.js b1ea523 
>   ambari-web/app/controllers/wizard/step6_controller.js ff8f36a 
>   ambari-web/app/controllers/wizard/step7/assign_master_controller.js 196932d 
>   ambari-web/app/mixins/common/serverValidator.js fca1550 
>   ambari-web/app/utils/blueprint.js 30e69af 
>   ambari-web/app/views/common/configs/widgets/config_widget_view.js 1078386 
>   ambari-web/test/controllers/wizard/step7/assign_master_controller_test.js 
> 2d467c2 
> 
> Diff: https://reviews.apache.org/r/46543/diff/
> 
> 
> Testing
> ---
> 
> Verified manually that the patch fixes the issue
> Verified that ambari-web unit tests works with the patch:
> 
>   27517 tests complete (31 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Jaimin Jetly
> 
>

Re: Review Request 46543: After HSI is enabled in install wizard the validation call does not send HSI in the component list of the host_group resulting in validation errors

2016-04-21 Thread Jaimin Jetly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46543/
---

(Updated April 21, 2016, 10:39 p.m.)


Review request for Ambari, Srimanth Gunturi and Yusaku Sako.


Bugs: AMBARI-15997
https://issues.apache.org/jira/browse/AMBARI-15997


Repository: ambari


Description (updated)
---

Validation API call made on leaving "Configure Services" page needs to include 
HIVE_INTERACTIVE_SERVER in the host component layout if enable inetarctive 
query is selected to be true.
Validation API call made in an installed cluster on saving configuration must 
add/delete HIVE_INTERACTIVE_SERVER if user has enable/disable interactive query


Diffs
-

  ambari-web/app/app.js 5fa4448 
  ambari-web/app/controllers/installer.js 9c92282 
  ambari-web/app/controllers/main/service/info/configs.js b589040 
  ambari-web/app/controllers/main/service/item.js b1ea523 
  ambari-web/app/controllers/wizard/step6_controller.js ff8f36a 
  ambari-web/app/controllers/wizard/step7/assign_master_controller.js 196932d 
  ambari-web/app/mixins/common/serverValidator.js fca1550 
  ambari-web/app/utils/blueprint.js 30e69af 
  ambari-web/app/views/common/configs/widgets/config_widget_view.js 1078386 
  ambari-web/test/controllers/wizard/step7/assign_master_controller_test.js 
2d467c2 

Diff: https://reviews.apache.org/r/46543/diff/


Testing
---

Verified manually that the patch fixes the issue
Verified that ambari-web unit tests works with the patch:

  27517 tests complete (31 seconds)
  154 tests pending


Thanks,

Jaimin Jetly

Re: Review Request 46544: AMBARI-16028: Namenode marked as INITIAL standby could potentially never start if other namenode is down

2016-04-21 Thread Jayush Luniya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46544/
---

(Updated April 21, 2016, 10:24 p.m.)

Review request for Ambari, Alejandro Fernandez, Robert Nettleton, and Sumit
Mohanty.

Bugs: AMBARI-16028
https://issues.apache.org/jira/browse/AMBARI-16028

Repository: ambari

Description (updated)
---

1. During Namenode HA blueprint deployment, we configure the name nodes to
start in active/standby mode based on the following properties
{
"hadoop-env": {
"properties" : {
"dfs_ha_initial_namenode_active" : "host1",
"dfs_ha_initial_namenode_standby" : "host2”
}
}
}
2. The current logic is to always bootstrap the name node marked as standby.
3. This will lead to the Namenode marked as Standby to never start under the
following situation

- Cluster is deployed successfully
- Both name nodes are stopped
- Start the name node marked as standby. Namenode will never start.
- This is because the standby name node will try to bootstrap again.
- However to bootstrap a name node an active name node is required. Based on
the HDFS logic the first step done when bootstrapping is to connect to the
Active Namenode.
- Also there is no need to bootstrap here as the name node should already be
bootstrapped and should come back up as “Active"

Fix:
- The fix is to maintain a bootstrap marker file (similar to the way we keep a
name node formatted marker file)
- In the INITIAL_START phase (during cluster deployment) we will always force
bootstrap so as to enforce the name node marked as Standby to wait for the
Active name node to come up, bootstrap and start in STANDBY node.
- Once we are out of INITIAL_START phase, we will bootstrap only if this name
node has not been bootstrapped in the past.
- We will not enforce bootstrapping only in the INITIAL_START phase because
there is a possibility during cluster deployment that both name nodes don’t
start and hence bootstrapping out of INITIAL_START phase would be required in
this case.

Diffs
-

ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
8b6c924

ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
d8ff3c5
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 1c08d57

Diff: https://reviews.apache.org/r/46544/diff/

Testing (updated)
---

Verified scenarios on live cluster.

mvn clean test -DskipSurefireTests
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 54.784s
[INFO] Finished at: Thu Apr 21 15:22:52 PDT 2016
[INFO] Final Memory: 64M/1172M
[INFO]

Thanks,

Jayush Luniya

Review Request 46544: AMBARI-16028: Namenode marked as INITIAL standby could potentially never start if other namenode is down

2016-04-21 Thread Jayush Luniya

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46544/
---

Review request for Ambari, Alejandro Fernandez, Robert Nettleton, and Sumit
Mohanty.

Bugs: AMBARI-16028
https://issues.apache.org/jira/browse/AMBARI-16028

Repository: ambari

Description
---

1. During Namenode HA blueprint deployment, we configure the name nodes to
start in active/standby mode based on the following properties
{
"hadoop-env": {
"properties" : {
"dfs_ha_initial_namenode_active" :
"jay-msft-1.c.pramod-thangali.internal",
"dfs_ha_initial_namenode_standby" :
"jay-msft-2.c.pramod-thangali.internal”
}
}
}
2. The current logic is to always bootstrap the name node marked as standby.
3. This will lead to the Namenode marked as Standby to never start under the
following situation
- Cluster is deployed successfully
- Both name nodes are stopped
- Start the name node marked as standby. Namenode will never start.
- This is because the standby name node will try to bootstrap again.
- However to bootstrap a name node an active name node is required. Based on
the HDFS logic the first step done when bootstrapping is to connect to the
Active Namenode.
- Also there is no need to bootstrap here as the name node should already be
bootstrapped and should come back up as “Active"

Diffs
-

ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/hdfs_namenode.py
8b6c924

ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
d8ff3c5
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_namenode.py 1c08d57

Diff: https://reviews.apache.org/r/46544/diff/

Testing
---

mvn clean test -DskipSurefireTests

Thanks,

Jayush Luniya

Review Request 46543: After HSI is enabled in install wizard the validation call does not send HSI in the component list of the host_group resulting in validation errors

2016-04-21 Thread Jaimin Jetly


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46543/
---

Review request for Ambari, Srimanth Gunturi and Yusaku Sako.


Bugs: AMBARI-15997
https://issues.apache.org/jira/browse/AMBARI-15997


Repository: ambari


Description
---

Validation API call made on leaving "Configure Services" page needs to include 
HIVE_INTERACTIVE_SERVER in the host component layout
Validation API call made in an installed cluster on saving configuration must 
add/delete HIVE_INTERACTIVE_SERVER if user has enable/disable interactive query


Diffs
-

  ambari-web/app/app.js 5fa4448 
  ambari-web/app/controllers/installer.js 9c92282 
  ambari-web/app/controllers/main/service/info/configs.js b589040 
  ambari-web/app/controllers/main/service/item.js b1ea523 
  ambari-web/app/controllers/wizard/step6_controller.js ff8f36a 
  ambari-web/app/controllers/wizard/step7/assign_master_controller.js 196932d 
  ambari-web/app/mixins/common/serverValidator.js fca1550 
  ambari-web/app/utils/blueprint.js 30e69af 
  ambari-web/app/views/common/configs/widgets/config_widget_view.js 1078386 
  ambari-web/test/controllers/wizard/step7/assign_master_controller_test.js 
2d467c2 

Diff: https://reviews.apache.org/r/46543/diff/


Testing
---

Verified manually that the patch fixes the issue
Verified that ambari-web unit tests works with the patch:

  27517 tests complete (31 seconds)
  154 tests pending


Thanks,

Jaimin Jetly

Re: Review Request 46507: Updates LogSearch Integration to obtain LogSearch credential from configuration

2016-04-21 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46507/#review129969
---


Ship it!




Ship It!

- Oliver Szabo


On April 21, 2016, 7:11 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46507/
> ---
> 
> (Updated April 21, 2016, 7:11 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16021
> https://issues.apache.org/jira/browse/AMBARI-16021
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch resolves issue AMBARI-16021. 
> 
> The LoggingRequestHelper implementation class initially used a hard-coded 
> credential to connect to the LogSearch service to perform logging query 
> requests.  A recent patch, AMBARI-15884, has added support for configuring 
> these credentials in the cluster's configuration, which is in line with 
> similar services in Ambari clusters. 
> 
> This patch implements the following:
> 
> 1. Updates the LogSearch integration code to obtain the LogSearch server's 
> credential from the current cluster's configuration.  If, for some reason, 
> that configuration is not available, the integration layer will attempt to 
> obtain the credential from Ambari's CredentialStoreService.  
> 2. Adds new unit tests for the LoggingRequestHelperImpl class, that verify 
> the basic behavior of the request methods in this class, including the new 
> support for obtaining credentials from configuration.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperFactoryImpl.java
>  970a92e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImpl.java
>  624977e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImplTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46507/diff/
> 
> 
> Testing
> ---
> 
> 1. Deployed a 1-node cluster with LogSearch enabled with my patch applied, 
> and verified that the cluster deployed successfully, that no extraneous logs 
> appeared in ambari-server.log or ambari-server.out, and also verified that 
> the REST APIs are returning the expected information. 
> 2. Deployed a 1-node cluster without LogSearch enabled, with my patch 
> applied, and verified that the cluster deployed successfully, and that no 
> LogSearch-related logs appeared in ambari-server.log or ambari-server.out. 
> 3. Ran the "mvn clean test" suite in ambari-server, which passed completely:
> 
> Results :
> 
> Tests run: 4257, Failures: 0, Errors: 0, Skipped: 32
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>

Re: Review Request 46513: YARN ATS Should Advertise a Version


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46513/#review129967
---


Ship it!




Ship It!

- Alejandro Fernandez


On April 21, 2016, 9:01 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46513/
> ---
> 
> (Updated April 21, 2016, 9:01 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Nate Cole.
> 
> 
> Bugs: AMBARI-16026
> https://issues.apache.org/jira/browse/AMBARI-16026
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In YARN's {{metainfo.xml}}, ATS does not have a {{versionAdvertised}} 
> element. As a result, the absence of this element causes Ambari to think that 
> YARN is not part of the {{hdp-select}} logic and we will never validate that 
> it is reporting the correct version on startup.
> 
> However, YARN ATS does have a version and an entry in {{hdp-select}}:
> ```
> if params.version and check_stack_feature(StackFeature.ROLLING_UPGRADE, 
> params.version):
>   conf_select.select(params.stack_name, "hadoop", params.version)
>   stack_select.select("hadoop-yarn-timelineserver", params.version)
> ```
> 
> ```
> [root@c6401 ~]# cat /usr/bin/hdp-select | grep "timeline"
>"hadoop-yarn-timelineserver": "hadoop-yarn",
>  "hadoop-yarn-timelineserver"],
> ```
> 
> This actually causes the ATS component to say it has a {{VERSION_MISMATCH}} 
> since upgrading HDP changes it's version when we were not expecting it to:
> 
> ```
> YARN_CLIENT 2.4.2.0-39 COMPLETE   UNKNOWN
> APP_TIMELINE_SERVER   2.4.2.0-39 VERSION_MISMATCH UNSECURED
> ZOOKEEPER_SERVER   2.4.2.0-39 COMPLETEUNSECURED
> ```
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/metainfo.xml 
> 2ebc3d7 
> 
> Diff: https://reviews.apache.org/r/46513/diff/
> 
> 
> Testing
> ---
> 
> Verified that ats existing in earlier version of HDP and successfully 
> upgraded from that old version.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 46507: Updates LogSearch Integration to obtain LogSearch credential from configuration

2016-04-21 Thread Sumit Mohanty


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46507/#review129965
---


Ship it!




Ship It!

- Sumit Mohanty


On April 21, 2016, 7:11 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46507/
> ---
> 
> (Updated April 21, 2016, 7:11 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16021
> https://issues.apache.org/jira/browse/AMBARI-16021
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch resolves issue AMBARI-16021. 
> 
> The LoggingRequestHelper implementation class initially used a hard-coded 
> credential to connect to the LogSearch service to perform logging query 
> requests.  A recent patch, AMBARI-15884, has added support for configuring 
> these credentials in the cluster's configuration, which is in line with 
> similar services in Ambari clusters. 
> 
> This patch implements the following:
> 
> 1. Updates the LogSearch integration code to obtain the LogSearch server's 
> credential from the current cluster's configuration.  If, for some reason, 
> that configuration is not available, the integration layer will attempt to 
> obtain the credential from Ambari's CredentialStoreService.  
> 2. Adds new unit tests for the LoggingRequestHelperImpl class, that verify 
> the basic behavior of the request methods in this class, including the new 
> support for obtaining credentials from configuration.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperFactoryImpl.java
>  970a92e 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImpl.java
>  624977e 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImplTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46507/diff/
> 
> 
> Testing
> ---
> 
> 1. Deployed a 1-node cluster with LogSearch enabled with my patch applied, 
> and verified that the cluster deployed successfully, that no extraneous logs 
> appeared in ambari-server.log or ambari-server.out, and also verified that 
> the REST APIs are returning the expected information. 
> 2. Deployed a 1-node cluster without LogSearch enabled, with my patch 
> applied, and verified that the cluster deployed successfully, and that no 
> LogSearch-related logs appeared in ambari-server.log or ambari-server.out. 
> 3. Ran the "mvn clean test" suite in ambari-server, which passed completely:
> 
> Results :
> 
> Tests run: 4257, Failures: 0, Errors: 0, Skipped: 32
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>

Re: Review Request 46460: AMBARI-16002 Integrate Red Hat Satellite option in Ambari Web


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46460/#review129960
---


Ship it!




Ship It!

- Nate Cole


On April 20, 2016, 7:32 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46460/
> ---
> 
> (Updated April 20, 2016, 7:32 p.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly, Nate Cole, Richard Zang, Xi Wang, 
> and Yusaku Sako.
> 
> 
> Bugs: AMBARI-16002
> https://issues.apache.org/jira/browse/AMBARI-16002
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Provide RedHat satellite option. It can be used wherever we show redhat6 
> urls, and applicable to redhat6/redhat7. For example, registering repos, etc.
> The API call to update links when using VDF can specify the new property. In 
> order to make it more generic (and default true), it is called 
> "ambari_managed_repositories". API looks like:
> PUT /api/v1/stacks/HDP/versions/2.3/repository_versions/2
> {
>   "RepositoryVersions" : {
> "id" : 2
>   },
>   "operating_systems" : [
> {
>   "OperatingSystems" : {
> "ambari_managed_repositories": false,
> "os_type" : "redhat6",
> "repository_version_id" : 2,
> "stack_name" : "HDP",
> "stack_version" : "2.3"
>   },
>   "repositories" : [
> {
>   "Repositories" : {
> "base_url" : "http://repos.ambari.apache.org/hdp/HDP-2.3.4.14-4;,
> "repo_id" : "HDP-2.3",
> "repo_name" : "HDP"
>   }
> },
> {
>   "Repositories" : {
> "base_url" : 
> "http://repos.ambari.apache.org/hdp/HDP-UTILS-1.1.0.20;,
> "repo_id" : "HDP-UTILS-1.1.0.20",
> "repo_name" : "HDP-UTILS"
>   }
> }
>   ]
> }
>   ]
> }
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js 
> 7a7d050 
>   ambari-web/app/mappers/repository_version_mapper.js 816308b 
>   ambari-web/app/models/stack_version/repository_version.js e02999f 
>   ambari-web/app/templates/main/admin/stack_upgrade/edit_repositories.hbs 
> 35677ab 
>   ambari-web/app/views/main/admin/stack_upgrade/upgrade_version_box_view.js 
> 88bfec2 
>   ambari-web/test/controllers/main/admin/stack_and_upgrade_controller_test.js 
> 00d7ce8 
> 
> Diff: https://reviews.apache.org/r/46460/diff/
> 
> 
> Testing
> ---
> 
> Modified unit test.
> Local ambari-web test passed.
> 25660 tests complete (24 seconds)
> 154 tests pending
> Manual testing done.
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Review Request 46513: YARN ATS Should Advertise a Version

2016-04-21 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46513/
---

Review request for Ambari, Alejandro Fernandez and Nate Cole.


Bugs: AMBARI-16026
https://issues.apache.org/jira/browse/AMBARI-16026


Repository: ambari


Description
---

In YARN's {{metainfo.xml}}, ATS does not have a {{versionAdvertised}} element. 
As a result, the absence of this element causes Ambari to think that YARN is 
not part of the {{hdp-select}} logic and we will never validate that it is 
reporting the correct version on startup.

However, YARN ATS does have a version and an entry in {{hdp-select}}:
```
if params.version and check_stack_feature(StackFeature.ROLLING_UPGRADE, 
params.version):
  conf_select.select(params.stack_name, "hadoop", params.version)
  stack_select.select("hadoop-yarn-timelineserver", params.version)
```

```
[root@c6401 ~]# cat /usr/bin/hdp-select | grep "timeline"
   "hadoop-yarn-timelineserver": "hadoop-yarn",
 "hadoop-yarn-timelineserver"],
```

This actually causes the ATS component to say it has a {{VERSION_MISMATCH}} 
since upgrading HDP changes it's version when we were not expecting it to:

```
YARN_CLIENT 2.4.2.0-39 COMPLETE UNKNOWN
APP_TIMELINE_SERVER 2.4.2.0-39 VERSION_MISMATCH UNSECURED
ZOOKEEPER_SERVER 2.4.2.0-39 COMPLETEUNSECURED
```


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.2/services/YARN/metainfo.xml 
2ebc3d7 

Diff: https://reviews.apache.org/r/46513/diff/


Testing
---

Verified that ats existing in earlier version of HDP and successfully upgraded 
from that old version.


Thanks,

Jonathan Hurley

Re: Review Request 46511: Installer wizard: Starting Services hangs because of LLAP status check.

2016-04-21 Thread Swapan Shridhar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46511/
---

(Updated April 21, 2016, 8:28 p.m.)


Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Sumit Mohanty, 
and Yusaku Sako.


Changes
---

Disabled only the LLAP status check in status() call.


Bugs: AMBARI-16025
https://issues.apache.org/jira/browse/AMBARI-16025


Repository: ambari


Description
---

Disabling the status check code as of now .
- Status check takes more than 5 secs to complete.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
 430b41f 

Diff: https://reviews.apache.org/r/46511/diff/


Testing
---

Tested HSI start.
Python UT passes.


Thanks,

Swapan Shridhar

Re: Review Request 45792: Integrate Version Registration in Select Stack Page

2016-04-21 Thread Xi Wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45792/
---

(Updated April 21, 2016, 1:23 p.m.)


Review request for Ambari, Jaimin Jetly, Zhe (Joe) Wang, Richard Zang, and 
Yusaku Sako.


Changes
---

Fixed some UI issues.


Bugs: AMBARI-15724
https://issues.apache.org/jira/browse/AMBARI-15724


Repository: ambari


Description
---

Implement New "Select Stack Page".
Should have two main options : use local repo & use public repo.


Diffs (updated)
-

  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/stackVersions/StackVersionsCreateCtrl.js
 5596a7f 
  
ambari-admin/src/main/resources/ui/admin-web/app/scripts/controllers/stackVersions/StackVersionsEditCtrl.js
 3da088b 
  ambari-admin/src/main/resources/ui/admin-web/app/styles/main.css edf8524 
  
ambari-admin/src/main/resources/ui/admin-web/app/views/stackVersions/stackVersionPage.html
 8e9d3e2 
  ambari-web/app/assets/data/wizard/stack/HDP_version_definitions.json 
PRE-CREATION 
  ambari-web/app/controllers/installer.js 9c92282 
  ambari-web/app/controllers/main/admin/stack_and_upgrade_controller.js a056d3b 
  ambari-web/app/controllers/main/service/add_controller.js 73abd59 
  ambari-web/app/controllers/wizard.js 52fb241 
  ambari-web/app/controllers/wizard/step1_controller.js c357bf4 
  ambari-web/app/controllers/wizard/step8_controller.js ad53291 
  ambari-web/app/mappers/repository_version_mapper.js 67cfe64 
  ambari-web/app/mappers/stack_mapper.js 75245e0 
  ambari-web/app/messages.js 6c0049d 
  ambari-web/app/models/stack.js 0b1c9d2 
  ambari-web/app/routes/add_service_routes.js 2aaf1c1 
  ambari-web/app/styles/application.less 890a43c 
  ambari-web/app/templates/main/admin/stack_upgrade/services.hbs a87c50c 
  ambari-web/app/templates/main/admin/stack_upgrade/versions.hbs a9fc440 
  ambari-web/app/templates/wizard/step1.hbs ab9a0dd 
  ambari-web/app/templates/wizard/step4.hbs cc4c68f 
  ambari-web/app/utils/ajax/ajax.js 72364f0 
  ambari-web/app/utils/db.js 865839c 
  ambari-web/app/views/main/admin/stack_upgrade/services_view.js 8805c47 
  ambari-web/app/views/wizard/step1_view.js 452e676 
  ambari-web/test/controllers/installer_test.js 353129b 
  ambari-web/test/controllers/wizard/step8_test.js 4974460 
  ambari-web/test/mappers/stack_mapper_test.js 0bc0f3c 
  ambari-web/test/views/main/admin/stack_upgrade/services_view_test.js 9a42c42 
  ambari-web/test/views/wizard/step1_view_test.js 0dacad8 

Diff: https://reviews.apache.org/r/45792/diff/


Testing
---

25612 tests complete (26 seconds)
  154 tests pending


Thanks,

Xi Wang

Re: Review Request 46511: Installer wizard: Starting Services hangs because of LLAP status check.

2016-04-21 Thread Swapan Shridhar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46511/
---

(Updated April 21, 2016, 8 p.m.)


Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Sumit Mohanty, 
and Yusaku Sako.


Bugs: AMBARI-16025
https://issues.apache.org/jira/browse/AMBARI-16025


Repository: ambari


Description (updated)
---

Disabling the status check code as of now .
- Status check takes more than 5 secs to complete.


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
 430b41f 
  ambari-server/src/test/python/stacks/2.5/HIVE/test_hive_server_int.py 44759a9 

Diff: https://reviews.apache.org/r/46511/diff/


Testing
---

Tested HSI start.
Python UT passes.


Thanks,

Swapan Shridhar

Re: Review Request 46508: AMBARI-15892 : Incorrect (Negative) values are shown for memory metrics (Patch 2)

2016-04-21 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46508/#review129945
---


Ship it!




Ship It!

- Sid Wagle


On April 21, 2016, 7:10 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46508/
> ---
> 
> (Updated April 21, 2016, 7:10 p.m.)
> 
> 
> Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-15892
> https://issues.apache.org/jira/browse/AMBARI-15892
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove double subtraction of mem_cached in Yarn Cluster memory computed 
> metric widget.
> 
> Change HBase Reads and Writes widget description from Count to Rate.
> 
> Added changes that were made through AMBARI-15026 to the Upgrade path. This 
> makes sure hadoop daemons reported data at 10 sec intervals instead of 1 
> minute interval.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
>  c999d52 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/widgets.json
>  ede9f49 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/YARN_widgets.json
>  442d9a2 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/widgets.json 
> d8f4e6a 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/YARN_widgets.json
>  c2acf1f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
>  f543f18 
> 
> Diff: https://reviews.apache.org/r/46508/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed.
> 
> Manually tested.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Review Request 46509: Remove performing service check during "Remove Standby Wizard"

2016-04-21 Thread bhuvnesh chaudhary


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46509/
---

Review request for Ambari, Alexander Denissov, jun aoki, Matt, and Oleksandr 
Diachenko.


Bugs: AMBARI-16024
https://issues.apache.org/jira/browse/AMBARI-16024


Repository: ambari


Description
---

Users will need to remove the HAWQ Standby Master using "Remove Standby Wizard" 
after enabling HDFS HA.
So the service check may fail during the wizard as the HAWQ catalog might be 
still pointing to the old filespace.


Diffs
-

  
ambari-web/app/controllers/main/admin/highAvailability/hawq/removeStandby/step3_controller.js
 5bcf34f 

Diff: https://reviews.apache.org/r/46509/diff/


Testing
---

yes.
~/github/ambari-vagrant/centos6.4/ambari/ambari-server/src/test/python/stacks/2.3/HAWQ
 ~/github/ambari-vagrant/centos6.4/ambari
test_hawq_master_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_master_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_segment_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_segment_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_hawq_standby_critical 
(test_alert_component_status.TestAlertComponentStatus) ... ok
test_hawq_standby_ok (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_missing_configs (test_alert_component_status.TestAlertComponentStatus) ... 
ok
test_exception_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... No 
handlers could be found for logger "ambari_alerts"
ok
test_missing_configs 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_slave_file 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_successful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_empty_db_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_unsuccessful_registration_status_plural 
(test_alert_segment_registration_status.TestAlertRegistrationStatus) ... ok
test_missing_configs (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_no_standby_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_none_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_configured_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_not_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronized_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_synchronizing_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_unknown_state (test_alert_sync_status.TestAlertSyncStatus) ... ok
test_configure_default (test_hawqmaster.TestHawqMaster) ... ok
test_install_default (test_hawqmaster.TestHawqMaster) ... ok
test_start_default (test_hawqmaster.TestHawqMaster) ... 2016-04-21 12:15:00,349 
- Skipping ssh key exchange with HAWQ hosts as hawq_ssh_exkeys is either set to 
false or is not available in hawq-env.xml
ok
test_stop_default (test_hawqmaster.TestHawqMaster) ... ok
test_configure_default (test_hawqsegment.TestHawqSegment) ... ok
test_install_default (test_hawqsegment.TestHawqSegment) ... ok
test_start_default (test_hawqsegment.TestHawqSegment) ... ok
test_stop_default (test_hawqsegment.TestHawqSegment) ... ok
test_configure_default (test_hawqstandby.TestHawqStandby) ... ok
test_install_default (test_hawqstandby.TestHawqStandby) ... ok
test_start_default (test_hawqstandby.TestHawqStandby) ... ok
test_stop_default (test_hawqstandby.TestHawqStandby) ... ok
test_generate_hawq_process_status_cmd (test_utils.TestUtils) ... ok

--
Ran 35 tests in 0.178s

OK


Thanks,

bhuvnesh chaudhary

Review Request 46507: Updates LogSearch Integration to obtain LogSearch credential from configuration

2016-04-21 Thread Robert Nettleton

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46507/
---

Review request for Ambari, Oliver Szabo and Sumit Mohanty.

Bugs: AMBARI-16021
https://issues.apache.org/jira/browse/AMBARI-16021

Repository: ambari

Description
---

This patch resolves issue AMBARI-16021.

The LoggingRequestHelper implementation class initially used a hard-coded
credential to connect to the LogSearch service to perform logging query
requests. A recent patch, AMBARI-15884, has added support for configuring
these credentials in the cluster's configuration, which is in line with similar
services in Ambari clusters.

This patch implements the following:

1. Updates the LogSearch integration code to obtain the LogSearch server's
credential from the current cluster's configuration. If, for some reason, that
configuration is not available, the integration layer will attempt to obtain
the credential from Ambari's CredentialStoreService.
2. Adds new unit tests for the LoggingRequestHelperImpl class, that verify the
basic behavior of the request methods in this class, including the new support
for obtaining credentials from configuration.

Diffs
-

ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperFactoryImpl.java
970a92e

ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImpl.java
624977e

ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImplTest.java
PRE-CREATION

Diff: https://reviews.apache.org/r/46507/diff/

Testing
---

1. Deployed a 1-node cluster with LogSearch enabled with my patch applied, and
verified that the cluster deployed successfully, that no extraneous logs
appeared in ambari-server.log or ambari-server.out, and also verified that the
REST APIs are returning the expected information.
2. Deployed a 1-node cluster without LogSearch enabled, with my patch applied,
and verified that the cluster deployed successfully, and that no
LogSearch-related logs appeared in ambari-server.log or ambari-server.out.
3. Ran the "mvn clean test" suite in ambari-server, which passed completely:

Results :

Tests run: 4257, Failures: 0, Errors: 0, Skipped: 32

Thanks,

Robert Nettleton

Review Request 46508: AMBARI-15892 : Incorrect (Negative) values are shown for memory metrics (Patch 2)

2016-04-21 Thread Aravindan Vijayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46508/
---

Review request for Ambari, Dmytro Sen, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-15892
https://issues.apache.org/jira/browse/AMBARI-15892


Repository: ambari


Description
---

Remove double subtraction of mem_cached in Yarn Cluster memory computed metric 
widget.

Change HBase Reads and Writes widget description from Count to Rate.

Added changes that were made through AMBARI-15026 to the Upgrade path. This 
makes sure hadoop daemons reported data at 10 sec intervals instead of 1 minute 
interval.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog222.java
 c999d52 
  
ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/widgets.json 
ede9f49 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/YARN_widgets.json
 442d9a2 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/HBASE/widgets.json 
d8f4e6a 
  
ambari-server/src/main/resources/stacks/HDP/2.3/services/YARN/YARN_widgets.json 
c2acf1f 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog222Test.java
 f543f18 

Diff: https://reviews.apache.org/r/46508/diff/


Testing
---

Unit tests passed.

Manually tested.


Thanks,

Aravindan Vijayan

Re: Review Request 46484: Enhance Health Check for the Cluster before upgrading.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46484/#review129931
---




ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
(line 67)


I thought we were going to filter for only a handful of alerts based on the 
name.
E.g., out of disk space, ambari performance, etc.


- Alejandro Fernandez


On April 21, 2016, 11:14 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46484/
> ---
> 
> (Updated April 21, 2016, 11:14 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-16005
> https://issues.apache.org/jira/browse/AMBARI-16005
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhance Health Check for the Cluster before upgrading.
> 
> * check for alerts (perhaps don't start RU/EU if any blocker alerts).
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  3e957b1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HealthCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46484/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46496: Host_status stuck in UNKNOWN status after blueprint deploy with host in heartbeat-lost

2016-04-21 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46496/#review129926
---


Ship it!




Ship It!

- Sid Wagle


On April 21, 2016, 3:51 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46496/
> ---
> 
> (Updated April 21, 2016, 3:51 p.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16013
> https://issues.apache.org/jira/browse/AMBARI-16013
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When hosts register to Ambari server the `TopologyManager` adds these to its 
> `availableHosts` collection. When a cluster is provisioned using Blueprints 
> `TopologyManager` tries to allocate required hosts to hostgroups from the 
> available hosts collection. In case hosts turned into HEARTBEAT_LOST state 
> these were not removed from `availableHosts` this resulting scheduling 
> logical tasks to unreachable hosts. When these unreachable hosts become 
> available re-register with Ambari server. The server since already scheduled 
> logical tasks for these it won't try again thus will never create role 
> commands to be executed by the hosts.
> 
> `TopologyManager` has been hooked now to the HEARTBEAT_LOST state transition 
> to remove the host in question from its internal `availableHosts` collection.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> d221112 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  5a0aca0 
> 
> Diff: https://reviews.apache.org/r/46496/diff/
> 
> 
> Testing
> ---
> 
> Manual testign with a 5 node cluster using Blueprints.
> 
> Unit tests:
> Results :
> 
> Tests run: 3561, Failures: 0, Errors: 0, Skipped: 36
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 46500: Regenerating keytabs on re-imaged hosts results in error during 'Creating Principals'

2016-04-21 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46500/#review129924
---


Ship it!




Ship It!

- Oliver Szabo


On April 21, 2016, 5:01 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46500/
> ---
> 
> (Updated April 21, 2016, 5:01 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Srimanth 
> Gunturi, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16009
> https://issues.apache.org/jira/browse/AMBARI-16009
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We had a 1600 unsecured cluster initially, from which 700 nodes were 
> destroyed. Though Ambari-server knew of 1600 hosts, only 900 were 
> heartbeating. At this point we secured the cluster and everything was good. 
> Then we brought back the 700 hosts, which started heartbeating with 
> ambari-server. 
> 
> At this point we did 'Regenerate Keytabs' which failed at the 'Create 
> Principals' step (image attached), as it was trying to re-create principal 
> which is already existing with kadmin, and with ambari-server.
> 
> #Create Principals
> Stderr:
> ```
> 2016-04-21 01:28:52,985 - Failed to create or update principal, 
> HTTP/host1.example@example.com - Failed to create service principal for 
> HTTP/host1.example@example.com
> STDOUT: Authenticating as principal admin/admin with password.
> 
> STDERR: WARNING: no policy specified for HTTP/host1.example@example.com; 
> defaulting to no policy
> add_principal: Principal or policy already exists while creating 
> "HTTP/host1.example@example.com".
> {noformat}
> 
> Stdout:
> {noformat}
> 2016-04-21 01:27:32,400 - Processing identities...
> 2016-04-21 01:28:29,874 - Processing principal, 
> HTTP/host1.example@example.com
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
>  f48052f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandlerTest.java
>  d15db17 
> 
> Diff: https://reviews.apache.org/r/46500/diff/
> 
> 
> Testing
> ---
> 
> Manually tested enabling Kerberos and Regenerating Keytabs both having 
> unexpected principals in or removed from the KDC as needed to attempt to 
> generate the issue. 
> 
> #Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:19:08.563s
> [INFO] Finished at: Thu Apr 21 12:45:33 EDT 2016
> [INFO] Final Memory: 60M/1866M
> [INFO] 
> 
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 46500: Regenerating keytabs on re-imaged hosts results in error during 'Creating Principals'

2016-04-21 Thread Sid Wagle


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46500/#review129923
---


Ship it!




Ship It!

- Sid Wagle


On April 21, 2016, 5:01 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46500/
> ---
> 
> (Updated April 21, 2016, 5:01 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Srimanth 
> Gunturi, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16009
> https://issues.apache.org/jira/browse/AMBARI-16009
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We had a 1600 unsecured cluster initially, from which 700 nodes were 
> destroyed. Though Ambari-server knew of 1600 hosts, only 900 were 
> heartbeating. At this point we secured the cluster and everything was good. 
> Then we brought back the 700 hosts, which started heartbeating with 
> ambari-server. 
> 
> At this point we did 'Regenerate Keytabs' which failed at the 'Create 
> Principals' step (image attached), as it was trying to re-create principal 
> which is already existing with kadmin, and with ambari-server.
> 
> #Create Principals
> Stderr:
> ```
> 2016-04-21 01:28:52,985 - Failed to create or update principal, 
> HTTP/host1.example@example.com - Failed to create service principal for 
> HTTP/host1.example@example.com
> STDOUT: Authenticating as principal admin/admin with password.
> 
> STDERR: WARNING: no policy specified for HTTP/host1.example@example.com; 
> defaulting to no policy
> add_principal: Principal or policy already exists while creating 
> "HTTP/host1.example@example.com".
> {noformat}
> 
> Stdout:
> {noformat}
> 2016-04-21 01:27:32,400 - Processing identities...
> 2016-04-21 01:28:29,874 - Processing principal, 
> HTTP/host1.example@example.com
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
>  f48052f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandlerTest.java
>  d15db17 
> 
> Diff: https://reviews.apache.org/r/46500/diff/
> 
> 
> Testing
> ---
> 
> Manually tested enabling Kerberos and Regenerating Keytabs both having 
> unexpected principals in or removed from the KDC as needed to attempt to 
> generate the issue. 
> 
> #Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:19:08.563s
> [INFO] Finished at: Thu Apr 21 12:45:33 EDT 2016
> [INFO] Final Memory: 60M/1866M
> [INFO] 
> 
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 46500: Regenerating keytabs on re-imaged hosts results in error during 'Creating Principals'

2016-04-21 Thread Srimanth Gunturi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46500/#review129921
---


Ship it!




Ship It!

- Srimanth Gunturi


On April 21, 2016, 5:01 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46500/
> ---
> 
> (Updated April 21, 2016, 5:01 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Srimanth 
> Gunturi, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16009
> https://issues.apache.org/jira/browse/AMBARI-16009
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We had a 1600 unsecured cluster initially, from which 700 nodes were 
> destroyed. Though Ambari-server knew of 1600 hosts, only 900 were 
> heartbeating. At this point we secured the cluster and everything was good. 
> Then we brought back the 700 hosts, which started heartbeating with 
> ambari-server. 
> 
> At this point we did 'Regenerate Keytabs' which failed at the 'Create 
> Principals' step (image attached), as it was trying to re-create principal 
> which is already existing with kadmin, and with ambari-server.
> 
> #Create Principals
> Stderr:
> ```
> 2016-04-21 01:28:52,985 - Failed to create or update principal, 
> HTTP/host1.example@example.com - Failed to create service principal for 
> HTTP/host1.example@example.com
> STDOUT: Authenticating as principal admin/admin with password.
> 
> STDERR: WARNING: no policy specified for HTTP/host1.example@example.com; 
> defaulting to no policy
> add_principal: Principal or policy already exists while creating 
> "HTTP/host1.example@example.com".
> {noformat}
> 
> Stdout:
> {noformat}
> 2016-04-21 01:27:32,400 - Processing identities...
> 2016-04-21 01:28:29,874 - Processing principal, 
> HTTP/host1.example@example.com
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
>  f48052f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandlerTest.java
>  d15db17 
> 
> Diff: https://reviews.apache.org/r/46500/diff/
> 
> 
> Testing
> ---
> 
> Manually tested enabling Kerberos and Regenerating Keytabs both having 
> unexpected principals in or removed from the KDC as needed to attempt to 
> generate the issue. 
> 
> #Local test results:
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 1:19:08.563s
> [INFO] Finished at: Thu Apr 21 12:45:33 EDT 2016
> [INFO] Final Memory: 60M/1866M
> [INFO] 
> 
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Review Request 46500: Regenerating keytabs on re-imaged hosts results in error during 'Creating Principals'

2016-04-21 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46500/
---

Review request for Ambari, Jonathan Hurley, Nate Cole, Oliver Szabo, Srimanth 
Gunturi, and Sid Wagle.


Bugs: AMBARI-16009
https://issues.apache.org/jira/browse/AMBARI-16009


Repository: ambari


Description
---

We had a 1600 unsecured cluster initially, from which 700 nodes were destroyed. 
Though Ambari-server knew of 1600 hosts, only 900 were heartbeating. At this 
point we secured the cluster and everything was good. Then we brought back the 
700 hosts, which started heartbeating with ambari-server. 

At this point we did 'Regenerate Keytabs' which failed at the 'Create 
Principals' step (image attached), as it was trying to re-create principal 
which is already existing with kadmin, and with ambari-server.

#Create Principals
Stderr:
```
2016-04-21 01:28:52,985 - Failed to create or update principal, 
HTTP/host1.example@example.com - Failed to create service principal for 
HTTP/host1.example@example.com
STDOUT: Authenticating as principal admin/admin with password.

STDERR: WARNING: no policy specified for HTTP/host1.example@example.com; 
defaulting to no policy
add_principal: Principal or policy already exists while creating 
"HTTP/host1.example@example.com".
{noformat}

Stdout:
{noformat}
2016-04-21 01:27:32,400 - Processing identities...
2016-04-21 01:28:29,874 - Processing principal, 
HTTP/host1.example@example.com
```


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandler.java
 f48052f 
  
ambari-server/src/test/java/org/apache/ambari/server/serveraction/kerberos/MITKerberosOperationHandlerTest.java
 d15db17 

Diff: https://reviews.apache.org/r/46500/diff/


Testing
---

Manually tested enabling Kerberos and Regenerating Keytabs both having 
unexpected principals in or removed from the KDC as needed to attempt to 
generate the issue. 

#Local test results:
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 1:19:08.563s
[INFO] Finished at: Thu Apr 21 12:45:33 EDT 2016
[INFO] Final Memory: 60M/1866M
[INFO] 

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 46496: Host_status stuck in UNKNOWN status after blueprint deploy with host in heartbeat-lost

2016-04-21 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46496/
---

(Updated April 21, 2016, 5:51 p.m.)


Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, Sumit 
Mohanty, and Sid Wagle.


Changes
---

Address review comments.


Bugs: AMBARI-16013
https://issues.apache.org/jira/browse/AMBARI-16013


Repository: ambari


Description
---

When hosts register to Ambari server the `TopologyManager` adds these to its 
`availableHosts` collection. When a cluster is provisioned using Blueprints 
`TopologyManager` tries to allocate required hosts to hostgroups from the 
available hosts collection. In case hosts turned into HEARTBEAT_LOST state 
these were not removed from `availableHosts` this resulting scheduling logical 
tasks to unreachable hosts. When these unreachable hosts become available 
re-register with Ambari server. The server since already scheduled logical 
tasks for these it won't try again thus will never create role commands to be 
executed by the hosts.

`TopologyManager` has been hooked now to the HEARTBEAT_LOST state transition to 
remove the host in question from its internal `availableHosts` collection.


Diffs (updated)
-

  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
d221112 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 5a0aca0 

Diff: https://reviews.apache.org/r/46496/diff/


Testing
---

Manual testign with a 5 node cluster using Blueprints.

Unit tests:
Results :

Tests run: 3561, Failures: 0, Errors: 0, Skipped: 36


Thanks,

Sebastian Toader

Re: Review Request 46496: Host_status stuck in UNKNOWN status after blueprint deploy with host in heartbeat-lost

2016-04-21 Thread Sandor Magyari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46496/#review129910
---


Ship it!




Ship It!

- Sandor Magyari


On April 21, 2016, 3:19 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46496/
> ---
> 
> (Updated April 21, 2016, 3:19 p.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16013
> https://issues.apache.org/jira/browse/AMBARI-16013
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When hosts register to Ambari server the `TopologyManager` adds these to its 
> `availableHosts` collection. When a cluster is provisioned using Blueprints 
> `TopologyManager` tries to allocate required hosts to hostgroups from the 
> available hosts collection. In case hosts turned into HEARTBEAT_LOST state 
> these were not removed from `availableHosts` this resulting scheduling 
> logical tasks to unreachable hosts. When these unreachable hosts become 
> available re-register with Ambari server. The server since already scheduled 
> logical tasks for these it won't try again thus will never create role 
> commands to be executed by the hosts.
> 
> `TopologyManager` has been hooked now to the HEARTBEAT_LOST state transition 
> to remove the host in question from its internal `availableHosts` collection.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> d221112 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  5a0aca0 
> 
> Diff: https://reviews.apache.org/r/46496/diff/
> 
> 
> Testing
> ---
> 
> Manual testign with a 5 node cluster using Blueprints.
> 
> Unit tests:
> Results :
> 
> Tests run: 3561, Failures: 0, Errors: 0, Skipped: 36
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 46496: Host_status stuck in UNKNOWN status after blueprint deploy with host in heartbeat-lost

2016-04-21 Thread Laszlo Puskas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46496/#review129906
---




ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 (line 395)


Add the host to the log message


- Laszlo Puskas


On April 21, 2016, 3:19 p.m., Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46496/
> ---
> 
> (Updated April 21, 2016, 3:19 p.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16013
> https://issues.apache.org/jira/browse/AMBARI-16013
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When hosts register to Ambari server the `TopologyManager` adds these to its 
> `availableHosts` collection. When a cluster is provisioned using Blueprints 
> `TopologyManager` tries to allocate required hosts to hostgroups from the 
> available hosts collection. In case hosts turned into HEARTBEAT_LOST state 
> these were not removed from `availableHosts` this resulting scheduling 
> logical tasks to unreachable hosts. When these unreachable hosts become 
> available re-register with Ambari server. The server since already scheduled 
> logical tasks for these it won't try again thus will never create role 
> commands to be executed by the hosts.
> 
> `TopologyManager` has been hooked now to the HEARTBEAT_LOST state transition 
> to remove the host in question from its internal `availableHosts` collection.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> d221112 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  5a0aca0 
> 
> Diff: https://reviews.apache.org/r/46496/diff/
> 
> 
> Testing
> ---
> 
> Manual testign with a 5 node cluster using Blueprints.
> 
> Unit tests:
> Results :
> 
> Tests run: 3561, Failures: 0, Errors: 0, Skipped: 36
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Re: Review Request 46496: Host_status stuck in UNKNOWN status after blueprint deploy with host in heartbeat-lost

2016-04-21 Thread Daniel Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46496/#review129905
---


Fix it, then Ship it!





ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 (line 51)


I cannot see where HostState is used.



ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 (lines 394 - 397)


isEmpty check is not necessary here, contains is enough.
Even the whole if statement is needed only for logging.
So remove operation can be executed directly if log message is not 
misleading in the case when availableHosts does not contain host. (is it 
possible?)


- Daniel Gergely


On ápr. 21, 2016, 3:19 du, Sebastian Toader wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46496/
> ---
> 
> (Updated ápr. 21, 2016, 3:19 du)
> 
> 
> Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, 
> Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-16013
> https://issues.apache.org/jira/browse/AMBARI-16013
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When hosts register to Ambari server the `TopologyManager` adds these to its 
> `availableHosts` collection. When a cluster is provisioned using Blueprints 
> `TopologyManager` tries to allocate required hosts to hostgroups from the 
> available hosts collection. In case hosts turned into HEARTBEAT_LOST state 
> these were not removed from `availableHosts` this resulting scheduling 
> logical tasks to unreachable hosts. When these unreachable hosts become 
> available re-register with Ambari server. The server since already scheduled 
> logical tasks for these it won't try again thus will never create role 
> commands to be executed by the hosts.
> 
> `TopologyManager` has been hooked now to the HEARTBEAT_LOST state transition 
> to remove the host in question from its internal `availableHosts` collection.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
> d221112 
>   
> ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
>  5a0aca0 
> 
> Diff: https://reviews.apache.org/r/46496/diff/
> 
> 
> Testing
> ---
> 
> Manual testign with a 5 node cluster using Blueprints.
> 
> Unit tests:
> Results :
> 
> Tests run: 3561, Failures: 0, Errors: 0, Skipped: 36
> 
> 
> Thanks,
> 
> Sebastian Toader
> 
>

Review Request 46496: Host_status stuck in UNKNOWN status after blueprint deploy with host in heartbeat-lost

2016-04-21 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46496/
---

Review request for Ambari, Daniel Gergely, Laszlo Puskas, Sandor Magyari, Sumit 
Mohanty, and Sid Wagle.


Bugs: AMBARI-16013
https://issues.apache.org/jira/browse/AMBARI-16013


Repository: ambari


Description
---

When hosts register to Ambari server the `TopologyManager` adds these to its 
`availableHosts` collection. When a cluster is provisioned using Blueprints 
`TopologyManager` tries to allocate required hosts to hostgroups from the 
available hosts collection. In case hosts turned into HEARTBEAT_LOST state 
these were not removed from `availableHosts` this resulting scheduling logical 
tasks to unreachable hosts. When these unreachable hosts become available 
re-register with Ambari server. The server since already scheduled logical 
tasks for these it won't try again thus will never create role commands to be 
executed by the hosts.

`TopologyManager` has been hooked now to the HEARTBEAT_LOST state transition to 
remove the host in question from its internal `availableHosts` collection.


Diffs
-

  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
d221112 
  
ambari-server/src/main/java/org/apache/ambari/server/topology/TopologyManager.java
 5a0aca0 

Diff: https://reviews.apache.org/r/46496/diff/


Testing
---

Manual testign with a 5 node cluster using Blueprints.

Unit tests:
Results :

Tests run: 3561, Failures: 0, Errors: 0, Skipped: 36


Thanks,

Sebastian Toader

Re: Review Request 45924: AMBARI-15681 : Ranger Admin/Usersync: Make log4j configurable from Ambari


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45924/#review129898
---


Ship it!




- Nate Cole


On April 18, 2016, 12:22 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45924/
> ---
> 
> (Updated April 18, 2016, 12:22 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Mahadev Konar, 
> Nate Cole, Sumit Mohanty, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-15681
> https://issues.apache.org/jira/browse/AMBARI-15681
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> **Problem Statement**
> 
> Currently log4j configuration for Ranger Admin / UserSync and TagSync cannot 
> be updated from Ambari. This forces the users to directly edit the 
> configuration file.
> 
> **Proposed Solution**
> Exposing log4j configurations for individual compomnents to Ambari as a 
> properties file.
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  86dbc48 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  7201c36 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  c0ab3e8 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  9544ee8 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  03b7e01 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/admin-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/tagsync-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER/configuration/usersync-log4j.xml
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/45924/diff/
> 
> 
> Testing
> ---
> 
> Verifed Ranger Admin and usersync log files being created based on configs 
> given in log4j.properties block from Ambari UI.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 46493: ambari-agent.log gets flooded up with exceptions from status commands

2016-04-21 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46493/#review129895
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On Квітень 21, 2016, 2:04 після полудня, Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46493/
> ---
> 
> (Updated Квітень 21, 2016, 2:04 після полудня)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-16014
> https://issues.apache.org/jira/browse/AMBARI-16014
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> is_linux_container_executor = (yarn_nodemanager_container_executor_class == 
> 'org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor')
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
>  line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found 
> in configurations dictionary!")
> Fail: Configuration parameter 'yarn-site' was not found in configurations 
> dictionary!
> INFO 2016-04-21 12:14:21,327 PythonReflectiveExecutor.py:65 - Reflective 
> command failed with exception:
> Traceback (most recent call last):
>   File 
> "/usr/lib/python2.6/site-packages/ambari_agent/PythonReflectiveExecutor.py", 
> line 57, in run_file
> imp.load_source('__main__', script)
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
>  line 149, in 
> ApplicationTimelineServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 251, in execute
> self.save_component_version_to_structured_out()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 166, in save_component_version_to_structured_out
> component_name = self.get_component_name()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 152, in get_component_name
> stack_to_component = self.get_stack_to_component()
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
>  line 67, in get_stack_to_component
> import params
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params.py",
>  line 28, in 
> from params_linux import *
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py",
>  line 144, in 
> is_linux_container_executor = 
> (yarn_nodemanager_container_executor_class == 
> 'org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor')
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
>  line 73, in __getattr__
> raise Fail("Configuration parameter '" + self.name + "' was not found 
> in configurations dictionary!")
> Fail: Configuration parameter 'yarn-site' was not found in configurations 
> dictionary!
> INFO 2016-04-21 12:14:21,390 PythonReflectiveExecutor.py:65 - Reflective 
> command failed with exception:
> Traceback (most recent call last):
>   File 
> "/usr/lib/python2.6/site-packages/ambari_agent/PythonReflectiveExecutor.py", 
> line 57, in run_file
> imp.load_source('__main__', script)
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
>  line 280, in 
> Resourcemanager().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 251, in execute
> self.save_component_version_to_structured_out()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 166, in save_component_version_to_structured_out
> component_name = self.get_component_name()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 152, in get_component_name
> stack_to_component = self.get_stack_to_component()
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
>  line 102, in get_stack_to_component
> import params
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params.py",
>  line 28, in 
> from params_linux import *
>   File 
> "/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py",
>  line 144, in 
> is_linux_container_executor = 
>

Review Request 46493: ambari-agent.log gets flooded up with exceptions from status commands


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46493/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-16014
https://issues.apache.org/jira/browse/AMBARI-16014


Repository: ambari


Description
---

is_linux_container_executor = (yarn_nodemanager_container_executor_class == 
'org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor')
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
 line 73, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found 
in configurations dictionary!")
Fail: Configuration parameter 'yarn-site' was not found in configurations 
dictionary!
INFO 2016-04-21 12:14:21,327 PythonReflectiveExecutor.py:65 - Reflective 
command failed with exception:
Traceback (most recent call last):
  File 
"/usr/lib/python2.6/site-packages/ambari_agent/PythonReflectiveExecutor.py", 
line 57, in run_file
imp.load_source('__main__', script)
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
 line 149, in 
ApplicationTimelineServer().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 251, in execute
self.save_component_version_to_structured_out()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 166, in save_component_version_to_structured_out
component_name = self.get_component_name()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 152, in get_component_name
stack_to_component = self.get_stack_to_component()
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/application_timeline_server.py",
 line 67, in get_stack_to_component
import params
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params.py",
 line 28, in 
from params_linux import *
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py",
 line 144, in 
is_linux_container_executor = 
(yarn_nodemanager_container_executor_class == 
'org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor')
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
 line 73, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found 
in configurations dictionary!")
Fail: Configuration parameter 'yarn-site' was not found in configurations 
dictionary!
INFO 2016-04-21 12:14:21,390 PythonReflectiveExecutor.py:65 - Reflective 
command failed with exception:
Traceback (most recent call last):
  File 
"/usr/lib/python2.6/site-packages/ambari_agent/PythonReflectiveExecutor.py", 
line 57, in run_file
imp.load_source('__main__', script)
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
 line 280, in 
Resourcemanager().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 251, in execute
self.save_component_version_to_structured_out()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 166, in save_component_version_to_structured_out
component_name = self.get_component_name()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 152, in get_component_name
stack_to_component = self.get_stack_to_component()
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/resourcemanager.py",
 line 102, in get_stack_to_component
import params
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params.py",
 line 28, in 
from params_linux import *
  File 
"/var/lib/ambari-agent/cache/common-services/YARN/2.1.0.2.0/package/scripts/params_linux.py",
 line 144, in 
is_linux_container_executor = 
(yarn_nodemanager_container_executor_class == 
'org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor')
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py",
 line 73, in __getattr__
raise Fail("Configuration parameter '" + self.name + "' was not found 
in configurations dictionary!")
Fail: Configuration parameter 'yarn-site' was not found in configurations 
dictionary!
INFO 2016-04-21 12:14:21,429 PythonReflectiveExecutor.py:65 - Reflective 
command failed with exception:
Traceback (most recent call last):
  File

Review Request 46492: Remove example AMS alert definitions from HDFS/alert.json

2016-04-21 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46492/
---

Review request for Ambari, Dmitro Lisnichenko and Sumit Mohanty.


Bugs: AMBARI-16011
https://issues.apache.org/jira/browse/AMBARI-16011


Repository: ambari


Description
---

Remove example AMS alert definitions from HDFS/alert.json


Diffs
-

  ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
21588f2 

Diff: https://reviews.apache.org/r/46492/diff/


Testing
---

unit tests passed. 

only alerts.json changed


Thanks,

Dmytro Sen

Re: Review Request 46492: Remove example AMS alert definitions from HDFS/alert.json


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46492/#review129894
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On April 21, 2016, 4:58 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46492/
> ---
> 
> (Updated April 21, 2016, 4:58 p.m.)
> 
> 
> Review request for Ambari, Dmitro Lisnichenko and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16011
> https://issues.apache.org/jira/browse/AMBARI-16011
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove example AMS alert definitions from HDFS/alert.json
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/alerts.json 
> 21588f2 
> 
> Diff: https://reviews.apache.org/r/46492/diff/
> 
> 
> Testing
> ---
> 
> unit tests passed. 
> 
> only alerts.json changed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 46490: Update HDFS HA alerts definitions during upgrade


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46490/#review129893
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On April 21, 2016, 4:51 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46490/
> ---
> 
> (Updated April 21, 2016, 4:51 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-16010
> https://issues.apache.org/jira/browse/AMBARI-16010
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During upgrade hdfs-site/dfs.nameservices should be replaced with 
> hdfs-site/dfs.internal.nameservices for all HDFS alerts.
> hdfs-site/dfs.nameservices might contain external nameservices
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
>  d4d47bf 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
>  c091262 
> 
> Diff: https://reviews.apache.org/r/46490/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 46490: Update HDFS HA alerts definitions during upgrade

2016-04-21 Thread Dmytro Sen


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46490/
---

Review request for Ambari, Andrew Onischuk and Sumit Mohanty.


Bugs: AMBARI-16010
https://issues.apache.org/jira/browse/AMBARI-16010


Repository: ambari


Description
---

During upgrade hdfs-site/dfs.nameservices should be replaced with 
hdfs-site/dfs.internal.nameservices for all HDFS alerts.
hdfs-site/dfs.nameservices might contain external nameservices


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog240.java
 d4d47bf 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog240Test.java
 c091262 

Diff: https://reviews.apache.org/r/46490/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 46484: Enhance Health Check for the Cluster before upgrading.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46484/#review129891
---




ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
(line 74)


CLUSTER type checks typically don't have hosts on getFailedOn() (at least, 
that's not the intent).  SERVICE types get service names, HOST types get host 
names.


- Nate Cole


On April 21, 2016, 7:14 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46484/
> ---
> 
> (Updated April 21, 2016, 7:14 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-16005
> https://issues.apache.org/jira/browse/AMBARI-16005
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhance Health Check for the Cluster before upgrading.
> 
> * check for alerts (perhaps don't start RU/EU if any blocker alerts).
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  3e957b1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HealthCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46484/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46484: Enhance Health Check for the Cluster before upgrading.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46484/#review129890
---




ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
(lines 33 - 34)


I've usually seen imports ordered like so:
import static first
import java...
import org...
import com...



ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
(lines 71 - 82)


A check like this can use some formatted output.  We can add any object to 
the getFailedDetail() that will get output as json.  This give the UI the 
option to do something interesting with text.

See for example ConfigurationMergeCheck.


- Nate Cole


On April 21, 2016, 7:14 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46484/
> ---
> 
> (Updated April 21, 2016, 7:14 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-16005
> https://issues.apache.org/jira/browse/AMBARI-16005
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhance Health Check for the Cluster before upgrading.
> 
> * check for alerts (perhaps don't start RU/EU if any blocker alerts).
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  3e957b1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HealthCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46484/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46448: Restart button took about 5 minutes to appear after a config change for zookeeper


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46448/
---

(Updated April 21, 2016, 4:24 p.m.)


Review request for Ambari, Jonathan Hurley and Vitalyi Brodetskyi.


Changes
---

Fixed comments


Bugs: AMBARI-15994
https://issues.apache.org/jira/browse/AMBARI-15994


Repository: ambari


Description
---

STR:
-Change zookeeper property "Length of single tick" from UI

This test changed the property Length of single tick from 2000 to 2001 for 
Zookeeper. This should have restart indicators for Zookeeper, Storm and Kafka. 
As seen from the UI, all the services had restart icons appearing late.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 5793b70 
  ambari-server/src/main/java/org/apache/ambari/server/state/ConfigHelper.java 
8d9a879 
  ambari-server/src/main/java/org/apache/ambari/server/state/Host.java fce2f91 
  ambari-server/src/main/java/org/apache/ambari/server/state/host/HostImpl.java 
0514720 
  
ambari-server/src/test/java/org/apache/ambari/server/state/ConfigHelperTest.java
 f887c3a 

Diff: https://reviews.apache.org/r/46448/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 46484: Enhance Health Check for the Cluster before upgrading.

2016-04-21 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46484/#review129888
---




ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
 (line 51)


Grammar: Cluster Health

I don't think we want to indicate here that it needs to be re-run; running 
it again won't really help unless action is taken first. Let's keep this as a 
general description.



ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
 (line 54)


The following issues have been detected on this cluster and should be 
addressed before upgrading:



ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
(line 43)


Doc.



ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
(line 67)


This needs to be "current" alerts:

`AlertsDAO.findCurrentByCluster()`

This could potentially pull back 100,000 entries.



ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
(lines 72 - 75)


Let's do something better here; maybe add some formatting. Also - the text 
of the alert isn't necessary, just the definition label:

": : "

The host may be null, so you'll need to account for that. Also, let's use a 
sorted set so that all CRITICALs and grouped and all WARNINGs are grouped.


- Jonathan Hurley


On April 21, 2016, 7:14 a.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46484/
> ---
> 
> (Updated April 21, 2016, 7:14 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
> Hurley, and Nate Cole.
> 
> 
> Bugs: AMBARI-16005
> https://issues.apache.org/jira/browse/AMBARI-16005
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Enhance Health Check for the Cluster before upgrading.
> 
> * check for alerts (perhaps don't start RU/EU if any blocker alerts).
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
>  3e957b1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
> PRE-CREATION 
>   
> ambari-server/src/test/java/org/apache/ambari/server/checks/HealthCheckTest.java
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46484/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 46487: Add sles 12 as the os family for HDP 2.5.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46487/#review129889
---


Ship it!




Ship It!

- Andrew Onischuk


On April 21, 2016, 1:06 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46487/
> ---
> 
> (Updated April 21, 2016, 1:06 p.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk and Dmytro Sen.
> 
> 
> Bugs: AMBARI-16007
> https://issues.apache.org/jira/browse/AMBARI-16007
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add sles 12 as the os family for HDP 2.5
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.5/repos/repoinfo.xml acf1c07 
> 
> Diff: https://reviews.apache.org/r/46487/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 46487: Add sles 12 as the os family for HDP 2.5.

2016-04-21 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46487/
---

Review request for Ambari, Andrew Onischuk and Dmytro Sen.


Bugs: AMBARI-16007
https://issues.apache.org/jira/browse/AMBARI-16007


Repository: ambari


Description
---

Add sles 12 as the os family for HDP 2.5


Diffs
-

  ambari-server/src/main/resources/stacks/HDP/2.5/repos/repoinfo.xml acf1c07 

Diff: https://reviews.apache.org/r/46487/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 46016: AMBARI-15752 Ambari support for additional config params for Ranger KMS to support HSM

2016-04-21 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46016/#review129887
---


Ship it!




Ship It!

- Jonathan Hurley


On April 20, 2016, 7:14 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46016/
> ---
> 
> (Updated April 20, 2016, 7:14 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jaimin Jetly, Jonathan 
> Hurley, Jayush Luniya, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-15752
> https://issues.apache.org/jira/browse/AMBARI-15752
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Add HSM support configuration properties for Ranger KMS in hdp-2.5
> 
> 
> Diffs
> -
> 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/constants.py
>  658c56f 
>   
> ambari-common/src/main/python/resource_management/libraries/functions/stack_features.py
>  cf56acf 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
>  f63d660 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  4bdbd80 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6/properties/stack_features.json
>  4cead39 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/configuration/dbks-site.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/configuration/kms-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/metainfo.xml
>  02fb6aa 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/RANGER_KMS/themes/theme_version_2.json
>  PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/46016/diff/
> 
> 
> Testing
> ---
> 
> Installed Ranger KMS centos6.
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 46486: Add logging to check_host.py

2016-04-21 Thread Vitalyi Brodetskyi


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46486/#review129886
---


Ship it!




Ship It!

- Vitalyi Brodetskyi


On April 21, 2016, 11:52 a.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46486/
> ---
> 
> (Updated April 21, 2016, 11:52 a.m.)
> 
> 
> Review request for Ambari and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-16006
> https://issues.apache.org/jira/browse/AMBARI-16006
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Sometimes host check after host registration lasted too long (more than 720
> sec.). I's affect some system tests. I can't reproduce this, so it would be
> good to add logging to this operation.
> 
> Currently, all the host check logs available are
> 
> 
> 
> 
> [root@c6404 data]# cat /var/lib/ambari-agent/data/output-3.txt
> Last Agent Env check started.
> Last Agent Env check completed successfully.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/custom_actions/scripts/check_host.py 
> 85ccef1 
>   ambari-server/src/test/python/custom_actions/TestCheckHost.py 8bc1774 
> 
> Diff: https://reviews.apache.org/r/46486/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 46434: DataNode and RegionServer during upgrade are reported as "failed" incorrectly

2016-04-21 Thread Daniel Gergely

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46434/
---

(Updated ápr. 21, 2016, 12:36 du)

Review request for Ambari, Alejandro Fernandez, Miklos Gergely, Oliver Szabo,
Sandor Magyari, and Sebastian Toader.

Changes
---

Review fix

Bugs: AMBARI-15991
https://issues.apache.org/jira/browse/AMBARI-15991

Repository: ambari

Description
---

If upgrade process takes longer than expected, DataNode and RegionServer is
reported as failed. It happens because it needs more time to finish update.

The fix for RegionServer checks if the process is running and if it is so, then
it is not considered as a failure.
For DataNode the process is also checked and if it is running then check is
repeated 2 times with 5 minutes wait. I had a limitation here, python scripts
are allowed to run for 20 minutes by default and this checking takes 16 mins (2
minutes initial check, 5 minutes sleep if there is a failure, 2 minutes
regaular check, 5 minutes sleep, 2 minutes final check).
If more time is needed, then default value of *server.task.timeout* and number
of repetition in 5 minutes check should be increased.

Diffs (updated)
-

ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
01a8156

ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
8f36001

ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
7ad9f39
ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 78b8171

Diff: https://reviews.apache.org/r/46434/diff/

Testing
---

I did manual testing on this:
For RegionServer the process check is tested.
For DataNodes I made an intentional exception to see if it keeps waiting. (this
is how I ran into the 20 minutes server task timeout)

--
Total run:970
Total errors:0
Total failures:0
OK

Thanks,

Daniel Gergely

Re: Review Request 46434: DataNode and RegionServer during upgrade are reported as "failed" incorrectly

2016-04-21 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46434/#review129882
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
 (line 59)


If the process is running thna the second part after the `or` of the 
condition will never be evaluated.

If the process is down than the second part of the condition will always 
fail, won't it?

So this `if` statement can be reduced to just ```if not 
process_is_running:```

Checking only if the process is running might noit be a robust solution. 
Shouldn't we have a similar approach as for DataNode?



ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
 (lines 77 - 90)


Retry logic is already available through annotations. Please consider if 
here `@safe_retry` or `@retry` can be used. See as an example RangerAdminV2 for 
how this annotations are used.


- Sebastian Toader


On April 21, 2016, 10:33 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46434/
> ---
> 
> (Updated April 21, 2016, 10:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Miklos Gergely, Oliver Szabo, 
> Sandor Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15991
> https://issues.apache.org/jira/browse/AMBARI-15991
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If upgrade process takes longer than expected, DataNode and RegionServer is 
> reported as failed. It happens because it needs more time to finish update.
> 
> The fix for RegionServer checks if the process is running and if it is so, 
> then it is not considered as a failure.
> For DataNode the process is also checked and if it is running then check is 
> repeated 2 times with 5 minutes wait. I had a limitation here, python scripts 
> are allowed to run for 20 minutes by default and this checking takes 16 mins 
> (2 minutes initial check, 5 minutes sleep if there is a failure, 2 minutes 
> regaular check, 5 minutes sleep, 2 minutes final check).
> If more time is needed, then default value of *server.task.timeout* and 
> number of repetition in 5 minutes check should be increased.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
>  01a8156 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
>  8f36001 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  7ad9f39 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 78b8171 
> 
> Diff: https://reviews.apache.org/r/46434/diff/
> 
> 
> Testing
> ---
> 
> I did manual testing on this:
> For RegionServer the process check is tested.
> For DataNodes I made an intentional exception to see if it keeps waiting. 
> (this is how I ran into the 20 minutes server task timeout)
> 
> --
> Total run:970
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 46486: Add logging to check_host.py


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46486/
---

(Updated April 21, 2016, 11:52 a.m.)


Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-16006
https://issues.apache.org/jira/browse/AMBARI-16006


Repository: ambari


Description
---

Sometimes host check after host registration lasted too long (more than 720
sec.). I's affect some system tests. I can't reproduce this, so it would be
good to add logging to this operation.

Currently, all the host check logs available are




[root@c6404 data]# cat /var/lib/ambari-agent/data/output-3.txt
Last Agent Env check started.
Last Agent Env check completed successfully.


Diffs (updated)
-

  ambari-server/src/main/resources/custom_actions/scripts/check_host.py 85ccef1 
  ambari-server/src/test/python/custom_actions/TestCheckHost.py 8bc1774 

Diff: https://reviews.apache.org/r/46486/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 46434: DataNode and RegionServer during upgrade are reported as "failed" incorrectly

2016-04-21 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46434/#review129880
---


Ship it!




Ship It!

- Oliver Szabo


On April 21, 2016, 8:33 a.m., Daniel Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46434/
> ---
> 
> (Updated April 21, 2016, 8:33 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Miklos Gergely, Oliver Szabo, 
> Sandor Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-15991
> https://issues.apache.org/jira/browse/AMBARI-15991
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If upgrade process takes longer than expected, DataNode and RegionServer is 
> reported as failed. It happens because it needs more time to finish update.
> 
> The fix for RegionServer checks if the process is running and if it is so, 
> then it is not considered as a failure.
> For DataNode the process is also checked and if it is running then check is 
> repeated 2 times with 5 minutes wait. I had a limitation here, python scripts 
> are allowed to run for 20 minutes by default and this checking takes 16 mins 
> (2 minutes initial check, 5 minutes sleep if there is a failure, 2 minutes 
> regaular check, 5 minutes sleep, 2 minutes final check).
> If more time is needed, then default value of *server.task.timeout* and 
> number of repetition in 5 minutes check should be increased.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/package/scripts/upgrade.py
>  01a8156 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/datanode_upgrade.py
>  8f36001 
>   
> ambari-server/src/main/resources/common-services/HDFS/2.1.0.2.0/package/scripts/params_linux.py
>  7ad9f39 
>   ambari-server/src/test/python/stacks/2.0.6/HDFS/test_datanode.py 78b8171 
> 
> Diff: https://reviews.apache.org/r/46434/diff/
> 
> 
> Testing
> ---
> 
> I did manual testing on this:
> For RegionServer the process check is tested.
> For DataNodes I made an intentional exception to see if it keeps waiting. 
> (this is how I ran into the 20 minutes server task timeout)
> 
> --
> Total run:970
> Total errors:0
> Total failures:0
> OK
> 
> 
> Thanks,
> 
> Daniel Gergely
> 
>

Re: Review Request 46405: AMBARI-15519 Add Service Wizard with nodes in the maintenance mode

2016-04-21 Thread Alexandr Antonenko



> On April 21, 2016, 11:23 a.m., Alexandr Antonenko wrote:
> > ambari-web/app/mixins/wizard/assign_master_components.js, line 978
> > 
> >
> > This check can be simplified, after you added: 
> > "(this.get('hosts').filterProperty('host_name', 
> > selectedHost).filterProperty('maintenance_state', 'OFF').length > 0)" 
> > 
> > there is no need in :
> > 
> > "this.get('hosts').mapProperty('host_name').contains(selectedHost)"
> > 
> > 
> > This is because : "this.get('hosts').filterProperty('host_name', 
> > selectedHost)" will do the same thing as 
> > this.get('hosts').mapProperty('host_name').contains(selectedHost)

Sorry for bringing this up after review, saw this only after looking on 
committed changes. This can have bad effect if we have huge cluster with 
hundreds of hosts. So it's better to remove unnecessary check.


- Alexandr


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46405/#review129878
---


On April 19, 2016, 8:36 p.m., Zhe (Joe) Wang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/46405/
> ---
> 
> (Updated April 19, 2016, 8:36 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko, Jaimin Jetly, Oleg 
> Nechiporenko, Richard Zang, Srimanth Gunturi, Xi Wang, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-15519
> https://issues.apache.org/jira/browse/AMBARI-15519
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> On the cluster where some nodes are in the maintenance mode Add Service 
> Wizard doesn't track this.
> So, user may select host in the maintenance mode to install some master 
> component. And this component won't be installed (only record in the DB will 
> be created for it).
> Another situation:
> there is some cluster where all nodes are in the maintenance mode. In this 
> case ASW is almost useless. Because all new components won't be installed 
> while it works.
> 
> ASW should track hosts in the maintenance mode and warn user about adding 
> components on them (Steps "Assign Masters", "Assign Slaves and Clients").
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard.js adce5ff 
>   ambari-web/app/mixins/wizard/assign_master_components.js cd8ef14 
>   ambari-web/app/utils/ajax/ajax.js a1d3f5f 
>   ambari-web/test/controllers/main/service/add_controller_test.js 90670a8 
>   ambari-web/test/controllers/wizard/step5_test.js 935d76d 
> 
> Diff: https://reviews.apache.org/r/46405/diff/
> 
> 
> Testing
> ---
> 
> Modified unit test.
> Local ambari-web test passed.
> 25680 tests complete (24 seconds)
> 154 tests pending
> Manual testing done.
> 
> 
> Thanks,
> 
> Zhe (Joe) Wang
> 
>

Review Request 46486: Add logging to check_host.py


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46486/
---

Review request for Ambari and Vitalyi Brodetskyi.


Bugs: AMBARI-16006
https://issues.apache.org/jira/browse/AMBARI-16006


Repository: ambari


Description
---

Sometimes host check after host registration lasted too long (more than 720
sec.). I's affect some system tests. I can't reproduce this, so it would be
good to add logging to this operation.

Currently, all the host check logs available are




[root@c6404 data]# cat /var/lib/ambari-agent/data/output-3.txt
Last Agent Env check started.
Last Agent Env check completed successfully.


Diffs
-

  ambari-server/src/main/resources/custom_actions/scripts/check_host.py 85ccef1 
  ambari-server/src/test/python/custom_actions/TestCheckHost.py 8bc1774 

Diff: https://reviews.apache.org/r/46486/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 46484: Enhance Health Check for the Cluster before upgrading.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46484/
---

Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Jonathan 
Hurley, and Nate Cole.


Bugs: AMBARI-16005
https://issues.apache.org/jira/browse/AMBARI-16005


Repository: ambari


Description
---

Enhance Health Check for the Cluster before upgrading.

* check for alerts (perhaps don't start RU/EU if any blocker alerts).


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/checks/CheckDescription.java
 3e957b1 
  ambari-server/src/main/java/org/apache/ambari/server/checks/HealthCheck.java 
PRE-CREATION 
  
ambari-server/src/test/java/org/apache/ambari/server/checks/HealthCheckTest.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/46484/diff/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 45998: AMBARI-15792: HIVE service_check doesn't work properly

2016-04-21 Thread Masahiro Tanaka



> On 4月 21, 2016, 12:25 a.m., Masahiro Tanaka wrote:
> > Thanks for the review Nate and Andrew!
> > Could you commit this?
> 
> Andrew Onischuk wrote:
> before we can commit we need an apache jira for this change, and +1 from 
> Hadoop QA. 
> 
> Do you have one? If yes specify its id in "Bugs" field here.
> 
> Masahiro Tanaka wrote:
> I forgot to add the jira number... I'm sorry.
> 
> Andrew Onischuk wrote:
> no worries
> 
> Andrew Onischuk wrote:
> Thanks for the contribution! Committed the patch to trunk. Please feel 
> free to close this review.

thank you!


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45998/#review129845
---


On 4月 21, 2016, 10:30 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45998/
> ---
> 
> (Updated 4月 21, 2016, 10:30 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, and Nate Cole.
> 
> 
> Bugs: AMBARI-15792
> https://issues.apache.org/jira/browse/AMBARI-15792
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In templetonSmoke.sh, there are
> 1) unnecessary `exit 0`
> 2) lack of redirect `>` command
> 3) unassigned variable
> 
> we should correct them all to check the HIVE service properly.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh
>  dd00e02 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service_check.py
>  9965bfa9 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 
> 6c4dc00 
> 
> Diff: https://reviews.apache.org/r/45998/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test and manual test with HDP2.4 and Ambari 2.2.1.0
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 45998: AMBARI-15792: HIVE service_check doesn't work properly



> On April 21, 2016, 12:25 a.m., Masahiro Tanaka wrote:
> > Thanks for the review Nate and Andrew!
> > Could you commit this?
> 
> Andrew Onischuk wrote:
> before we can commit we need an apache jira for this change, and +1 from 
> Hadoop QA. 
> 
> Do you have one? If yes specify its id in "Bugs" field here.
> 
> Masahiro Tanaka wrote:
> I forgot to add the jira number... I'm sorry.
> 
> Andrew Onischuk wrote:
> no worries

Thanks for the contribution! Committed the patch to trunk. Please feel free to 
close this review.


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45998/#review129845
---


On April 21, 2016, 10:30 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45998/
> ---
> 
> (Updated April 21, 2016, 10:30 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, and Nate Cole.
> 
> 
> Bugs: AMBARI-15792
> https://issues.apache.org/jira/browse/AMBARI-15792
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In templetonSmoke.sh, there are
> 1) unnecessary `exit 0`
> 2) lack of redirect `>` command
> 3) unassigned variable
> 
> we should correct them all to check the HIVE service properly.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh
>  dd00e02 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service_check.py
>  9965bfa9 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 
> 6c4dc00 
> 
> Diff: https://reviews.apache.org/r/45998/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test and manual test with HDP2.4 and Ambari 2.2.1.0
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 45998: AMBARI-15792: HIVE service_check doesn't work properly

2016-04-21 Thread Masahiro Tanaka



> On 4月 21, 2016, 12:25 a.m., Masahiro Tanaka wrote:
> > Thanks for the review Nate and Andrew!
> > Could you commit this?
> 
> Andrew Onischuk wrote:
> before we can commit we need an apache jira for this change, and +1 from 
> Hadoop QA. 
> 
> Do you have one? If yes specify its id in "Bugs" field here.

I forgot to add the jira number... I'm sorry.


- Masahiro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45998/#review129845
---


On 4月 21, 2016, 10:30 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45998/
> ---
> 
> (Updated 4月 21, 2016, 10:30 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, and Nate Cole.
> 
> 
> Bugs: AMBARI-15792
> https://issues.apache.org/jira/browse/AMBARI-15792
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In templetonSmoke.sh, there are
> 1) unnecessary `exit 0`
> 2) lack of redirect `>` command
> 3) unassigned variable
> 
> we should correct them all to check the HIVE service properly.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh
>  dd00e02 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service_check.py
>  9965bfa9 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 
> 6c4dc00 
> 
> Diff: https://reviews.apache.org/r/45998/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test and manual test with HDP2.4 and Ambari 2.2.1.0
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 45998: AMBARI-15792: HIVE service_check doesn't work properly



> On April 21, 2016, 12:25 a.m., Masahiro Tanaka wrote:
> > Thanks for the review Nate and Andrew!
> > Could you commit this?
> 
> Andrew Onischuk wrote:
> before we can commit we need an apache jira for this change, and +1 from 
> Hadoop QA. 
> 
> Do you have one? If yes specify its id in "Bugs" field here.
> 
> Masahiro Tanaka wrote:
> I forgot to add the jira number... I'm sorry.

no worries


- Andrew


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/45998/#review129845
---


On April 21, 2016, 10:30 a.m., Masahiro Tanaka wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/45998/
> ---
> 
> (Updated April 21, 2016, 10:30 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Jaimin Jetly, and Nate Cole.
> 
> 
> Bugs: AMBARI-15792
> https://issues.apache.org/jira/browse/AMBARI-15792
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In templetonSmoke.sh, there are
> 1) unnecessary `exit 0`
> 2) lack of redirect `>` command
> 3) unassigned variable
> 
> we should correct them all to check the HIVE service properly.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/files/templetonSmoke.sh
>  dd00e02 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/webhcat_service_check.py
>  9965bfa9 
>   ambari-server/src/test/python/stacks/2.0.6/HIVE/test_hive_service_check.py 
> 6c4dc00 
> 
> Diff: https://reviews.apache.org/r/45998/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test and manual test with HDP2.4 and Ambari 2.2.1.0
> 
> 
> Thanks,
> 
> Masahiro Tanaka
> 
>

Re: Review Request 45998: HIVE service_check doesn't work properly