Re: Review Request 49484: Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition (2.4/2.5 branch)

2016-10-19 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49484/#review153348
---


Ship it!




Ship It!

- Sumit Mohanty


On Oct. 20, 2016, 3:12 a.m., Saisai Shao wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49484/
> ---
> 
> (Updated Oct. 20, 2016, 3:12 a.m.)
> 
> 
> Review request for Ambari, Jayush Luniya and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17489
> https://issues.apache.org/jira/browse/AMBARI-17489
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Remove spark.yarn.max.executor.failures configuration in Spark Ambari 
> definition
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
>  6465b72 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> cdbe319 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> 9045443 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
>  3478603 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> 826cf16 
> 
> Diff: https://reviews.apache.org/r/49484/diff/
> 
> 
> Testing
> ---
> 
> Local verification.
> 
> 
> Thanks,
> 
> Saisai Shao
> 
>

Re: Review Request 49484: Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition (2.4/2.5 branch)

2016-10-19 Thread Saisai Shao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49484/
---

(Updated Oct. 20, 2016, 3:12 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Summary (updated)
-

Remove spark.yarn.max.executor.failures configuration in Spark Ambari 
definition (2.4/2.5 branch)


Bugs: AMBARI-17489
https://issues.apache.org/jira/browse/AMBARI-17489


Repository: ambari


Description
---

Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition


Diffs
-

  
ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
 6465b72 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
cdbe319 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
9045443 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 3478603 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
826cf16 

Diff: https://reviews.apache.org/r/49484/diff/


Testing
---

Local verification.


Thanks,

Saisai Shao

Re: Review Request 49484: Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition

2016-10-19 Thread Saisai Shao 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49484/
---

(Updated Oct. 20, 2016, 3:07 a.m.)


Review request for Ambari, Jayush Luniya and Sumit Mohanty.


Bugs: AMBARI-17489
https://issues.apache.org/jira/browse/AMBARI-17489


Repository: ambari


Description
---

Remove spark.yarn.max.executor.failures configuration in Spark Ambari definition


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/SPARK/1.6.0/configuration/spark-defaults.xml
 6465b72 
  ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
cdbe319 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
9045443 
  
ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.5.xml
 3478603 
  ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
826cf16 

Diff: https://reviews.apache.org/r/49484/diff/


Testing
---

Local verification.


Thanks,

Saisai Shao

Re: Review Request 52842: AMBARI-18593 : Provide ability to use downsampling function on certain metrics like client side topN

2016-10-19 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52842/#review153330
---




ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
 (line 29)


Add come javadoc.



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
 (line 52)


formatting.



ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
 (line 96)


The only problem I see here is we loose context of what topN funtion was 
applied if we copy same value everywhere. What about only for the column = 
function and rest as NULL?



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
 (line 686)


1. Why %.user=% vs op=% ?
2. I was expecting op=% vs op=_%, is that special syntx?


- Sid Wagle


On Oct. 18, 2016, 6:35 p.m., Aravindan Vijayan wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52842/
> ---
> 
> (Updated Oct. 18, 2016, 6:35 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Sen, Sumit Mohanty, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18593
> https://issues.apache.org/jira/browse/AMBARI-18593
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDFS exposes top user activity broken down by operations in jmx (nntop).
> These metrics should be captured in AMS and exposed in Grafana's HDFS 
> dashboards.
> 
> Downsampling should likely be a function like MIN, MAX, AVG, SUM of 
> underlying timeseries specified from the client.
> 
> 
> Diffs
> -
> 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/AbstractTimelineAggregator.java
>  ba7807b 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/CustomDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerUtils.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/TopNDownSampler.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricClusterAggregator.java
>  c056d79 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/v2/TimelineMetricHostAggregator.java
>  118c695 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/PhoenixTransactSQL.java
>  177e444 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/main/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/query/TopNCondition.java
>  f7060e0 
>   
> ambari-metrics/ambari-metrics-timelineservice/src/test/java/org/apache/hadoop/yarn/server/applicationhistoryservice/metrics/timeline/aggregators/DownSamplerTest.java
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/ams-site.xml
>  b6e5da9 
> 
> Diff: https://reviews.apache.org/r/52842/diff/
> 
> 
> Testing
> ---
> 
> Manually tested.
> Unit tests added.
> mvn clean test on ambari-metrics pending.
> 
> 
> Thanks,
> 
> Aravindan Vijayan
> 
>

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

2016-10-19 Thread Amarnath reddy pappu 


> On Oct. 18, 2016, 9:15 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java,
> >  line 2428
> > 
> >
> > nit:  "Indicates the delay, in milliseconds, for the log4j monitor to 
> > check for changes"
> > 
> > Something like that.  We need to be very detailed so users/admins can 
> > understand exactly what a config knob is for.

implemented


> On Oct. 18, 2016, 9:15 p.m., Nate Cole wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java,
> >  lines 963-964
> > 
> >
> > These can be collapsed:
> > LOG.error("Exception in setting log4j monitor delay of {} for {}", 
> > monitorDelay, log4jpath, e);

implemented


- Amarnath reddy


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review153176
---


On Oct. 19, 2016, 10:51 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 19, 2016, 10:51 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

2016-10-19 Thread Amarnath reddy pappu 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/
---

(Updated Oct. 19, 2016, 10:51 p.m.)


Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, and 
Sid Wagle.


Changes
---

Hi,

Implemented most of the review comments.

Have attached final code before shipping it.

Regards,
Amar


Bugs: AMBARI-18592
https://issues.apache.org/jira/browse/AMBARI-18592


Repository: ambari


Description
---

Ability to enable the DEBUG logs in Ambari-server without server restart

1. PropertyConfigurator would check log4j.properties update every 5mins. if 
there is any change in the file/timestamp it will reload the log4j 
configurations.
2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
3. Have modified only for ambari-server
4.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 378db18 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 c05840e 

Diff: https://reviews.apache.org/r/52839/diff/


Testing
---

1. Change the log4j.properties from INFO to DEBUG
[Expected Result] server should start logging the DEBUG logs with in 5mins of 
time.
[Resutl] PASS

2. Change the log4j.properties from DEBUG to INFO
[Expected Result] server should reduce the logging to INFO mode with in 5mins 
of time.
[Resutl] PASS


Thanks,

Amarnath reddy pappu

Re: Review Request 53030: Ambari upgrade failed while running 'Alter Table blueprint' - blueprint_name column

2016-10-19 Thread Sumit Mohanty 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53030/#review153306
---


Ship it!




Ship It!

- Sumit Mohanty


On Oct. 19, 2016, 7:17 p.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/53030/
> ---
> 
> (Updated Oct. 19, 2016, 7:17 p.m.)
> 
> 
> Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-18640
> https://issues.apache.org/jira/browse/AMBARI-18640
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Observed errors in today's run during Ambari upgrade from 2.2.1.1 to 
> 2.4.2.0-36
> ambari-server --hash
> c6da6776f029f15d3a7d6009697371eee4e5f4c5
> 
> Ambari DB - MySQL; Secure HDP-2.4.0.0 cluster deployed via UI
> 
> *Upgrade Log indicates below:*
> {code}
> 18 Oct 2016 14:17:59,115  INFO [main] DBAccessorImpl:824 - Executing query: 
> ALTER TABLE users  MODIFY user_name VARCHAR(100)
> 18 Oct 2016 14:17:59,154  INFO [main] DBAccessorImpl:824 - Executing query: 
> ALTER TABLE users  MODIFY user_name VARCHAR(100) NOT NULL
> 18 Oct 2016 14:17:59,191  INFO [main] DBAccessorImpl:824 - Executing query: 
> ALTER TABLE host_role_command  MODIFY role VARCHAR(100)
> 18 Oct 2016 14:17:59,428  INFO [main] DBAccessorImpl:824 - Executing query: 
> ALTER TABLE host_role_command  MODIFY status VARCHAR(100)
> 18 Oct 2016 14:17:59,656  INFO [main] DBAccessorImpl:824 - Executing query: 
> ALTER TABLE blueprint  MODIFY blueprint_name VARCHAR(100)
> 18 Oct 2016 14:17:59,678 ERROR [main] DBAccessorImpl:830 - Error executing 
> query: ALTER TABLE blueprint  MODIFY blueprint_name VARCHAR(100)
> java.sql.SQLException: Cannot change column 'blueprint_name': used in a 
> foreign key constraint 'FK_blueprint_setting_name' of table 
> 'ambaricustom.blueprint_setting'
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:996)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2484)
> at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:848)
> at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:742)
> at 
> org.apache.ambari.server.orm.DBAccessorImpl.executeQuery(DBAccessorImpl.java:827)
> at 
> org.apache.ambari.server.orm.DBAccessorImpl.executeQuery(DBAccessorImpl.java:819)
> at 
> org.apache.ambari.server.orm.DBAccessorImpl.alterColumn(DBAccessorImpl.java:610)
> at 
> org.apache.ambari.server.upgrade.UpgradeCatalog242.updateTablesForMysql(UpgradeCatalog242.java:120)
> at 
> org.apache.ambari.server.upgrade.UpgradeCatalog242.executeDDLUpdates(UpgradeCatalog242.java:95)
> at 
> org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeSchema(AbstractUpgradeCatalog.java:889)
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeUpgrade(SchemaUpgradeHelper.java:206)
> at 
> org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:349)
> 18 Oct 2016 14:17:59,680 ERROR [main] SchemaUpgradeHelper:208 - Upgrade 
> failed.
> java.sql.SQLException: Cannot change column 'blueprint_name': used in a 
> foreign key constraint 'FK_blueprint_setting_name' of table 
> 'ambaricustom.blueprint_setting'
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:996)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
> at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
> at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
> at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
> at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2484)
> at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:848)
> 
> {code}
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog242.java
>  6fa3e68 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog242Test.java
>  81f8451 
> 
> Diff: https://reviews.apache.org/r/53030/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 53030: Ambari upgrade failed while running 'Alter Table blueprint' - blueprint_name column

2016-10-19 Thread Vitalyi Brodetskyi 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/53030/
---

Review request for Ambari, Myroslav Papirkovskyy, Sumit Mohanty, and Sid Wagle.


Bugs: AMBARI-18640
https://issues.apache.org/jira/browse/AMBARI-18640


Repository: ambari


Description
---

Observed errors in today's run during Ambari upgrade from 2.2.1.1 to 2.4.2.0-36
ambari-server --hash
c6da6776f029f15d3a7d6009697371eee4e5f4c5

Ambari DB - MySQL; Secure HDP-2.4.0.0 cluster deployed via UI

*Upgrade Log indicates below:*
{code}
18 Oct 2016 14:17:59,115  INFO [main] DBAccessorImpl:824 - Executing query: 
ALTER TABLE users  MODIFY user_name VARCHAR(100)
18 Oct 2016 14:17:59,154  INFO [main] DBAccessorImpl:824 - Executing query: 
ALTER TABLE users  MODIFY user_name VARCHAR(100) NOT NULL
18 Oct 2016 14:17:59,191  INFO [main] DBAccessorImpl:824 - Executing query: 
ALTER TABLE host_role_command  MODIFY role VARCHAR(100)
18 Oct 2016 14:17:59,428  INFO [main] DBAccessorImpl:824 - Executing query: 
ALTER TABLE host_role_command  MODIFY status VARCHAR(100)
18 Oct 2016 14:17:59,656  INFO [main] DBAccessorImpl:824 - Executing query: 
ALTER TABLE blueprint  MODIFY blueprint_name VARCHAR(100)
18 Oct 2016 14:17:59,678 ERROR [main] DBAccessorImpl:830 - Error executing 
query: ALTER TABLE blueprint  MODIFY blueprint_name VARCHAR(100)
java.sql.SQLException: Cannot change column 'blueprint_name': used in a foreign 
key constraint 'FK_blueprint_setting_name' of table 
'ambaricustom.blueprint_setting'
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:996)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2484)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:848)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:742)
at 
org.apache.ambari.server.orm.DBAccessorImpl.executeQuery(DBAccessorImpl.java:827)
at 
org.apache.ambari.server.orm.DBAccessorImpl.executeQuery(DBAccessorImpl.java:819)
at 
org.apache.ambari.server.orm.DBAccessorImpl.alterColumn(DBAccessorImpl.java:610)
at 
org.apache.ambari.server.upgrade.UpgradeCatalog242.updateTablesForMysql(UpgradeCatalog242.java:120)
at 
org.apache.ambari.server.upgrade.UpgradeCatalog242.executeDDLUpdates(UpgradeCatalog242.java:95)
at 
org.apache.ambari.server.upgrade.AbstractUpgradeCatalog.upgradeSchema(AbstractUpgradeCatalog.java:889)
at 
org.apache.ambari.server.upgrade.SchemaUpgradeHelper.executeUpgrade(SchemaUpgradeHelper.java:206)
at 
org.apache.ambari.server.upgrade.SchemaUpgradeHelper.main(SchemaUpgradeHelper.java:349)
18 Oct 2016 14:17:59,680 ERROR [main] SchemaUpgradeHelper:208 - Upgrade failed.
java.sql.SQLException: Cannot change column 'blueprint_name': used in a foreign 
key constraint 'FK_blueprint_setting_name' of table 
'ambaricustom.blueprint_setting'
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:996)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3887)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3823)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2435)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2582)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2526)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2484)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:848)

{code}


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog242.java
 6fa3e68 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog242Test.java
 81f8451 

Diff: https://reviews.apache.org/r/53030/diff/


Testing
---

mvn clean test


Thanks,

Vitalyi Brodetskyi

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

2016-10-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/
---

(Updated Oct. 19, 2016, 5:52 p.m.)


Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.


Changes
---

Missed one code block while rebasing. Unit tests passed again.


Bugs: AMBARI-18619
https://issues.apache.org/jira/browse/AMBARI-18619


Repository: ambari


Description
---

STR:

* Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
host.
* Run multiple service checks in parallel, but notice that they typically run 
on the same 1 or 2 hosts.

Currently, Ambari relies on getting the list of candidate hosts from the DB and 
excludes all hosts that are in maintenance mode. From that list, it picks the 
first host that is healthy (i.e., heartbeating). This means that the logic does 
not pick a random host.

Instead, Ambari should always pick a random host and prefer to schedule on 
hosts that have 0 in-progress commands.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 a671f0c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 7418c1b 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 ef6d4c5 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 17760c7 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
 5556071 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 c06debb 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
 d38108f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
 2ccbcda 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
 d96e7b5 

Diff: https://reviews.apache.org/r/52964/diff/


Testing
---

mvn clean test


Thanks,

Alejandro Fernandez

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

2016-10-19 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/
---

(Updated Oct. 19, 2016, 5:48 p.m.)


Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, Jonathan 
Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.


Changes
---

Rebased.


Bugs: AMBARI-18619
https://issues.apache.org/jira/browse/AMBARI-18619


Repository: ambari


Description
---

STR:

* Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
host.
* Run multiple service checks in parallel, but notice that they typically run 
on the same 1 or 2 hosts.

Currently, Ambari relies on getting the list of candidate hosts from the DB and 
excludes all hosts that are in maintenance mode. From that list, it picks the 
first host that is healthy (i.e., heartbeating). This means that the logic does 
not pick a random host.

Instead, Ambari should always pick a random host and prefer to schedule on 
hosts that have 0 in-progress commands.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 a671f0c 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
 7418c1b 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
 ef6d4c5 
  
ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
 17760c7 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
 5556071 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 c06debb 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/ActiveWidgetLayoutResourceProviderTest.java
 d38108f 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserAuthorizationResourceProviderTest.java
 2ccbcda 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/internal/UserResourceProviderTest.java
 d96e7b5 

Diff: https://reviews.apache.org/r/52964/diff/


Testing
---

mvn clean test


Thanks,

Alejandro Fernandez

Re: Review Request 52964: Optimize Service Checks to it picks a random host and prefers hosts with 0 active commands

2016-10-19 Thread Sid Wagle 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52964/#review153297
---


Ship it!




Ship It!

- Sid Wagle


On Oct. 19, 2016, 1:35 a.m., Alejandro Fernandez wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52964/
> ---
> 
> (Updated Oct. 19, 2016, 1:35 a.m.)
> 
> 
> Review request for Ambari, Dmytro Grinenko, Di Li, Dmitro Lisnichenko, 
> Jonathan Hurley, Nate Cole, Sumit Mohanty, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18619
> https://issues.apache.org/jira/browse/AMBARI-18619
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> STR:
> 
> * Deploy a 3-node cluster with Ambari 2.4 and HDP 2.5 with clients on every 
> host.
> * Run multiple service checks in parallel, but notice that they typically run 
> on the same 1 or 2 hosts.
> 
> Currently, Ambari relies on getting the list of candidate hosts from the DB 
> and excludes all hosts that are in maintenance mode. From that list, it picks 
> the first host that is healthy (i.e., heartbeating). This means that the 
> logic does not pick a random host.
> 
> Instead, Ambari should always pick a random host and prefer to schedule on 
> hosts that have 0 in-progress commands.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  a671f0c 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  746bca4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  5d8f279 
>   
> ambari-server/src/main/java/org/apache/ambari/server/orm/dao/HostRoleCommandDAO.java
>  17760c7 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelperTest.java
>  e82900d 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  197b925 
> 
> Diff: https://reviews.apache.org/r/52964/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>

Re: Review Request 52996: For rolling upgrade of Kafka 0.10.0.1, set configs for backward compatibility

2016-10-19 Thread Nate Cole 


> On Oct. 18, 2016, 10:52 p.m., Jonathan Hurley wrote:
> > Out of curiousity, instead of providing a message about removing the two 
> > properties, could we just have a new stage right before finalization that 
> > does it automatically? I know it won't restart anything, but at least the 
> > configs will be correct.

Thanks for the review!  From Kafka team:

"I think safe approach is to let the user know and let them make a decision of 
deleting the properties when they seem fit. Till then it will run with older 
log format version.
second approach delete the property and go through another rolling restart of 
the cluster.
Yes lets put a message for "log.message.format.version" once all their clients 
are upgraded. We shouldn't be deleting this property."


- Nate


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52996/#review153239
---


On Oct. 18, 2016, 8:57 p.m., Nate Cole wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52996/
> ---
> 
> (Updated Oct. 18, 2016, 8:57 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-18630
> https://issues.apache.org/jira/browse/AMBARI-18630
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> In the 2.5 upgrade pack, the following properties need to be set
> {code}
> inter.broker.protocol.version=0.9.0.0
> log.message.format.version=0.9.0.0
> {code}
> 
> after upgrade is done we should delete the inter.broker.protocol.version.
> Users should remove log.message.format.version once they update their clients.
> 
> (Still waiting on final confirmation for behavior from Kafka team)
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> e8a2e2f 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.5.xml 
> e89d62b 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> e127dff 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.5.xml 
> fa1a555 
> 
> Diff: https://reviews.apache.org/r/52996/diff/
> 
> 
> Testing
> ---
> 
> Manual.  Automated pending.
> 
> 
> Thanks,
> 
> Nate Cole
> 
>

Re: Review Request 52688: Capacity Scheduler View: Calculating absolute capacity for node labels and showing in sunburst chart

2016-10-19 Thread Akhil PB 


> On Oct. 12, 2016, 8:35 a.m., Sreenath Somarajapuram wrote:
> > contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js,
> >  line 120
> > 
> >
> > Why is this conditional operator required?

Not really required, could be avoided.


> On Oct. 12, 2016, 8:35 a.m., Sreenath Somarajapuram wrote:
> > contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js,
> >  line 252
> > 
> >
> > Is this conditional operator required?

No, could be made simple as logical comparison.


- Akhil


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52688/#review152284
---


On Oct. 19, 2016, 11:42 a.m., Akhil PB wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52688/
> ---
> 
> (Updated Oct. 19, 2016, 11:42 a.m.)
> 
> 
> Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and 
> Sreenath Somarajapuram.
> 
> 
> Bugs: AMBARI-18561
> https://issues.apache.org/jira/browse/AMBARI-18561
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Calculating absolute capacity for node labels
> Render sunburst chart based on node label capacity when selected
> Read-only mode for non-operator users in preemption and queue mappings
> Bug fixes
> 
> 
> Diffs
> -
> 
>   contrib/views/capacity-scheduler/src/main/resources/ui/app/app.js 4349538 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/labelCapacityBar.js
>  032b2ec 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueMapping.js
>  1d07351 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/components/sunburstChart.js
>  d5b19ae 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/advanced.js
>  af15fa5 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/capsched.js
>  631edbb 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js
>  9d84704 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js
>  e7fba41 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/scheduler.js
>  aa139b1 
>   contrib/views/capacity-scheduler/src/main/resources/ui/app/models/queue.js 
> e2d67bd 
>   contrib/views/capacity-scheduler/src/main/resources/ui/app/router.js 
> d838958 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/styles/application.less
>  1ab1740 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/advanced.hbs
>  9973708 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/preemption.hbs
>  6d398f6 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/editLabelCapacity.hbs
>  8fe627e 
>   
> contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueMapping.hbs
>  f143911 
> 
> Diff: https://reviews.apache.org/r/52688/diff/
> 
> 
> Testing
> ---
> 
> Manual Unit Testing
> 
> 
> Thanks,
> 
> Akhil PB
> 
>

Re: Review Request 52688: Capacity Scheduler View: Calculating absolute capacity for node labels and showing in sunburst chart

2016-10-19 Thread Akhil PB 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52688/
---

(Updated Oct. 19, 2016, 11:42 a.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and Sreenath 
Somarajapuram.


Changes
---

Updated


Bugs: AMBARI-18561
https://issues.apache.org/jira/browse/AMBARI-18561


Repository: ambari


Description
---

Calculating absolute capacity for node labels
Render sunburst chart based on node label capacity when selected
Read-only mode for non-operator users in preemption and queue mappings
Bug fixes


Diffs (updated)
-

  contrib/views/capacity-scheduler/src/main/resources/ui/app/app.js 4349538 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/labelCapacityBar.js
 032b2ec 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/queueMapping.js
 1d07351 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/sunburstChart.js
 d5b19ae 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/advanced.js
 af15fa5 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/capsched.js
 631edbb 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/editqueue.js
 9d84704 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/queuesconf.js
 e7fba41 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/controllers/scheduler.js
 aa139b1 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/models/queue.js 
e2d67bd 
  contrib/views/capacity-scheduler/src/main/resources/ui/app/router.js d838958 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/styles/application.less
 1ab1740 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/advanced.hbs
 9973708 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/capsched/partials/preemption.hbs
 6d398f6 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/editLabelCapacity.hbs
 8fe627e 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/components/queueMapping.hbs
 f143911 

Diff: https://reviews.apache.org/r/52688/diff/


Testing
---

Manual Unit Testing


Thanks,

Akhil PB

Re: Review Request 52839: Ability to enable the DEBUG/INFO logs in Ambari-server without server restart

2016-10-19 Thread Attila Doroszlai 


> On Oct. 18, 2016, 7:37 a.m., Attila Doroszlai wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java,
> >  lines 953-954
> > 
> >
> > Variable names `monitor_delay` and `monitor_dealy_long` should be 
> > camelCase, ie. `monitorDelay`.
> > 
> > Also, `monitor_dealy_long` has typo, should be `monitorDelayLong`.
> 
> Amarnath reddy pappu wrote:
> implemented

Please check diff version 3, it's the same as version 2.


- Attila


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/52839/#review153034
---


On Oct. 18, 2016, 8:29 p.m., Amarnath reddy pappu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/52839/
> ---
> 
> (Updated Oct. 18, 2016, 8:29 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Jonathan Hurley, Nate Cole, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-18592
> https://issues.apache.org/jira/browse/AMBARI-18592
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ability to enable the DEBUG logs in Ambari-server without server restart
> 
> 1. PropertyConfigurator would check log4j.properties update every 5mins. if 
> there is any change in the file/timestamp it will reload the log4j 
> configurations.
> 2. have defined to LOG4JMONITOR_DELAY for 5mins - 30 milliseconds
> 3. Have modified only for ambari-server
> 4.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  378db18 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  c05840e 
> 
> Diff: https://reviews.apache.org/r/52839/diff/
> 
> 
> Testing
> ---
> 
> 1. Change the log4j.properties from INFO to DEBUG
> [Expected Result] server should start logging the DEBUG logs with in 5mins of 
> time.
> [Resutl] PASS
> 
> 2. Change the log4j.properties from DEBUG to INFO
> [Expected Result] server should reduce the logging to INFO mode with in 5mins 
> of time.
> [Resutl] PASS
> 
> 
> Thanks,
> 
> Amarnath reddy pappu
> 
>