date:20160701


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49504/#review140331
---


Ship it!




Ship It!

- Dmytro Sen


On Июль 1, 2016, 10:51 д.п., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49504/
> ---
> 
> (Updated Июль 1, 2016, 10:51 д.п.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-17514
> https://issues.apache.org/jira/browse/AMBARI-17514
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> For ORACLE:
> Case 1
> db_host=host:port:SID, example: db_host=localhost:1521:ORCL
> connection_string = jdbc:oracle:thin:@db_host
> Case 2
> db_host=host:port/ServiceName, example: db_host=localhost:1521/XE
> connection_string = jdbc:oracle:thin:@//db_host
> Adding message if user forgot to set jdbc using "ambari-server setup 
> --jdbc-db= --jdbc-driver="
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  ba8d3b3 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  e35ea5f 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
>  133760b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dce6576 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 6a3df08 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 62d926e 
> 
> Diff: https://reviews.apache.org/r/49504/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Re: Review Request 49504: StackAdvisor fix for Oracle 12C and Add message if user forgot to set jdbc

2016-07-01 Thread Dmytro Grinenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49504/#review140332
---


Ship it!




Ship It!

- Dmytro Grinenko


On July 1, 2016, 10:51 a.m., Vitalyi Brodetskyi wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49504/
> ---
> 
> (Updated July 1, 2016, 10:51 a.m.)
> 
> 
> Review request for Ambari, Andrew Onischuk, Dmytro Sen, and Velmurugan 
> Periasamy.
> 
> 
> Bugs: AMBARI-17514
> https://issues.apache.org/jira/browse/AMBARI-17514
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> For ORACLE:
> Case 1
> db_host=host:port:SID, example: db_host=localhost:1521:ORCL
> connection_string = jdbc:oracle:thin:@db_host
> Case 2
> db_host=host:port/ServiceName, example: db_host=localhost:1521/XE
> connection_string = jdbc:oracle:thin:@//db_host
> Adding message if user forgot to set jdbc using "ambari-server setup 
> --jdbc-db= --jdbc-driver="
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/params.py
>  ba8d3b3 
>   
> ambari-server/src/main/resources/common-services/RANGER/0.4.0/package/scripts/setup_ranger_xml.py
>  e35ea5f 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
>  133760b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dce6576 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> 6a3df08 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 62d926e 
> 
> Diff: https://reviews.apache.org/r/49504/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Vitalyi Brodetskyi
> 
>

Review Request 49505: AMBARI-17521: If Solr is configured to use implicit routing, then replicationFactor is ignored

2016-07-01 Thread Don Bosco Durai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49505/
---

Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, and 
Sumit Mohanty.


Bugs: AMBARI-17521
https://issues.apache.org/jira/browse/AMBARI-17521


Repository: ambari


Description
---

Now passing replicationFactor to create method for implicit routing use case 
also


Diffs
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
 982a72d 
  ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh b84bfc2 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 c13105a 

Diff: https://reviews.apache.org/r/49505/diff/


Testing
---

Tested with 6 nodes SolrCloud cluster


Thanks,

Don Bosco Durai

Re: Review Request 49505: AMBARI-17521: If Solr is configured to use implicit routing, then replicationFactor is ignored

2016-07-01 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49505/#review140334
---


Ship it!




Ship It!

- Oliver Szabo


On July 1, 2016, 11:20 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49505/
> ---
> 
> (Updated July 1, 2016, 11:20 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17521
> https://issues.apache.org/jira/browse/AMBARI-17521
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Now passing replicationFactor to create method for implicit routing use case 
> also
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  982a72d 
>   ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh b84bfc2 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  c13105a 
> 
> Diff: https://reviews.apache.org/r/49505/diff/
> 
> 
> Testing
> ---
> 
> Tested with 6 nodes SolrCloud cluster
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>

Re: Review Request 49455: Optimized classpath scannig for upgrade check impelementations

2016-07-01 Thread Laszlo Puskas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49455/
---

(Updated July 1, 2016, 12:12 p.m.)


Review request for Ambari, Daniel Gergely, Sumit Mohanty, and Sebastian Toader.


Summary (updated)
-

Optimized classpath scannig for upgrade check impelementations


Bugs: AMBARI-17505
https://issues.apache.org/jira/browse/AMBARI-17505


Repository: ambari


Description
---

Problem:
During startup the ambari server scasns the classpath for finding components to 
be bound in the IoC context.
When binding upgrade check implementations the full ambari package is scanned 
that leads to prolonged startup time.

Solution:
As upgrade check implementations reside in a dedicated package, the scanner is 
modified to lookup them in this very package.
(on the local env this shortens the startup time by ~25 seconds)


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
 e0bda13 

Diff: https://reviews.apache.org/r/49455/diff/


Testing
---

Unit tests running.


Thanks,

Laszlo Puskas

Re: Review Request 49455: Optimized classpath scannig for upgrade check impelementations

2016-07-01 Thread Laszlo Puskas



> On June 30, 2016, 11:47 p.m., Alejandro Fernandez wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java,
> >  line 648
> > 
> >
> > Can we use reflection to look up the package name instead of hardcoding 
> > it?

Fixed this, however ther are other packagenames hardcoded here; long term it 
would be good to have the whole classpath scanning logic refactored so that all 
beans are looked up with a single scan ...


- Laszlo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49455/#review140277
---


On July 1, 2016, 12:12 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49455/
> ---
> 
> (Updated July 1, 2016, 12:12 p.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Sumit Mohanty, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-17505
> https://issues.apache.org/jira/browse/AMBARI-17505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> During startup the ambari server scasns the classpath for finding components 
> to be bound in the IoC context.
> When binding upgrade check implementations the full ambari package is scanned 
> that leads to prolonged startup time.
> 
> Solution:
> As upgrade check implementations reside in a dedicated package, the scanner 
> is modified to lookup them in this very package.
> (on the local env this shortens the startup time by ~25 seconds)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  e0bda13 
> 
> Diff: https://reviews.apache.org/r/49455/diff/
> 
> 
> Testing
> ---
> 
> Unit tests running.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Review Request 49507: AMBARI-17520: Update the policy_user property to use storm user principal specified in Storms config

2016-07-01 Thread Gautam Borad


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49507/
---

Review request for Ambari, Alejandro Fernandez, Mugdha Varadkar, Robert Levas, 
Sriharsha Chintalapani, Srimanth Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17520
https://issues.apache.org/jira/browse/AMBARI-17520


Repository: ambari


Description
---

Update the policy_user property in Advanced ranger-storm-plugin-properties of 
Ranger with the value of the storm user bare principal specified in Storms 
Ambari config.
With this the principal used for storm will also be added to default ranger 
policy and will prevent Storm service check failures.


Diffs
-

  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 073bb1c 
  
ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
 2fee04f 

Diff: https://reviews.apache.org/r/49507/diff/


Testing
---

Tested Ranger storm plugin on centos6 cluster. Kerberized the cluster and 
checked that Storm service check is working fine.


Thanks,

Gautam Borad

Re: Review Request 49505: AMBARI-17521: If Solr is configured to use implicit routing, then replicationFactor is ignored

2016-07-01 Thread Robert Nettleton


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49505/#review140344
---


Ship it!




Ship It!

- Robert Nettleton


On July 1, 2016, 11:20 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49505/
> ---
> 
> (Updated July 1, 2016, 11:20 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17521
> https://issues.apache.org/jira/browse/AMBARI-17521
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Now passing replicationFactor to create method for implicit routing use case 
> also
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  982a72d 
>   ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh b84bfc2 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  c13105a 
> 
> Diff: https://reviews.apache.org/r/49505/diff/
> 
> 
> Testing
> ---
> 
> Tested with 6 nodes SolrCloud cluster
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>

Review Request 49508: AMBARI-17522 Handle Ranger Kms upgrade in kerberos env

2016-07-01 Thread Mugdha Varadkar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49508/
---

Review request for Ambari, Alejandro Fernandez, Gautam Borad, Robert Levas, 
Srimanth Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17522
https://issues.apache.org/jira/browse/AMBARI-17522


Repository: ambari


Description
---

Problem:
Upgrade from 2.4 to 2.5 will not have rangerkms principal/keytab identity (as 
it is added in stack 2.5) to create/get repository from Ranger Service.
Ranger KMS service had spnego principal/keytab identity in 2.4.
Ranger KMS service is a dependent on Ranger Service. For previous stack all 
calls were using urllib2 authentication to create/get repository in Ranger 
Service in kerberos env. Ranger Service participate in kerberos from 2.5 
onwards so all call to create/get repository will be using curl --negotiate.

Solution:
If rangerkms principal/keytab is not available during upgrade, using spnego 
principal/keytab.


Diffs
-

  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
 133760b 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
 dce6576 

Diff: https://reviews.apache.org/r/49508/diff/


Testing
---

Running tests for stack:2.5 service:RANGER_KMS
test_configure_default (test_kms_server.TestRangerKMS) ... 2016-07-01 
18:00:57,586 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
ok
test_configure_secured (test_kms_server.TestRangerKMS) ... 2016-07-01 
18:00:57,607 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
ok
test_start_default (test_kms_server.TestRangerKMS) ... 2016-07-01 18:00:57,630 
- Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-07-01 18:00:57,634 - Rangeradmin: Skip ranger admin if it's down !
ok
test_start_secured (test_kms_server.TestRangerKMS) ... 2016-07-01 18:00:57,661 
- Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-07-01 18:00:57,667 - RangeradminV2: Skip ranger admin if it's down !
2016-07-01 18:00:57,667 - KMS repository c1_kms exist
ok
test_stop_default (test_kms_server.TestRangerKMS) ... 2016-07-01 18:00:57,685 - 
Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
ok

--
Ran 5 tests in 0.135s

OK


Thanks,

Mugdha Varadkar

Re: Review Request 49507: AMBARI-17520: Update the policy_user property to use storm user principal specified in Storms config

2016-07-01 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49507/#review140346
---



Was this tested with the Storm Kerberos identity set to something like 
`storm1234@${realm}`?


ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 (line 303)


Why are we hard-coding `{storm-user}-{cluster-name}` here?   If it is 
related to the Storm Kerberos identitiy, then there is no guarentee that the 
user won't change this when configuring Kerberos identities.


- Robert Levas


On July 1, 2016, 9:14 a.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49507/
> ---
> 
> (Updated July 1, 2016, 9:14 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Mugdha Varadkar, Robert 
> Levas, Sriharsha Chintalapani, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17520
> https://issues.apache.org/jira/browse/AMBARI-17520
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update the policy_user property in Advanced ranger-storm-plugin-properties of 
> Ranger with the value of the storm user bare principal specified in Storms 
> Ambari config.
> With this the principal used for storm will also be added to default ranger 
> policy and will prevent Storm service check failures.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
>  2fee04f 
> 
> Diff: https://reviews.apache.org/r/49507/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger storm plugin on centos6 cluster. Kerberized the cluster and 
> checked that Storm service check is working fine.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 48852: Ambari displays a warning about config values not being at optimal values right after a clean install with no customization


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/48852/#review140347
---


Ship it!




Ship It!

- Dmytro Sen


On Июнь 17, 2016, 2:54 п.п., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/48852/
> ---
> 
> (Updated Июнь 17, 2016, 2:54 п.п.)
> 
> 
> Review request for Ambari, Dmytro Sen, Myroslav Papirkovskyy, and Srimanth 
> Gunturi.
> 
> 
> Bugs: AMBARI-17296
> https://issues.apache.org/jira/browse/AMBARI-17296
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> With nothing changed manually on a new install, Ambari displayed a message
> saying that certain configs are not at the recommended values.  
> Given this was a new install with no modifications - the values should have
> been populated by Ambari itself.
> 
> I don't know what the final values was because the cluster install failed
> (likely due to repo issues)
> 
> Attaching a screenshot.
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py 
> 22d29e5 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> c32306d 
>   ambari-server/src/main/resources/stacks/stack_advisor.py d8685c3 
>   ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py 
> d22b453 
>   ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py 
> 1bd385f 
> 
> Diff: https://reviews.apache.org/r/48852/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49508: AMBARI-17522 Handle Ranger Kms upgrade in kerberos env

2016-07-01 Thread Mugdha Varadkar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49508/
---

(Updated July 1, 2016, 1:48 p.m.)


Review request for Ambari, Alejandro Fernandez, Gautam Borad, Robert Levas, 
Srimanth Gunturi, and Velmurugan Periasamy.


Bugs: AMBARI-17522
https://issues.apache.org/jira/browse/AMBARI-17522


Repository: ambari


Description (updated)
---

Problem:
Upgrade from 2.4 to 2.5 will not have rangerkms principal/keytab identity (as 
it is added in stack 2.5) to create/get repository in Ranger Service.
Ranger KMS service had spnego principal/keytab identity in 2.4.
Ranger KMS service is a dependent on Ranger Service. For previous stack all 
calls were using urllib2 authentication to create/get repository in Ranger 
Service in kerberos env. Ranger Service participate in kerberos from 2.5 
onwards so all call to create/get repository will be using curl --negotiate.

Solution:
If rangerkms principal/keytab is not available during upgrade, using spnego 
principal/keytab.


Diffs
-

  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
 133760b 
  
ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
 dce6576 

Diff: https://reviews.apache.org/r/49508/diff/


Testing
---

Running tests for stack:2.5 service:RANGER_KMS
test_configure_default (test_kms_server.TestRangerKMS) ... 2016-07-01 
18:00:57,586 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
ok
test_configure_secured (test_kms_server.TestRangerKMS) ... 2016-07-01 
18:00:57,607 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
ok
test_start_default (test_kms_server.TestRangerKMS) ... 2016-07-01 18:00:57,630 
- Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-07-01 18:00:57,634 - Rangeradmin: Skip ranger admin if it's down !
ok
test_start_secured (test_kms_server.TestRangerKMS) ... 2016-07-01 18:00:57,661 
- Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
2016-07-01 18:00:57,667 - RangeradminV2: Skip ranger admin if it's down !
2016-07-01 18:00:57,667 - KMS repository c1_kms exist
ok
test_stop_default (test_kms_server.TestRangerKMS) ... 2016-07-01 18:00:57,685 - 
Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
ok

--
Ran 5 tests in 0.135s

OK


Thanks,

Mugdha Varadkar

Re: Review Request 49505: AMBARI-17521: If Solr is configured to use implicit routing, then replicationFactor is ignored

2016-07-01 Thread Miklos Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49505/#review140348
---


Ship it!




Ship It!

- Miklos Gergely


On July 1, 2016, 11:20 a.m., Don Bosco Durai wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49505/
> ---
> 
> (Updated July 1, 2016, 11:20 a.m.)
> 
> 
> Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17521
> https://issues.apache.org/jira/browse/AMBARI-17521
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Now passing replicationFactor to create method for implicit routing use case 
> also
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-logfeeder/src/main/resources/logfeeder.properties
>  982a72d 
>   ambari-logsearch/ambari-logsearch-logfeeder/src/main/scripts/run.sh b84bfc2 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
>  c13105a 
> 
> Diff: https://reviews.apache.org/r/49505/diff/
> 
> 
> Testing
> ---
> 
> Tested with 6 nodes SolrCloud cluster
> 
> 
> Thanks,
> 
> Don Bosco Durai
> 
>

Review Request 49511: Falcon start fails rarely

2016-07-01 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49511/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17524
https://issues.apache.org/jira/browse/AMBARI-17524


Repository: ambari


Description
---

Traceback (most recent call last):
  File 
"/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py",
 line 177, in 
FalconServer().execute()
  File 
"/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
 line 280, in execute
method(env)
  File 
"/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py",
 line 50, in start
falcon('server', action='start', upgrade_type=upgrade_type)
  File "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", 
line 89, in thunk
return fn(*args, **kwargs)
  File 
"/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py",
 line 194, in falcon
environment=environment_dictionary)
  File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", 
line 155, in __init__
self.env.run()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 160, in run
self.run_action(resource, action)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
line 124, in run_action
provider_action()
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
 line 273, in action_run
tries=self.resource.tries, try_sleep=self.resource.try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 71, 
in inner
result = function(command, **kwargs)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 93, 
in checked_call
tries=tries, try_sleep=try_sleep)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 141, 
in _call_wrapper
result = _call(command, **kwargs_copy)
  File 
"/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 294, 
in _call
raise Fail(err_msg)
resource_management.core.exceptions.Fail: Execution of 
'/usr/hdp/current/falcon-server/bin/falcon-start -port 15000' returned 1. 
Hadoop home is set, adding libraries from 
'/usr/hdp/current/hadoop-client/bin/hadoop classpath' into falcon classpath
falcon running as process 3443. Stop it first.


This happens because falcon start was not idempotent unlike other start
command.


Diffs
-

  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
 c2f1f53 
  ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 6f81b15 

Diff: https://reviews.apache.org/r/49511/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 49510: Zeppelin: remove Livy component dependency


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/
---

Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-17523
https://issues.apache.org/jira/browse/AMBARI-17523


Repository: ambari


Description
---

- remove Livy component dependency from metainfo.xml
- code clean up


Diffs
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
 4f19b43 
  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
 fd6cbb6 
  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
 936df81 

Diff: https://reviews.apache.org/r/49510/diff/


Testing
---

manually tested on CentOS


Thanks,

Renjith Kamath

Re: Review Request 49510: Zeppelin: remove Livy component dependency

2016-07-01 Thread Prabhjyot Singh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/#review140350
---




ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
 (line 115)


Since you have already initilized hive_* (line 101-104) properties you 
don't need else condition.



ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
 (line 124)


Same here, this else is also redundant.


- Prabhjyot Singh


On July 1, 2016, 1:52 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49510/
> ---
> 
> (Updated July 1, 2016, 1:52 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17523
> https://issues.apache.org/jira/browse/AMBARI-17523
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - remove Livy component dependency from metainfo.xml
> - code clean up
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  4f19b43 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  fd6cbb6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  936df81 
> 
> Diff: https://reviews.apache.org/r/49510/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49511: Falcon start fails rarely

2016-07-01 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49511/#review140352
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On July 1, 2016, 4:52 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49511/
> ---
> 
> (Updated July 1, 2016, 4:52 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17524
> https://issues.apache.org/jira/browse/AMBARI-17524
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py",
>  line 177, in 
> FalconServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py",
>  line 50, in start
> falcon('server', action='start', upgrade_type=upgrade_type)
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py",
>  line 194, in falcon
> environment=environment_dictionary)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 71, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 93, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 141, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 294, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/usr/hdp/current/falcon-server/bin/falcon-start -port 15000' returned 1. 
> Hadoop home is set, adding libraries from 
> '/usr/hdp/current/hadoop-client/bin/hadoop classpath' into falcon classpath
> falcon running as process 3443. Stop it first.
> 
> 
> This happens because falcon start was not idempotent unlike other start
> command.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  c2f1f53 
>   ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 
> 6f81b15 
> 
> Diff: https://reviews.apache.org/r/49511/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49508: AMBARI-17522 Handle Ranger Kms upgrade in kerberos env

2016-07-01 Thread Robert Levas


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49508/#review140354
---


Ship it!




Ship It!

- Robert Levas


On July 1, 2016, 9:48 a.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49508/
> ---
> 
> (Updated July 1, 2016, 9:48 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, Robert Levas, 
> Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17522
> https://issues.apache.org/jira/browse/AMBARI-17522
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> Upgrade from 2.4 to 2.5 will not have rangerkms principal/keytab identity (as 
> it is added in stack 2.5) to create/get repository in Ranger Service.
> Ranger KMS service had spnego principal/keytab identity in 2.4.
> Ranger KMS service is a dependent on Ranger Service. For previous stack all 
> calls were using urllib2 authentication to create/get repository in Ranger 
> Service in kerberos env. Ranger Service participate in kerberos from 2.5 
> onwards so all call to create/get repository will be using curl --negotiate.
> 
> Solution:
> If rangerkms principal/keytab is not available during upgrade, using spnego 
> principal/keytab.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/kms.py
>  133760b 
>   
> ambari-server/src/main/resources/common-services/RANGER_KMS/0.5.0.2.3/package/scripts/params.py
>  dce6576 
> 
> Diff: https://reviews.apache.org/r/49508/diff/
> 
> 
> Testing
> ---
> 
> Running tests for stack:2.5 service:RANGER_KMS
> test_configure_default (test_kms_server.TestRangerKMS) ... 2016-07-01 
> 18:00:57,586 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
> ok
> test_configure_secured (test_kms_server.TestRangerKMS) ... 2016-07-01 
> 18:00:57,607 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
> ok
> test_start_default (test_kms_server.TestRangerKMS) ... 2016-07-01 
> 18:00:57,630 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
> 2016-07-01 18:00:57,634 - Rangeradmin: Skip ranger admin if it's down !
> ok
> test_start_secured (test_kms_server.TestRangerKMS) ... 2016-07-01 
> 18:00:57,661 - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
> 2016-07-01 18:00:57,667 - RangeradminV2: Skip ranger admin if it's down !
> 2016-07-01 18:00:57,667 - KMS repository c1_kms exist
> ok
> test_stop_default (test_kms_server.TestRangerKMS) ... 2016-07-01 18:00:57,685 
> - Using hadoop conf dir: /usr/hdp/current/hadoop-client/conf
> ok
> 
> --
> Ran 5 tests in 0.135s
> 
> OK
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 49486: AMBARI-17519. Use Host based principal for Hive Server interactive's LLAP config 'hive.llap.zk.sm.principal' instead of headless ones.


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49486/#review140355
---


Ship it!




Ship It!

- Sumit Mohanty


On July 1, 2016, 10:15 a.m., Swapan Shridhar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49486/
> ---
> 
> (Updated July 1, 2016, 10:15 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17519
> https://issues.apache.org/jira/browse/AMBARI-17519
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - We are currently using headless principal for 'hive.llap.zk.sm.principal' 
> as username@REALM.
> - Fix: We need to use host based principal instead for headless one.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/alerts.json 
> bffc030 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/alerts/alert_llap_app_status.py
>  6232dff 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
>  bc8a9f0 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/params_linux.py
>  17f7380 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/service_check.py
>  d342308 
>   ambari-server/src/main/resources/stacks/HDP/2.5/services/YARN/kerberos.json 
> e8a2887 
> 
> Diff: https://reviews.apache.org/r/49486/diff/
> 
> 
> Testing
> ---
> 
> Python UT passes.
> 
> 
> Thanks,
> 
> Swapan Shridhar
> 
>

Re: Review Request 49510: Zeppelin: remove Livy component dependency


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/
---

(Updated July 1, 2016, 2:23 p.m.)


Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav Nagar, 
Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit Mohanty.


Bugs: AMBARI-17523
https://issues.apache.org/jira/browse/AMBARI-17523


Repository: ambari


Description
---

- remove Livy component dependency from metainfo.xml
- code clean up


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
 4f19b43 
  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
 fd6cbb6 
  
ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
 936df81 

Diff: https://reviews.apache.org/r/49510/diff/


Testing
---

manually tested on CentOS


Thanks,

Renjith Kamath

Re: Review Request 49510: Zeppelin: remove Livy component dependency


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/#review140356
---




ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
 


I understand that this is not honored by UI and only blueprint looks into 
this - why should this be removed?


- Sumit Mohanty


On July 1, 2016, 2:23 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49510/
> ---
> 
> (Updated July 1, 2016, 2:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17523
> https://issues.apache.org/jira/browse/AMBARI-17523
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - remove Livy component dependency from metainfo.xml
> - code clean up
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  4f19b43 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  fd6cbb6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  936df81 
> 
> Diff: https://reviews.apache.org/r/49510/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49510: Zeppelin: remove Livy component dependency


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/#review140357
---




ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
 (line 204)


Is this tested in non-root scenarios?
You can discuss with Andrew O to see if this will pose an issue for 
non-root scenarios.


- Sumit Mohanty


On July 1, 2016, 2:23 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49510/
> ---
> 
> (Updated July 1, 2016, 2:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17523
> https://issues.apache.org/jira/browse/AMBARI-17523
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - remove Livy component dependency from metainfo.xml
> - code clean up
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  4f19b43 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  fd6cbb6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  936df81 
> 
> Diff: https://reviews.apache.org/r/49510/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49474: Ambari LogSearch Integration should request logging metadata on a separate thread

2016-07-01 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49474/#review140353
---




ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 (line 36)


JavaDoc.



ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 (lines 66 - 67)


Remove?



ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 (line 79)


Based on what we've seen, the requests to the LOGSEARCH endpoint don't 
return until the connection times out (which could be never depending on what's 
configured). This will then cause this Executor to fill it's queue with a 
backlog of requests.

Instead, maybe it would be better to:
- Only enqueue if the request isn't already enqueued
- Use a bounded executor and rejection policy



ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 (line 115)


Use {} to prevent concatenation on DEBUG statements.



ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 (lines 149 - 151)


Maybe only enqueue requests if the same request isn't on the queue.



ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
 (line 163)


JavaDoc.



ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProvider.java
 (lines 78 - 79)


This is interesting; how would this service not be available if it's 
injected?


I'm a bit confused by the changes made here, namely that 
`sendLogLevelQueryRequest` was replaced with `sendGetLogFileNamesRequest`. Is 
`sendLogLevelQueryRequest` no longer used? If so, it can be removed, right?

- Jonathan Hurley


On June 30, 2016, 5:11 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49474/
> ---
> 
> (Updated June 30, 2016, 5:11 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Mahadev Konar, Oliver Szabo, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17510
> https://issues.apache.org/jira/browse/AMBARI-17510
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch implements some updates to the Ambari LogSearch integration layer 
> in order to resolve AMBARI-17510. 
> 
> The initial implementation of the LoggingSearchPropertyProvider directly made 
> calls to the LogSearch Server's REST endpoint, in order to obtain LogSearch 
> metadata, and to attach this information to the Ambari HostComponent REST 
> resource.  Recent testing has shown that there are some deployment scenarios 
> (larger clusters, rolling upgrade) that demonstrate a performance problem in 
> the LogSearch integration code.  
> 
> This patch addresses this problem by implementing the following:
> 
> 1. A new, injectable service 
> (org.apache.ambari.server.controller.logging.LogSearchDataRetrievalService) 
> has been introduced into the LoggingSearchPropertyProvider.  The property 
> provider now consults this service to obtain the required LogSearch data.  
> This new service is based on Jonathan Hurley's work in 
> org.apache.ambari.server.state.services.MetricsRetrievalService.  
> 2. The LogSearchDataRetrievalService utilizes an internal cache to maintain a 
> map of the LogSearch data, which typically will not change.  If the cache 
> does not contain this information for a given HostComponent, the retrieval 
> service will make the required REST call to the LogSearch service to obtain 
> this data, and then add this to the cache.  The remote REST call now happens 
> on a separate thread from the Ambari REST request handler, which removes the 
> possibility of causing degradation on the Ambari REST endpoing. 
> 3. The LoggingRequestHelperImpl class has been slightly updated, in order to 
> add 5 second connect/read timeouts on the LogSearch requests.  This should 
> prevent any threads from hanging infinitely if the LogSearch server is slow 
> to respond.  In a future patch, we might want to make these timeouts 
> configurable.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  947a9f4 
>   
> ambari-server/src/main/java/org/ap

Re: Review Request 49287: After switching to external database in hive, user should be allowed to delete mysql server

2016-07-01 Thread Alexandr Antonenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49287/#review140362
---


Ship it!




Ship It!

- Alexandr Antonenko


On June 27, 2016, 11:20 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49287/
> ---
> 
> (Updated June 27, 2016, 11:20 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-17358
> https://issues.apache.org/jira/browse/AMBARI-17358
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> When user switches from MySQL to Existing database in hive and stop the MySQL 
> Server
> 
> 1) Ambari UI > Hive > Service Actions > Stop becomes unavailable, so Hive 
> can't be stopped anymore
> 
> 2) Hive status is showing up as red Triangle indicating there is a problem 
> with Hive
> 
> Providing an option to delete MySQL server when an existing database is used 
> in Hive would help in overcoming the issues.
> 
> 
> Solution:
> An option is included to delete MySQL Server, the delete will be enabled only 
> when Hive is using an existing(or external) database and the MySQL server is 
> not started.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/stack_service_component.js a350f15 
>   ambari-web/app/routes/main.js 4545f54 
>   ambari-web/app/views/main/host/details/host_component_view.js 910c71f 
>   ambari-web/test/models/stack_service_component_test.js 7e971ce 
>   ambari-web/test/views/main/host/details/host_component_view_test.js 648f0f6 
> 
> Diff: https://reviews.apache.org/r/49287/diff/
> 
> 
> Testing
> ---
> 
> Ran mvn test
> 
> 28974 tests complete (43 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 49287: After switching to external database in hive, user should be allowed to delete mysql server

2016-07-01 Thread Anita Jebaraj



> On July 1, 2016, 2:58 p.m., Alexandr Antonenko wrote:
> > Ship It!

can you please help in pushing the code.


- Anita


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49287/#review140362
---


On June 27, 2016, 11:20 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49287/
> ---
> 
> (Updated June 27, 2016, 11:20 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-17358
> https://issues.apache.org/jira/browse/AMBARI-17358
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> When user switches from MySQL to Existing database in hive and stop the MySQL 
> Server
> 
> 1) Ambari UI > Hive > Service Actions > Stop becomes unavailable, so Hive 
> can't be stopped anymore
> 
> 2) Hive status is showing up as red Triangle indicating there is a problem 
> with Hive
> 
> Providing an option to delete MySQL server when an existing database is used 
> in Hive would help in overcoming the issues.
> 
> 
> Solution:
> An option is included to delete MySQL Server, the delete will be enabled only 
> when Hive is using an existing(or external) database and the MySQL server is 
> not started.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/stack_service_component.js a350f15 
>   ambari-web/app/routes/main.js 4545f54 
>   ambari-web/app/views/main/host/details/host_component_view.js 910c71f 
>   ambari-web/test/models/stack_service_component_test.js 7e971ce 
>   ambari-web/test/views/main/host/details/host_component_view_test.js 648f0f6 
> 
> Diff: https://reviews.apache.org/r/49287/diff/
> 
> 
> Testing
> ---
> 
> Ran mvn test
> 
> 28974 tests complete (43 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Review Request 49513: Ambari API for api/v1/clusters/?fields=hosts/* does not return the metrics information


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49513/
---

Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-17526
https://issues.apache.org/jira/browse/AMBARI-17526


Repository: ambari


Description
---

http://:8080/api/v1/clusters/?fields=hosts/* no 
longer returns the metrics for the hosts.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
 9a7454c 
  ambari-server/src/main/resources/ganglia_properties.json 05360da 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java
 c7dabf1 

Diff: https://reviews.apache.org/r/49513/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 49513: Ambari API for api/v1/clusters/?fields=hosts/* does not return the metrics information


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49513/#review140366
---



Don't we need change in AMSReportPropertyProvider, I thought some of the Host 
calls go to the RPP, if not igore this comment.


ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
 (line 228)


Consider rename to hostNamesBatch



ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
 (line 460)


Consider rename to splitHostnamesInBatches.


- Sid Wagle


On July 1, 2016, 3:16 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49513/
> ---
> 
> (Updated July 1, 2016, 3:16 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-17526
> https://issues.apache.org/jira/browse/AMBARI-17526
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> http://:8080/api/v1/clusters/?fields=hosts/* no 
> longer returns the metrics for the hosts.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
>  9a7454c 
>   ambari-server/src/main/resources/ganglia_properties.json 05360da 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java
>  c7dabf1 
> 
> Diff: https://reviews.apache.org/r/49513/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49513: Ambari API for api/v1/clusters/?fields=hosts/* does not return the metrics information


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49513/
---

(Updated Июль 1, 2016, 3:26 п.п.)


Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
Wagle.


Bugs: AMBARI-17526
https://issues.apache.org/jira/browse/AMBARI-17526


Repository: ambari


Description
---

http://:8080/api/v1/clusters/?fields=hosts/* no 
longer returns the metrics for the hosts.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
 9a7454c 
  ambari-server/src/main/resources/ganglia_properties.json 05360da 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java
 c7dabf1 

Diff: https://reviews.apache.org/r/49513/diff/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 49513: Ambari API for api/v1/clusters/?fields=hosts/* does not return the metrics information



> On Июль 1, 2016, 3:21 п.п., Sid Wagle wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java,
> >  line 463
> > 
> >
> > Consider rename to splitHostnamesInBatches.

Done


> On Июль 1, 2016, 3:21 п.п., Sid Wagle wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java,
> >  line 228
> > 
> >
> > Consider rename to hostNamesBatch

Done


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49513/#review140366
---


On Июль 1, 2016, 3:26 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49513/
> ---
> 
> (Updated Июль 1, 2016, 3:26 п.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-17526
> https://issues.apache.org/jira/browse/AMBARI-17526
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> http://:8080/api/v1/clusters/?fields=hosts/* no 
> longer returns the metrics for the hosts.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
>  9a7454c 
>   ambari-server/src/main/resources/ganglia_properties.json 05360da 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java
>  c7dabf1 
> 
> Diff: https://reviews.apache.org/r/49513/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49287: After switching to external database in hive, user should be allowed to delete mysql server

2016-07-01 Thread Alexandr Antonenko



> On June 30, 2016, 8:23 a.m., Alexandr Antonenko wrote:
> > ambari-web/app/views/main/host/details/host_component_view.js, line 196
> > 
> >
> > this will not work out, as there is no hive-env, only "hive-site". You 
> > have to do a request to get this config
> 
> Anita Jebaraj wrote:
> Hi, yes hive-env will not be available, so I did a getConfigByTags() at 
> the load of the host summary page, so that the value for hive-env will be 
> obtained.

Sorry, I missed that change


- Alexandr


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49287/#review140121
---


On June 27, 2016, 11:20 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49287/
> ---
> 
> (Updated June 27, 2016, 11:20 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-17358
> https://issues.apache.org/jira/browse/AMBARI-17358
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> When user switches from MySQL to Existing database in hive and stop the MySQL 
> Server
> 
> 1) Ambari UI > Hive > Service Actions > Stop becomes unavailable, so Hive 
> can't be stopped anymore
> 
> 2) Hive status is showing up as red Triangle indicating there is a problem 
> with Hive
> 
> Providing an option to delete MySQL server when an existing database is used 
> in Hive would help in overcoming the issues.
> 
> 
> Solution:
> An option is included to delete MySQL Server, the delete will be enabled only 
> when Hive is using an existing(or external) database and the MySQL server is 
> not started.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/stack_service_component.js a350f15 
>   ambari-web/app/routes/main.js 4545f54 
>   ambari-web/app/views/main/host/details/host_component_view.js 910c71f 
>   ambari-web/test/models/stack_service_component_test.js 7e971ce 
>   ambari-web/test/views/main/host/details/host_component_view_test.js 648f0f6 
> 
> Diff: https://reviews.apache.org/r/49287/diff/
> 
> 
> Testing
> ---
> 
> Ran mvn test
> 
> 28974 tests complete (43 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Review Request 49515: Setup logging to find memory leak rootcause

2016-07-01 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49515/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17527
https://issues.apache.org/jira/browse/AMBARI-17527


Repository: ambari


Description
---

We found out that the reason for memory leak is gc being disabled. 
We need to setup logging to figure
out who and under what circumstances actually disables gc.  
After we do that we can find QE cluster with leak and investigate the
information.

This information is essential for fixing the leak.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/main.py 201264c 

Diff: https://reviews.apache.org/r/49515/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Review Request 49514: Support Storm 1.0 in Ambari Metrics for Storm


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49514/
---

Review request for Ambari, Aravindan Vijayan, Jungtaek Lim, and Sid Wagle.


Bugs: AMBARI-17080
https://issues.apache.org/jira/browse/AMBARI-17080


Repository: ambari


Description
---

Support 2 storm sinks.


Diffs
-

  ambari-metrics/ambari-metrics-assembly/pom.xml 5e2d819 
  ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink-windows.xml 
e82d2d4 
  ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink.xml 4a3b7c5 
  ambari-metrics/ambari-metrics-storm-sink-legacy/pom.xml PRE-CREATION 
  ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/assemblies/empty.xml 
PRE-CREATION 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
 PRE-CREATION 
  ambari-metrics/ambari-metrics-storm-sink/pom.xml 1591d39 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/conf/storm-metrics2.properties.j2
 4553224 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/NumberUtil.java
 PRE-CREATION 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
 ab5f1e4 
  
ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 6ab12e1 
  
ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
 e582a95 
  ambari-metrics/pom.xml 7221ab5 
  
ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
 0d029e8 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
 073bb1c 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
 2d50767 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
 6551067 
  
ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
 9acf173 
  
ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
 f3bbce8 

Diff: https://reviews.apache.org/r/49514/diff/


Testing
---

Unit testst passed


Thanks,

Dmytro Sen

Re: Review Request 49515: Setup logging to find memory leak rootcause

2016-07-01 Thread Dmitro Lisnichenko


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49515/#review140368
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On July 1, 2016, 6:32 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49515/
> ---
> 
> (Updated July 1, 2016, 6:32 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17527
> https://issues.apache.org/jira/browse/AMBARI-17527
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> We found out that the reason for memory leak is gc being disabled. 
> We need to setup logging to figure
> out who and under what circumstances actually disables gc.  
> After we do that we can find QE cluster with leak and investigate the
> information.
> 
> This information is essential for fixing the leak.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 201264c 
> 
> Diff: https://reviews.apache.org/r/49515/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Re: Review Request 49513: Ambari API for api/v1/clusters/?fields=hosts/* does not return the metrics information


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49513/#review140369
---


Ship it!




Ship It!

- Sid Wagle


On July 1, 2016, 3:26 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49513/
> ---
> 
> (Updated July 1, 2016, 3:26 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-17526
> https://issues.apache.org/jira/browse/AMBARI-17526
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> http://:8080/api/v1/clusters/?fields=hosts/* no 
> longer returns the metrics for the hosts.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
>  9a7454c 
>   ambari-server/src/main/resources/ganglia_properties.json 05360da 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java
>  c7dabf1 
> 
> Diff: https://reviews.apache.org/r/49513/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49514: Support Storm 1.0 in Ambari Metrics for Storm


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49514/#review140370
---




ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
 (line 194)


Good catch, however, why not look for "._" and replace that only if needed.



ambari-metrics/ambari-metrics-storm-sink/pom.xml (line 35)


Since this isn't apache version, checking this in will break apache builds ?



ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
 (line 30)


So both old and new properties are ok to be present at the same time?



ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
 (line 96)


No check for stack name?


- Sid Wagle


On July 1, 2016, 3:32 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49514/
> ---
> 
> (Updated July 1, 2016, 3:32 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jungtaek Lim, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17080
> https://issues.apache.org/jira/browse/AMBARI-17080
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Support 2 storm sinks.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-assembly/pom.xml 5e2d819 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink-windows.xml 
> e82d2d4 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink.xml 4a3b7c5 
>   ambari-metrics/ambari-metrics-storm-sink-legacy/pom.xml PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/assemblies/empty.xml 
> PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  PRE-CREATION 
>   ambari-metrics/ambari-metrics-storm-sink/pom.xml 1591d39 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/conf/storm-metrics2.properties.j2
>  4553224 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/NumberUtil.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  ab5f1e4 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  6ab12e1 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  e582a95 
>   ambari-metrics/pom.xml 7221ab5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
>  0d029e8 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  2d50767 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
>  6551067 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
>  9acf173 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
>  f3bbce8 
> 
> Diff: https://reviews.apache.org/r/49514/diff/
> 
> 
> Testing
> ---
> 
> Unit testst passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49510: Zeppelin: remove Livy component dependency

2016-07-01 Thread Prabhjyot Singh


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/#review140372
---


Ship it!




Ship It!

- Prabhjyot Singh


On July 1, 2016, 2:23 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49510/
> ---
> 
> (Updated July 1, 2016, 2:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17523
> https://issues.apache.org/jira/browse/AMBARI-17523
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - remove Livy component dependency from metainfo.xml
> - code clean up
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  4f19b43 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  fd6cbb6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  936df81 
> 
> Diff: https://reviews.apache.org/r/49510/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49510: Zeppelin: remove Livy component dependency



> On July 1, 2016, 2:29 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py,
> >  line 204
> > 
> >
> > Is this tested in non-root scenarios?
> > You can discuss with Andrew O to see if this will pose an issue for 
> > non-root scenarios.

I'm re-testing this. Earlier this was tested for non-root and was working. Now 
we have a few permission updates from the package in post install script.


- Renjith


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/#review140357
---


On July 1, 2016, 2:23 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49510/
> ---
> 
> (Updated July 1, 2016, 2:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17523
> https://issues.apache.org/jira/browse/AMBARI-17523
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - remove Livy component dependency from metainfo.xml
> - code clean up
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  4f19b43 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  fd6cbb6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  936df81 
> 
> Diff: https://reviews.apache.org/r/49510/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49510: Zeppelin: remove Livy component dependency



> On July 1, 2016, 2:27 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml,
> >  line 42
> > 
> >
> > I understand that this is not honored by UI and only blueprint looks 
> > into this - why should this be removed?

Livy install is going to be handled in the documentation. This is removed for 
consistency.


- Renjith


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/#review140356
---


On July 1, 2016, 2:23 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49510/
> ---
> 
> (Updated July 1, 2016, 2:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17523
> https://issues.apache.org/jira/browse/AMBARI-17523
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - remove Livy component dependency from metainfo.xml
> - code clean up
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  4f19b43 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  fd6cbb6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  936df81 
> 
> Diff: https://reviews.apache.org/r/49510/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49514: Support Storm 1.0 in Ambari Metrics for Storm



> On Июль 1, 2016, 3:54 п.п., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-storm-sink/pom.xml, line 35
> > 
> >
> > Since this isn't apache version, checking this in will break apache 
> > builds ?

Yes it will. we need storm-core 1.0.1.2.5.0.0-830 in public maven repo before 
commiting


> On Июль 1, 2016, 3:54 п.п., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml,
> >  line 30
> > 
> >
> > So both old and new properties are ok to be present at the same time?

yes


> On Июль 1, 2016, 3:54 п.п., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py,
> >  line 96
> > 
> >
> > No check for stack name?

I'll fix


> On Июль 1, 2016, 3:54 п.п., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java,
> >  line 194
> > 
> >
> > Good catch, however, why not look for "._" and replace that only if 
> > needed.

There is no change here. It's a copy of previous StormTimelineMetricsSink.java


- Dmytro


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49514/#review140370
---


On Июль 1, 2016, 3:32 п.п., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49514/
> ---
> 
> (Updated Июль 1, 2016, 3:32 п.п.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jungtaek Lim, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17080
> https://issues.apache.org/jira/browse/AMBARI-17080
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Support 2 storm sinks.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-assembly/pom.xml 5e2d819 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink-windows.xml 
> e82d2d4 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink.xml 4a3b7c5 
>   ambari-metrics/ambari-metrics-storm-sink-legacy/pom.xml PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/assemblies/empty.xml 
> PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  PRE-CREATION 
>   ambari-metrics/ambari-metrics-storm-sink/pom.xml 1591d39 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/conf/storm-metrics2.properties.j2
>  4553224 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/NumberUtil.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  ab5f1e4 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  6ab12e1 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  e582a95 
>   ambari-metrics/pom.xml 7221ab5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
>  0d029e8 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  2d50767 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
>  6551067 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
>  9acf173 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
>  f3bbce8 
> 
> Diff: https://reviews.apache.org/r/49514/diff/
> 
> 
> Testing
> ---
> 
> Unit testst passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49474: Ambari LogSearch Integration should request logging metadata on a separate thread

2016-07-01 Thread Robert Nettleton

> On July 1, 2016, 2:31 p.m., Jonathan Hurley wrote:
> >

Thanks for the review! 

I'll be uploading an updated patch shortly with the fixes for these issues.

> On July 1, 2016, 2:31 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProvider.java,
> >  lines 92-93
> > 
> >
> > This is interesting; how would this service not be available if it's 
> > injected?

There's no specific case that might cause this reference to be null.

There are two reasons I left this check for null in the code:

1. I moved the access to the LoggingRequestHelper to the retrieval service, and 
so this change allowed me to keep the code changes to a minimum in order to 
introduce the new service.
2. I've noticed that our DI system depends a lot on object ordering, and I've 
seen instances in the past where an object is silently not injected, which 
could cause a NullPointerException.  I'd like to keep this check in place, to 
guard against future object graph changes causing an NPE in this area of the 
code.

> On July 1, 2016, 2:31 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java,
> >  line 79
> > 
> >
> > Based on what we've seen, the requests to the LOGSEARCH endpoint don't 
> > return until the connection times out (which could be never depending on 
> > what's configured). This will then cause this Executor to fill it's queue 
> > with a backlog of requests.
> > 
> > Instead, maybe it would be better to:
> > - Only enqueue if the request isn't already enqueued
> > - Use a bounded executor and rejection policy

I agree with your suggestion, but would like to address this issue in a 
separate patch. 

I've filed the following JIRA to track this effort:

https://issues.apache.org/jira/browse/AMBARI-17529

> On July 1, 2016, 2:31 p.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java,
> >  lines 149-151
> > 
> >
> > Maybe only enqueue requests if the same request isn't on the queue.

I agree with your suggestion, but would like to address this issue in a 
separate patch. 

I've filed the following JIRA to track this effort:

https://issues.apache.org/jira/browse/AMBARI-17529

On July 1, 2016, 2:31 p.m., Robert Nettleton wrote:
> > I'm a bit confused by the changes made here, namely that 
> > `sendLogLevelQueryRequest` was replaced with `sendGetLogFileNamesRequest`. 
> > Is `sendLogLevelQueryRequest` no longer used? If so, it can be removed, 
> > right?

Thanks for catching this.

The calls to sendLogLevelQueryRequest are removed, since the Ambari UI will not 
use these calls in Ambari 2.4.  I left the underlying code in the 
LoggingRequestHelper's implementation, since I'd like to re-introduce this call 
(in a more performant way) in Ambari 2.5.  Since that code is also unit-tested, 
it seemed to make sense to leave it in to be used in the next release.

- Robert

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49474/#review140353
---

On June 30, 2016, 9:11 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49474/
> ---
> 
> (Updated June 30, 2016, 9:11 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Mahadev Konar, Oliver Szabo, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17510
> https://issues.apache.org/jira/browse/AMBARI-17510
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch implements some updates to the Ambari LogSearch integration layer 
> in order to resolve AMBARI-17510. 
> 
> The initial implementation of the LoggingSearchPropertyProvider directly made 
> calls to the LogSearch Server's REST endpoint, in order to obtain LogSearch 
> metadata, and to attach this information to the Ambari HostComponent REST 
> resource.  Recent testing has shown that there are some deployment scenarios 
> (larger clusters, rolling upgrade) that demonstrate a performance problem in 
> the LogSearch integration code.  
> 
> This patch addresses this problem by implementing the following:
> 
> 1. A new, injectable service 
> (org.apache.ambari.server.controller.logging.LogSearchDataRetrievalService) 
> has been introduced into the LoggingSearchPropertyProvider.  The property 
> provider now consults this service to obtain the required LogSearch data.  
> T

Re: Review Request 49474: Ambari LogSearch Integration should request logging metadata on a separate thread

2016-07-01 Thread Robert Nettleton

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49474/
---

(Updated July 1, 2016, 4:25 p.m.)

Review request for Ambari, Jonathan Hurley, Mahadev Konar, Oliver Szabo, and
Sumit Mohanty.

Changes
---

Updated patch to include review feedback.

Bugs: AMBARI-17510
https://issues.apache.org/jira/browse/AMBARI-17510

Repository: ambari

Description
---

This patch implements some updates to the Ambari LogSearch integration layer in
order to resolve AMBARI-17510.

The initial implementation of the LoggingSearchPropertyProvider directly made
calls to the LogSearch Server's REST endpoint, in order to obtain LogSearch
metadata, and to attach this information to the Ambari HostComponent REST
resource. Recent testing has shown that there are some deployment scenarios
(larger clusters, rolling upgrade) that demonstrate a performance problem in
the LogSearch integration code.

This patch addresses this problem by implementing the following:

1. A new, injectable service
(org.apache.ambari.server.controller.logging.LogSearchDataRetrievalService) has
been introduced into the LoggingSearchPropertyProvider. The property provider
now consults this service to obtain the required LogSearch data. This new
service is based on Jonathan Hurley's work in
org.apache.ambari.server.state.services.MetricsRetrievalService.
2. The LogSearchDataRetrievalService utilizes an internal cache to maintain a
map of the LogSearch data, which typically will not change. If the cache does
not contain this information for a given HostComponent, the retrieval service
will make the required REST call to the LogSearch service to obtain this data,
and then add this to the cache. The remote REST call now happens on a separate
thread from the Ambari REST request handler, which removes the possibility of
causing degradation on the Ambari REST endpoing.
3. The LoggingRequestHelperImpl class has been slightly updated, in order to
add 5 second connect/read timeouts on the LogSearch requests. This should
prevent any threads from hanging infinitely if the LogSearch server is slow to
respond. In a future patch, we might want to make these timeouts configurable.

Diffs (updated)
-

ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
947a9f4

ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
fe7e757

ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
5ac66d8

ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
PRE-CREATION

ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImpl.java
d8c71e2

ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProvider.java
ff7e7f5

ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProviderTest.java
593f660

Diff: https://reviews.apache.org/r/49474/diff/

Testing
---

1. Ran the ambari-server "mvn clean test" suite, and all Java and Python unit
tests passed with this change applied.
2. Deployed a 3-node cluster with HDFS, Yarn, and LogSearch selected with this
patch applied. Verified that the cluster deployed successfully, and that the
"host_components" REST resource (the resource associated with this performance
problem) returns properly. Ran 6-7 separate REST clients making concurrent
requests on the "host_componets" resource in Ambari for over 40 minutes, and
verified that there is no noticeable slowdown. Verified that ambari-server
remains responsive after this test. Also verified via DEBUG-level logging that
the LogSearch remote REST requests are indeed happening on a separate thread
now.
3. Deployed a 3-node cluster with HDFS and Yarn selected (LogSearch not
selected) with this patch applied. Verified that the cluster deploys properly,
and that the Ambari REST "host_components" resource returns properly when
LogSearch is not included in the cluster.

Thanks,

Robert Nettleton

Re: Review Request 49474: Ambari LogSearch Integration should request logging metadata on a separate thread

2016-07-01 Thread Jonathan Hurley



On July 1, 2016, 10:31 a.m., Robert Nettleton wrote:
> > I'm a bit confused by the changes made here, namely that 
> > `sendLogLevelQueryRequest` was replaced with `sendGetLogFileNamesRequest`. 
> > Is `sendLogLevelQueryRequest` no longer used? If so, it can be removed, 
> > right?
> 
> Robert Nettleton wrote:
> Thanks for catching this.
> 
> The calls to sendLogLevelQueryRequest are removed, since the Ambari UI 
> will not use these calls in Ambari 2.4.  I left the underlying code in the 
> LoggingRequestHelper's implementation, since I'd like to re-introduce this 
> call (in a more performant way) in Ambari 2.5.  Since that code is also 
> unit-tested, it seemed to make sense to leave it in to be used in the next 
> release.

Makes sense; thanks for the explanation. I just wanted to make sure it wasn't 
something that was by accident.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49474/#review140353
---


On July 1, 2016, 12:25 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49474/
> ---
> 
> (Updated July 1, 2016, 12:25 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Mahadev Konar, Oliver Szabo, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17510
> https://issues.apache.org/jira/browse/AMBARI-17510
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch implements some updates to the Ambari LogSearch integration layer 
> in order to resolve AMBARI-17510. 
> 
> The initial implementation of the LoggingSearchPropertyProvider directly made 
> calls to the LogSearch Server's REST endpoint, in order to obtain LogSearch 
> metadata, and to attach this information to the Ambari HostComponent REST 
> resource.  Recent testing has shown that there are some deployment scenarios 
> (larger clusters, rolling upgrade) that demonstrate a performance problem in 
> the LogSearch integration code.  
> 
> This patch addresses this problem by implementing the following:
> 
> 1. A new, injectable service 
> (org.apache.ambari.server.controller.logging.LogSearchDataRetrievalService) 
> has been introduced into the LoggingSearchPropertyProvider.  The property 
> provider now consults this service to obtain the required LogSearch data.  
> This new service is based on Jonathan Hurley's work in 
> org.apache.ambari.server.state.services.MetricsRetrievalService.  
> 2. The LogSearchDataRetrievalService utilizes an internal cache to maintain a 
> map of the LogSearch data, which typically will not change.  If the cache 
> does not contain this information for a given HostComponent, the retrieval 
> service will make the required REST call to the LogSearch service to obtain 
> this data, and then add this to the cache.  The remote REST call now happens 
> on a separate thread from the Ambari REST request handler, which removes the 
> possibility of causing degradation on the Ambari REST endpoing. 
> 3. The LoggingRequestHelperImpl class has been slightly updated, in order to 
> add 5 second connect/read timeouts on the LogSearch requests.  This should 
> prevent any threads from hanging infinitely if the LogSearch server is slow 
> to respond.  In a future patch, we might want to make these timeouts 
> configurable.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  947a9f4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  fe7e757 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  5ac66d8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImpl.java
>  d8c71e2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProvider.java
>  ff7e7f5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProviderTest.java
>  593f660 
> 
> Diff: https://reviews.apache.org/r/49474/diff/
> 
> 
> Testing
> ---
> 
> 1. Ran the ambari-server "mvn clean test" suite, and all Java and Python unit 
> tests passed with this change applied.
> 2. Deployed a 3-node cluster with HDFS, Yarn, and LogSearch selected with 
> this patch applied. Verified that the cluster deployed successfully, and that 
> the "host_components" REST resource (the resource associated with this 
> performance problem) returns properly.  Ran 6-7 separate REST clients making 
> concurrent requests on the "host_

Re: Review Request 49474: Ambari LogSearch Integration should request logging metadata on a separate thread

2016-07-01 Thread Jonathan Hurley


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49474/#review140379
---


Ship it!




Ship It!

- Jonathan Hurley


On July 1, 2016, 12:25 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49474/
> ---
> 
> (Updated July 1, 2016, 12:25 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Mahadev Konar, Oliver Szabo, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17510
> https://issues.apache.org/jira/browse/AMBARI-17510
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch implements some updates to the Ambari LogSearch integration layer 
> in order to resolve AMBARI-17510. 
> 
> The initial implementation of the LoggingSearchPropertyProvider directly made 
> calls to the LogSearch Server's REST endpoint, in order to obtain LogSearch 
> metadata, and to attach this information to the Ambari HostComponent REST 
> resource.  Recent testing has shown that there are some deployment scenarios 
> (larger clusters, rolling upgrade) that demonstrate a performance problem in 
> the LogSearch integration code.  
> 
> This patch addresses this problem by implementing the following:
> 
> 1. A new, injectable service 
> (org.apache.ambari.server.controller.logging.LogSearchDataRetrievalService) 
> has been introduced into the LoggingSearchPropertyProvider.  The property 
> provider now consults this service to obtain the required LogSearch data.  
> This new service is based on Jonathan Hurley's work in 
> org.apache.ambari.server.state.services.MetricsRetrievalService.  
> 2. The LogSearchDataRetrievalService utilizes an internal cache to maintain a 
> map of the LogSearch data, which typically will not change.  If the cache 
> does not contain this information for a given HostComponent, the retrieval 
> service will make the required REST call to the LogSearch service to obtain 
> this data, and then add this to the cache.  The remote REST call now happens 
> on a separate thread from the Ambari REST request handler, which removes the 
> possibility of causing degradation on the Ambari REST endpoing. 
> 3. The LoggingRequestHelperImpl class has been slightly updated, in order to 
> add 5 second connect/read timeouts on the LogSearch requests.  This should 
> prevent any threads from hanging infinitely if the LogSearch server is slow 
> to respond.  In a future patch, we might want to make these timeouts 
> configurable.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  947a9f4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  fe7e757 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  5ac66d8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImpl.java
>  d8c71e2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProvider.java
>  ff7e7f5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProviderTest.java
>  593f660 
> 
> Diff: https://reviews.apache.org/r/49474/diff/
> 
> 
> Testing
> ---
> 
> 1. Ran the ambari-server "mvn clean test" suite, and all Java and Python unit 
> tests passed with this change applied.
> 2. Deployed a 3-node cluster with HDFS, Yarn, and LogSearch selected with 
> this patch applied. Verified that the cluster deployed successfully, and that 
> the "host_components" REST resource (the resource associated with this 
> performance problem) returns properly.  Ran 6-7 separate REST clients making 
> concurrent requests on the "host_componets" resource in Ambari for over 40 
> minutes, and verified that there is no noticeable slowdown.  Verified that 
> ambari-server remains responsive after this test. Also verified via 
> DEBUG-level logging that the LogSearch remote REST requests are indeed 
> happening on a separate thread now.  
> 3. Deployed a 3-node cluster with HDFS and Yarn selected (LogSearch not 
> selected) with this patch applied.  Verified that the cluster deploys 
> properly, and that the Ambari REST "host_components" resource returns 
> properly when LogSearch is not included in the cluster.
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>

Re: Review Request 49474: Ambari LogSearch Integration should request logging metadata on a separate thread

2016-07-01 Thread Oliver Szabo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49474/#review140381
---


Ship it!




Ship It!

- Oliver Szabo


On July 1, 2016, 4:25 p.m., Robert Nettleton wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49474/
> ---
> 
> (Updated July 1, 2016, 4:25 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Mahadev Konar, Oliver Szabo, and 
> Sumit Mohanty.
> 
> 
> Bugs: AMBARI-17510
> https://issues.apache.org/jira/browse/AMBARI-17510
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> This patch implements some updates to the Ambari LogSearch integration layer 
> in order to resolve AMBARI-17510. 
> 
> The initial implementation of the LoggingSearchPropertyProvider directly made 
> calls to the LogSearch Server's REST endpoint, in order to obtain LogSearch 
> metadata, and to attach this information to the Ambari HostComponent REST 
> resource.  Recent testing has shown that there are some deployment scenarios 
> (larger clusters, rolling upgrade) that demonstrate a performance problem in 
> the LogSearch integration code.  
> 
> This patch addresses this problem by implementing the following:
> 
> 1. A new, injectable service 
> (org.apache.ambari.server.controller.logging.LogSearchDataRetrievalService) 
> has been introduced into the LoggingSearchPropertyProvider.  The property 
> provider now consults this service to obtain the required LogSearch data.  
> This new service is based on Jonathan Hurley's work in 
> org.apache.ambari.server.state.services.MetricsRetrievalService.  
> 2. The LogSearchDataRetrievalService utilizes an internal cache to maintain a 
> map of the LogSearch data, which typically will not change.  If the cache 
> does not contain this information for a given HostComponent, the retrieval 
> service will make the required REST call to the LogSearch service to obtain 
> this data, and then add this to the cache.  The remote REST call now happens 
> on a separate thread from the Ambari REST request handler, which removes the 
> possibility of causing degradation on the Ambari REST endpoing. 
> 3. The LoggingRequestHelperImpl class has been slightly updated, in order to 
> add 5 second connect/read timeouts on the LogSearch requests.  This should 
> prevent any threads from hanging infinitely if the LogSearch server is slow 
> to respond.  In a future patch, we might want to make these timeouts 
> configurable.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementController.java
>  947a9f4 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariManagementControllerImpl.java
>  fe7e757 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/AbstractProviderModule.java
>  5ac66d8 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LogSearchDataRetrievalService.java
>  PRE-CREATION 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingRequestHelperImpl.java
>  d8c71e2 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProvider.java
>  ff7e7f5 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/logging/LoggingSearchPropertyProviderTest.java
>  593f660 
> 
> Diff: https://reviews.apache.org/r/49474/diff/
> 
> 
> Testing
> ---
> 
> 1. Ran the ambari-server "mvn clean test" suite, and all Java and Python unit 
> tests passed with this change applied.
> 2. Deployed a 3-node cluster with HDFS, Yarn, and LogSearch selected with 
> this patch applied. Verified that the cluster deployed successfully, and that 
> the "host_components" REST resource (the resource associated with this 
> performance problem) returns properly.  Ran 6-7 separate REST clients making 
> concurrent requests on the "host_componets" resource in Ambari for over 40 
> minutes, and verified that there is no noticeable slowdown.  Verified that 
> ambari-server remains responsive after this test. Also verified via 
> DEBUG-level logging that the LogSearch remote REST requests are indeed 
> happening on a separate thread now.  
> 3. Deployed a 3-node cluster with HDFS and Yarn selected (LogSearch not 
> selected) with this patch applied.  Verified that the cluster deploys 
> properly, and that the Ambari REST "host_components" resource returns 
> properly when LogSearch is not included in the cluster.
> 
> 
> Thanks,
> 
> Robert Nettleton
> 
>

Review Request 49519: Fix Logfeeder visibilities

2016-07-01 Thread Miklos Gergely


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49519/
---

Review request for Ambari, Hayat Behlim, Oliver Szabo, Robert Nettleton, and 
Sumit Mohanty.


Bugs: AMBARI-17528
https://issues.apache.org/jira/browse/AMBARI-17528


Repository: ambari


Description
---

Many variables/functions have a broader visibility than needed.

Also remvoed some more unused code.


Diffs
-

  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/AliasUtil.java
 c3b36df 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/ConfigBlock.java
 088472e 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/InputMgr.java
 d83642e 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeeder.java
 c5d4fd5 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederAMSClient.java
 1b9171b 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/LogFeederUtil.java
 78d0499 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/MetricsMgr.java
 4a8f7d0 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/OutputMgr.java
 f6d3481 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/Filter.java
 d34eed6 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterGrok.java
 31e1cd1 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/filter/FilterKeyValue.java
 7b1e5e0 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/Input.java
 2e38255 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputFile.java
 3538ba0 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/InputS3File.java
 9d5f970 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/input/reader/GZIPReader.java
 7c455f6 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/FetchConfigFromSolr.java
 4833d3f 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/filter/ApplyLogFilter.java
 c71d4b9 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/logconfig/filter/DefaultDataFilter.java
 e67512b 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/Mapper.java
 b87ce50 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperDate.java
 f293ede 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperFieldName.java
 afbb126 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/mapper/MapperFieldValue.java
 00a69df 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/output/Output.java
 0624c59 
  
ambari-logsearch/ambari-logsearch-logfeeder/src/main/java/org/apache/ambari/logfeeder/util/SolrUtil.java
 19dd404 

Diff: https://reviews.apache.org/r/49519/diff/


Testing
---

Tested on local cluster.

Tests run: 36, Failures: 0, Errors: 0, Skipped: 0


Thanks,

Miklos Gergely

Re: Review Request 49287: After switching to external database in hive, user should be allowed to delete mysql server

2016-07-01 Thread Alexandr Antonenko



> On July 1, 2016, 2:58 p.m., Alexandr Antonenko wrote:
> > Ship It!
> 
> Anita Jebaraj wrote:
> can you please help in pushing the code.

done


- Alexandr


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49287/#review140362
---


On June 27, 2016, 11:20 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49287/
> ---
> 
> (Updated June 27, 2016, 11:20 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Di Li.
> 
> 
> Bugs: AMBARI-17358
> https://issues.apache.org/jira/browse/AMBARI-17358
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> When user switches from MySQL to Existing database in hive and stop the MySQL 
> Server
> 
> 1) Ambari UI > Hive > Service Actions > Stop becomes unavailable, so Hive 
> can't be stopped anymore
> 
> 2) Hive status is showing up as red Triangle indicating there is a problem 
> with Hive
> 
> Providing an option to delete MySQL server when an existing database is used 
> in Hive would help in overcoming the issues.
> 
> 
> Solution:
> An option is included to delete MySQL Server, the delete will be enabled only 
> when Hive is using an existing(or external) database and the MySQL server is 
> not started.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/stack_service_component.js a350f15 
>   ambari-web/app/routes/main.js 4545f54 
>   ambari-web/app/views/main/host/details/host_component_view.js 910c71f 
>   ambari-web/test/models/stack_service_component_test.js 7e971ce 
>   ambari-web/test/views/main/host/details/host_component_view_test.js 648f0f6 
> 
> Diff: https://reviews.apache.org/r/49287/diff/
> 
> 
> Testing
> ---
> 
> Ran mvn test
> 
> 28974 tests complete (43 seconds)
>   154 tests pending
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 49385: Hive and Oozie db displayed incorrectly on the installer review page

2016-07-01 Thread Sangeeta Ravindran



> On June 30, 2016, 10:52 a.m., Andrii Tkach wrote:
> > ambari-web/test/controllers/wizard/step8_test.js, line 898
> > 
> >
> > Remove unnecessary logs

I have removed this in the new patch.


> On June 30, 2016, 10:52 a.m., Andrii Tkach wrote:
> > ambari-web/test/controllers/wizard/step8_test.js, line 924
> > 
> >
> > stub and restore should be placed in beforeEach and afterEach 
> > correspondingly. Use this.mock = sinon.stub(...) and then this.mock.returns 
> > in it(...)

Thanks. I have made the changes that you suggested.


- Sangeeta


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49385/#review140134
---


On June 30, 2016, 8:57 p.m., Sangeeta Ravindran wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49385/
> ---
> 
> (Updated June 30, 2016, 8:57 p.m.)
> 
> 
> Review request for Ambari, Alexandr Antonenko and Andrii Tkach.
> 
> 
> Bugs: AMBARI-17469
> https://issues.apache.org/jira/browse/AMBARI-17469
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During Hive install, in the review page, the default value of 
> hive_admin_database (MySQL) is concatenated to the selected database type, no 
> matter which database is selected.  For e.g. if Existing PostgreSQL Database 
> is selected as the Hive database, the review page displays the following for 
> Hive database: 
> 
> Database : MySQL (Existing PostgreSQL Database)
>  
> In case of Oozie, because there is no oozie_admin_database property, a blank 
> is displayed for database although an existing database was selected
>  
> Database :  
>  
> This seems to be because of the logic in the method that determines the 
> database value to be displayed.
>  
> var dbFull = serviceConfigProperties.findProperty('name', 
> serviceName.toLowerCase() + '_database'),
>  db = serviceConfigProperties.findProperty('name', 
> serviceName.toLowerCase() + '_ambari_database');
> return db && dbFull ? db.value + ' (' + dbFull.value + ')' : '';
> 
> The value of hive_ambari_database returns MySQL and hence in case of Hive, 
> MySQL always gets appended.
>  
> There is no oozie_ambari_database property defined. Hence db is undefined and 
> an emtpy string is returned instead of the actual database type selected.
>  
> Fix involves changing the logic to not include the value of 
> serviceName_ambari_database since it will not have the right value unless the 
> default value is selected for Hive/Oozie database.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/controllers/wizard/step8_controller.js 3971cf5 
>   ambari-web/test/controllers/wizard/step8_test.js 74e042b 
> 
> Diff: https://reviews.apache.org/r/49385/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> Added a test case to verify the value displayed for database.
> Ran mvn test
> 
> 28979 tests complete (48 seconds)
> 154 tests pending
> 
> 
> File Attachments
> 
> 
> Updated Patch with review comments incorporated
>   
> https://reviews.apache.org/media/uploaded/files/2016/06/30/e5fa58fa-3940-4c82-ad92-c8070c824528__AMBARI-17469.patch
> 
> 
> Thanks,
> 
> Sangeeta Ravindran
> 
>

Re: Review Request 49514: Support Storm 1.0 in Ambari Metrics for Storm


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49514/#review140384
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
 (line 40)


Does the parallelism = 1, mean 1 bolt getting all metric reports in a 
topology. Is it counter-intuitive for performance, something we should check 
with storm group.


- Sid Wagle


On July 1, 2016, 3:32 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49514/
> ---
> 
> (Updated July 1, 2016, 3:32 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jungtaek Lim, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17080
> https://issues.apache.org/jira/browse/AMBARI-17080
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Support 2 storm sinks.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-assembly/pom.xml 5e2d819 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink-windows.xml 
> e82d2d4 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink.xml 4a3b7c5 
>   ambari-metrics/ambari-metrics-storm-sink-legacy/pom.xml PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/assemblies/empty.xml 
> PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  PRE-CREATION 
>   ambari-metrics/ambari-metrics-storm-sink/pom.xml 1591d39 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/conf/storm-metrics2.properties.j2
>  4553224 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/NumberUtil.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  ab5f1e4 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  6ab12e1 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  e582a95 
>   ambari-metrics/pom.xml 7221ab5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
>  0d029e8 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  2d50767 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
>  6551067 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
>  9acf173 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
>  f3bbce8 
> 
> Diff: https://reviews.apache.org/r/49514/diff/
> 
> 
> Testing
> ---
> 
> Unit testst passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 49521: Move service advisor tests for HAWQ and PXF

2016-07-01 Thread Lav Jain


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49521/
---

Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
Luniya, Matt, and Tim Thorpe.


Bugs: AMBARI-16647
https://issues.apache.org/jira/browse/AMBARI-16647


Repository: ambari


Description
---

Currently the service advisor tests for HAWQ and PXF are under 
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py
Move the tests to the service specific service_advisor files:
ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py
ambari-server/src/test/python/common-services/PXF/test_service_advisor.py


Diffs
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
f0e8b33 
  ambari-server/src/main/resources/stacks/stack_advisor.py d8685c3 
  
ambari-server/src/test/python/common-services/HAWQ/test_alert_component_status.py
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/HAWQ/test_alert_segment_registration_status.py
 PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_alert_sync_status.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_hawqmaster.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_hawqsegment.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_hawqstandby.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_utils.py PRE-CREATION 
  ambari-server/src/test/python/common-services/PXF/test_alerts_api_status.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/PXF/test_pxf.py PRE-CREATION 
  ambari-server/src/test/python/common-services/PXF/test_service_advisor.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/configs/hawq_default.json 
PRE-CREATION 
  ambari-server/src/test/python/common-services/configs/hosts-1-host.json 
PRE-CREATION 
  ambari-server/src/test/python/common-services/configs/hosts-3-hosts.json 
PRE-CREATION 
  ambari-server/src/test/python/common-services/configs/pxf_default.json 
PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-hawq-1-host.json 
PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-hawq-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-hawq-pxf-hdfs.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-master_ambari_colo-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-master_standby_colo-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-nohawq-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-normal-hawq-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-normal-nohawq-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-standby_ambari_colo-3-hosts.json
 PRE-CREATION 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_component_status.py 
b2e1d4d 
  
ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_segment_registration_status.py
 6bb5930 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_sync_status.py 
fd4f474 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 88fb008 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 8639ca5 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py b406723 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 12f4fa1 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_utils.py 07737eb 
  ambari-server/src/test/python/stacks/2.3/PXF/test_alerts_api_status.py 
ee187e2 
  ambari-server/src/test/python/stacks/2.3/PXF/test_pxf.py 1147a7e 
  ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py 4ea3bfb 
  ambari-server/src/test/python/stacks/2.3/common/hosts-1-host.json 5d6a5b6 
  ambari-server/src/test/python/stacks/2.3/common/hosts-3-hosts.json 3c0511e 
  ambari-server/src/test/python/stacks/2.3/common/services-hawq-1-host.json 
515ba7d 
  ambari-server/src/test/python/stacks/2.3/common/services-hawq-3-hosts.json 
515ba7d 
  ambari-server/src/test/python/stacks/2.3/common/services-hawq-pxf-hdfs.json 
0bf459d 
  
ambari-server/src/test/python/stacks/2.3/common/services-master_ambari_colo-3-hosts.json
 1657ccf 
  
ambari-server/src/test/python/stacks/2.3/common/services-master_standby_colo-3-hosts.json
 cd5d02c 
  ambari-server/src/test/python/stacks/2.3/common/services-nohawq-3-hosts.json 
beeb62d 
  
ambari-server/src/test/python/stacks/2.3/common/services-normal-hawq-3-hosts.json
 5495d77 
  
ambari-server/src/test/python

Re: Review Request 49455: Optimized classpath scannig for upgrade check impelementations

2016-07-01 Thread Sebastian Toader


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49455/#review140389
---


Ship it!




Ship It!

- Sebastian Toader


On July 1, 2016, 2:12 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49455/
> ---
> 
> (Updated July 1, 2016, 2:12 p.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Sumit Mohanty, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-17505
> https://issues.apache.org/jira/browse/AMBARI-17505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> During startup the ambari server scasns the classpath for finding components 
> to be bound in the IoC context.
> When binding upgrade check implementations the full ambari package is scanned 
> that leads to prolonged startup time.
> 
> Solution:
> As upgrade check implementations reside in a dedicated package, the scanner 
> is modified to lookup them in this very package.
> (on the local env this shortens the startup time by ~25 seconds)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  e0bda13 
> 
> Diff: https://reviews.apache.org/r/49455/diff/
> 
> 
> Testing
> ---
> 
> Unit tests running.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 49455: Optimized classpath scannig for upgrade check impelementations

2016-07-01 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49455/#review140390
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 1, 2016, 12:12 p.m., Laszlo Puskas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49455/
> ---
> 
> (Updated July 1, 2016, 12:12 p.m.)
> 
> 
> Review request for Ambari, Daniel Gergely, Sumit Mohanty, and Sebastian 
> Toader.
> 
> 
> Bugs: AMBARI-17505
> https://issues.apache.org/jira/browse/AMBARI-17505
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Problem:
> During startup the ambari server scasns the classpath for finding components 
> to be bound in the IoC context.
> When binding upgrade check implementations the full ambari package is scanned 
> that leads to prolonged startup time.
> 
> Solution:
> As upgrade check implementations reside in a dedicated package, the scanner 
> is modified to lookup them in this very package.
> (on the local env this shortens the startup time by ~25 seconds)
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/ControllerModule.java
>  e0bda13 
> 
> Diff: https://reviews.apache.org/r/49455/diff/
> 
> 
> Testing
> ---
> 
> Unit tests running.
> 
> 
> Thanks,
> 
> Laszlo Puskas
> 
>

Re: Review Request 49510: Zeppelin: remove Livy component dependency



> On July 1, 2016, 2:29 p.m., Sumit Mohanty wrote:
> > ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py,
> >  line 204
> > 
> >
> > Is this tested in non-root scenarios?
> > You can discuss with Andrew O to see if this will pose an issue for 
> > non-root scenarios.
> 
> Renjith Kamath wrote:
> I'm re-testing this. Earlier this was tested for non-root and was 
> working. Now we have a few permission updates from the package in post 
> install script.

This works fine for non-root user scenarios.


- Renjith


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/#review140357
---


On July 1, 2016, 2:23 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49510/
> ---
> 
> (Updated July 1, 2016, 2:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17523
> https://issues.apache.org/jira/browse/AMBARI-17523
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - remove Livy component dependency from metainfo.xml
> - code clean up
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  4f19b43 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  fd6cbb6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  936df81 
> 
> Diff: https://reviews.apache.org/r/49510/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49510: Zeppelin: remove Livy component dependency

2016-07-01 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49510/#review140392
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 1, 2016, 2:23 p.m., Renjith Kamath wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49510/
> ---
> 
> (Updated July 1, 2016, 2:23 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, DIPAYAN BHOWMICK, Gaurav 
> Nagar, Pallav Kulshreshtha, Prabhjyot Singh, Rohit Choudhary, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-17523
> https://issues.apache.org/jira/browse/AMBARI-17523
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> - remove Livy component dependency from metainfo.xml
> - code clean up
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/metainfo.xml
>  4f19b43 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  fd6cbb6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  936df81 
> 
> Diff: https://reviews.apache.org/r/49510/diff/
> 
> 
> Testing
> ---
> 
> manually tested on CentOS
> 
> 
> Thanks,
> 
> Renjith Kamath
> 
>

Re: Review Request 49507: AMBARI-17520: Update the policy_user property to use storm user principal specified in Storms config

2016-07-01 Thread Velmurugan Periasamy



> On July 1, 2016, 1:45 p.m., Robert Levas wrote:
> > Was this tested with the Storm Kerberos identity set to something like 
> > `storm1234@${realm}`?

Looks like it is working fine. I also applied the patch and tried enabling 
storm plugin. Noticed Ranger policies were created with permissions to modified 
principal.


> On July 1, 2016, 1:45 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py,
> >  line 303
> > 
> >
> > Why are we hard-coding `{storm-user}-{cluster-name}` here?   If it is 
> > related to the Storm Kerberos identitiy, then there is no guarentee that 
> > the user won't change this when configuring Kerberos identities.

I think this is the standard format for storm values in Ambari kerberos config. 
I will let Gautam confirm that.


- Velmurugan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49507/#review140346
---


On July 1, 2016, 1:14 p.m., Gautam Borad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49507/
> ---
> 
> (Updated July 1, 2016, 1:14 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Mugdha Varadkar, Robert 
> Levas, Sriharsha Chintalapani, Srimanth Gunturi, and Velmurugan Periasamy.
> 
> 
> Bugs: AMBARI-17520
> https://issues.apache.org/jira/browse/AMBARI-17520
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Update the policy_user property in Advanced ranger-storm-plugin-properties of 
> Ranger with the value of the storm user bare principal specified in Storms 
> Ambari config.
> With this the principal used for storm will also be added to default ranger 
> policy and will prevent Storm service check failures.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.3/configuration/ranger-storm-plugin-properties.xml
>  2fee04f 
> 
> Diff: https://reviews.apache.org/r/49507/diff/
> 
> 
> Testing
> ---
> 
> Tested Ranger storm plugin on centos6 cluster. Kerberized the cluster and 
> checked that Storm service check is working fine.
> 
> 
> Thanks,
> 
> Gautam Borad
> 
>

Re: Review Request 49521: Move service advisor tests for HAWQ and PXF

2016-07-01 Thread Lav Jain


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49521/
---

(Updated July 1, 2016, 8:21 p.m.)


Review request for Ambari, Alexander Denissov, bhuvnesh chaudhary, Jayush 
Luniya, Matt, and Tim Thorpe.


Changes
---

Merged with latest


Bugs: AMBARI-16647
https://issues.apache.org/jira/browse/AMBARI-16647


Repository: ambari


Description
---

Currently the service advisor tests for HAWQ and PXF are under 
ambari-server/src/test/python/stacks/2.3/common/test_stack_advisor.py
Move the tests to the service specific service_advisor files:
ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py
ambari-server/src/test/python/common-services/PXF/test_service_advisor.py


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/HAWQ/2.0.0/service_advisor.py 
e254094 
  ambari-server/src/main/resources/stacks/stack_advisor.py d8685c3 
  
ambari-server/src/test/python/common-services/HAWQ/test_alert_component_status.py
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/HAWQ/test_alert_segment_registration_status.py
 PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_alert_sync_status.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_hawqmaster.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_hawqsegment.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_hawqstandby.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_service_advisor.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/HAWQ/test_utils.py PRE-CREATION 
  ambari-server/src/test/python/common-services/PXF/test_alerts_api_status.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/PXF/test_pxf.py PRE-CREATION 
  ambari-server/src/test/python/common-services/PXF/test_service_advisor.py 
PRE-CREATION 
  ambari-server/src/test/python/common-services/configs/hawq_default.json 
PRE-CREATION 
  ambari-server/src/test/python/common-services/configs/hosts-1-host.json 
PRE-CREATION 
  ambari-server/src/test/python/common-services/configs/hosts-3-hosts.json 
PRE-CREATION 
  ambari-server/src/test/python/common-services/configs/pxf_default.json 
PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-hawq-1-host.json 
PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-hawq-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-hawq-pxf-hdfs.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-master_ambari_colo-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-master_standby_colo-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-nohawq-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-normal-hawq-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-normal-nohawq-3-hosts.json
 PRE-CREATION 
  
ambari-server/src/test/python/common-services/configs/services-standby_ambari_colo-3-hosts.json
 PRE-CREATION 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_component_status.py 
b2e1d4d 
  
ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_segment_registration_status.py
 6bb5930 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_alert_sync_status.py 
fd4f474 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqmaster.py 88fb008 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqsegment.py 8639ca5 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_hawqstandby.py b406723 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_service_advisor.py 8d97baa 
  ambari-server/src/test/python/stacks/2.3/HAWQ/test_utils.py 07737eb 
  ambari-server/src/test/python/stacks/2.3/PXF/test_alerts_api_status.py 
ee187e2 
  ambari-server/src/test/python/stacks/2.3/PXF/test_pxf.py 1147a7e 
  ambari-server/src/test/python/stacks/2.3/PXF/test_service_advisor.py 4ea3bfb 
  ambari-server/src/test/python/stacks/2.3/common/hosts-1-host.json 5d6a5b6 
  ambari-server/src/test/python/stacks/2.3/common/hosts-3-hosts.json 3c0511e 
  ambari-server/src/test/python/stacks/2.3/common/services-hawq-1-host.json 
515ba7d 
  ambari-server/src/test/python/stacks/2.3/common/services-hawq-3-hosts.json 
515ba7d 
  ambari-server/src/test/python/stacks/2.3/common/services-hawq-pxf-hdfs.json 
0bf459d 
  
ambari-server/src/test/python/stacks/2.3/common/services-master_ambari_colo-3-hosts.json
 1657ccf 
  
ambari-server/src/test/python/stacks/2.3/common/services-master_standby_colo-3-hosts.json
 cd5d02c 
  ambari-server/src/test/python/stacks/2.3/common/services-nohawq-3-hosts.json 
beeb62d 
  
ambari-server/src/test/python/stacks/2.3

Re: Review Request 49511: Falcon start fails rarely

2016-07-01 Thread Alejandro Fernandez


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49511/#review140429
---


Ship it!




Ship It!

- Alejandro Fernandez


On July 1, 2016, 1:52 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49511/
> ---
> 
> (Updated July 1, 2016, 1:52 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17524
> https://issues.apache.org/jira/browse/AMBARI-17524
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Traceback (most recent call last):
>   File 
> "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py",
>  line 177, in 
> FalconServer().execute()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py",
>  line 280, in execute
> method(env)
>   File 
> "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon_server.py",
>  line 50, in start
> falcon('server', action='start', upgrade_type=upgrade_type)
>   File 
> "/usr/lib/python2.6/site-packages/ambari_commons/os_family_impl.py", line 89, 
> in thunk
> return fn(*args, **kwargs)
>   File 
> "/var/lib/ambari-agent/cache/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py",
>  line 194, in falcon
> environment=environment_dictionary)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 
> 155, in __init__
> self.env.run()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 160, in run
> self.run_action(resource, action)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/environment.py", 
> line 124, in run_action
> provider_action()
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/providers/system.py",
>  line 273, in action_run
> tries=self.resource.tries, try_sleep=self.resource.try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 71, in inner
> result = function(command, **kwargs)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 93, in checked_call
> tries=tries, try_sleep=try_sleep)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 141, in _call_wrapper
> result = _call(command, **kwargs_copy)
>   File 
> "/usr/lib/python2.6/site-packages/resource_management/core/shell.py", line 
> 294, in _call
> raise Fail(err_msg)
> resource_management.core.exceptions.Fail: Execution of 
> '/usr/hdp/current/falcon-server/bin/falcon-start -port 15000' returned 1. 
> Hadoop home is set, adding libraries from 
> '/usr/hdp/current/hadoop-client/bin/hadoop classpath' into falcon classpath
> falcon running as process 3443. Stop it first.
> 
> 
> This happens because falcon start was not idempotent unlike other start
> command.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/package/scripts/falcon.py
>  c2f1f53 
>   ambari-server/src/test/python/stacks/2.1/FALCON/test_falcon_server.py 
> 6f81b15 
> 
> Diff: https://reviews.apache.org/r/49511/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 49535: Ambari Agent memory Leak fix.

2016-07-01 Thread Andrew Onischuk


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49535/
---

Review request for Ambari and Dmitro Lisnichenko.


Bugs: AMBARI-17539
https://issues.apache.org/jira/browse/AMBARI-17539


Repository: ambari


Description
---

Ambari Agent memory Leak fix.


Diffs
-

  ambari-agent/src/main/python/ambari_agent/main.py 4db89f8 

Diff: https://reviews.apache.org/r/49535/diff/


Testing
---

mvn clean test


Thanks,

Andrew Onischuk

Re: Review Request 49535: Ambari Agent memory Leak fix.

2016-07-01 Thread Victor Galgo


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49535/#review140439
---


Ship it!




Ship It!

- Victor Galgo


On July 1, 2016, 9:48 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49535/
> ---
> 
> (Updated July 1, 2016, 9:48 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-17539
> https://issues.apache.org/jira/browse/AMBARI-17539
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Ambari Agent memory Leak fix.
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/main.py 4db89f8 
> 
> Diff: https://reviews.apache.org/r/49535/diff/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 49537: Spark history server stopped after deploy

2016-07-01 Thread Weiqing Yang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49537/
---

Review request for Ambari and Alejandro Fernandez.


Bugs: AMBARI-17537
https://issues.apache.org/jira/browse/AMBARI-17537


Repository: ambari


Description
---

After deploy Spark history server is stopped.
16/06/27 19:55:45 INFO HistoryServer: History provider class: 
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider
Exception in thread "main" java.lang.ClassNotFoundException: 
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.apache.spark.deploy.history.HistoryServer$.main(HistoryServer.scala:224)
at org.apache.spark.deploy.history.HistoryServer.main(HistoryServer.scala).

Why this jira happened?
In the jira, the spark version is 1.5.1. which doesnot has the class 
org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider. Instead, it 
has "org.apache.spark.deploy.yarn.history.YarnHistoryProvider" and 
"org.apache.spark.deploy.history.yarn.YarnHistoryProvider".

How to fix? 
Actually, in spark internal logic, 
"org.apache.spark.deploy.yarn.history.YarnHistoryProvider" is a wrapper of 
"org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider" and 
"org.apache.spark.deploy.history.yarn.YarnHistoryProvider".
The property "spark.history.provider" appears twice in the stack definition, 
having two values: one is 
"org.apache.spark.deploy.yarn.history.YarnHistoryProvider", and the other one 
is "org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider". In this 
patch, remove the one whose value is 
"org.apache.spark.deploy.history.yarn.server.YarnHistoryProvider".
The jira is originally reported by Denis Tarasyuk.


Diffs
-

  
ambari-server/src/main/resources/common-services/SPARK/1.2.1/configuration/spark-defaults.xml
 84eb805 

Diff: https://reviews.apache.org/r/49537/diff/


Testing
---

Tests passed in the cluster: Ambari 2.4 + HDP 2.3.2


Thanks,

Weiqing Yang

Re: Review Request 49513: Ambari API for api/v1/clusters/?fields=hosts/* does not return the metrics information

2016-07-01 Thread Aravindan Vijayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49513/#review140454
---


Ship it!




Ship It!

- Aravindan Vijayan


On July 1, 2016, 3:26 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49513/
> ---
> 
> (Updated July 1, 2016, 3:26 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Myroslav Papirkovskyy, and Sid 
> Wagle.
> 
> 
> Bugs: AMBARI-17526
> https://issues.apache.org/jira/browse/AMBARI-17526
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> http://:8080/api/v1/clusters/?fields=hosts/* no 
> longer returns the metrics for the hosts.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProvider.java
>  9a7454c 
>   ambari-server/src/main/resources/ganglia_properties.json 05360da 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/metrics/timeline/AMSPropertyProviderTest.java
>  c7dabf1 
> 
> Diff: https://reviews.apache.org/r/49513/diff/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49514: Support Storm 1.0 in Ambari Metrics for Storm

2016-07-01 Thread Jungtaek Lim



> On July 1, 2016, 5:09 p.m., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml,
> >  line 40
> > 
> >
> > Does the parallelism = 1, mean 1 bolt getting all metric reports in a 
> > topology. Is it counter-intuitive for performance, something we should 
> > check with storm group.

Its default value has been 1, so setting it explicitly means letting users to 
modify it easily.


- Jungtaek


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49514/#review140384
---


On July 1, 2016, 3:32 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49514/
> ---
> 
> (Updated July 1, 2016, 3:32 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jungtaek Lim, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17080
> https://issues.apache.org/jira/browse/AMBARI-17080
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Support 2 storm sinks.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-assembly/pom.xml 5e2d819 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink-windows.xml 
> e82d2d4 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink.xml 4a3b7c5 
>   ambari-metrics/ambari-metrics-storm-sink-legacy/pom.xml PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/assemblies/empty.xml 
> PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  PRE-CREATION 
>   ambari-metrics/ambari-metrics-storm-sink/pom.xml 1591d39 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/conf/storm-metrics2.properties.j2
>  4553224 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/NumberUtil.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  ab5f1e4 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  6ab12e1 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  e582a95 
>   ambari-metrics/pom.xml 7221ab5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
>  0d029e8 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  2d50767 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
>  6551067 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
>  9acf173 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
>  f3bbce8 
> 
> Diff: https://reviews.apache.org/r/49514/diff/
> 
> 
> Testing
> ---
> 
> Unit testst passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49514: Support Storm 1.0 in Ambari Metrics for Storm

2016-07-01 Thread Jungtaek Lim



> On July 1, 2016, 3:54 p.m., Sid Wagle wrote:
> > ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java,
> >  line 194
> > 
> >
> > Good catch, however, why not look for "._" and replace that only if 
> > needed.
> 
> Dmytro Sen wrote:
> There is no change here. It's a copy of previous 
> StormTimelineMetricsSink.java

It's originally for escaping '._' but only escaping that case is not intuitive 
for users. Escaping all _ will make users to predict escaped metric name easily.


> On July 1, 2016, 3:54 p.m., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml,
> >  line 30
> > 
> >
> > So both old and new properties are ok to be present at the same time?
> 
> Dmytro Sen wrote:
> yes

yes since storm runs properly with unknown configs are presented.


- Jungtaek


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49514/#review140370
---


On July 1, 2016, 3:32 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49514/
> ---
> 
> (Updated July 1, 2016, 3:32 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jungtaek Lim, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17080
> https://issues.apache.org/jira/browse/AMBARI-17080
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Support 2 storm sinks.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-assembly/pom.xml 5e2d819 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink-windows.xml 
> e82d2d4 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink.xml 4a3b7c5 
>   ambari-metrics/ambari-metrics-storm-sink-legacy/pom.xml PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/assemblies/empty.xml 
> PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  PRE-CREATION 
>   ambari-metrics/ambari-metrics-storm-sink/pom.xml 1591d39 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/conf/storm-metrics2.properties.j2
>  4553224 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/NumberUtil.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  ab5f1e4 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  6ab12e1 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  e582a95 
>   ambari-metrics/pom.xml 7221ab5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
>  0d029e8 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  2d50767 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
>  6551067 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
>  9acf173 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
>  f3bbce8 
> 
> Diff: https://reviews.apache.org/r/49514/diff/
> 
> 
> Testing
> ---
> 
> Unit testst passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 49539: AMBARI-17541: LogSearch should also show audit logs from Apache Ranger Solr

2016-07-01 Thread Don Bosco Durai


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49539/
---

Review request for Ambari, Dharmesh Makwana, Miklos Gergely, Oliver Szabo, 
Robert Nettleton, and Sumit Mohanty.


Bugs: AMBARI-17541
https://issues.apache.org/jira/browse/AMBARI-17541


Repository: ambari


Description
---

Creating solr colection alias with audit_logs and ranger_audits


Diffs
-

  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/AuditSolrDao.java
 f1789c1 
  
ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/dao/SolrDaoBase.java
 3a1a1ca 

Diff: https://reviews.apache.org/r/49539/diff/


Testing
---

Tested in 3 node ambari cluster


Thanks,

Don Bosco Durai

Re: Review Request 49514: Support Storm 1.0 in Ambari Metrics for Storm

2016-07-01 Thread Aravindan Vijayan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49514/#review140459
---




ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
 (line 58)


Just trying to understand this. 

Does this mean that on a new HDP 2.5 cluster or on a cluster that has been 
upgraded to HDP 2.5, upgrading Ambari 2.4.0 to a higher version will delete 
this config?


- Aravindan Vijayan


On July 1, 2016, 3:32 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49514/
> ---
> 
> (Updated July 1, 2016, 3:32 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jungtaek Lim, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17080
> https://issues.apache.org/jira/browse/AMBARI-17080
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Support 2 storm sinks.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-assembly/pom.xml 5e2d819 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink-windows.xml 
> e82d2d4 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink.xml 4a3b7c5 
>   ambari-metrics/ambari-metrics-storm-sink-legacy/pom.xml PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/assemblies/empty.xml 
> PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  PRE-CREATION 
>   ambari-metrics/ambari-metrics-storm-sink/pom.xml 1591d39 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/conf/storm-metrics2.properties.j2
>  4553224 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/NumberUtil.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  ab5f1e4 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  6ab12e1 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  e582a95 
>   ambari-metrics/pom.xml 7221ab5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
>  0d029e8 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  2d50767 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
>  6551067 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
>  9acf173 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
>  f3bbce8 
> 
> Diff: https://reviews.apache.org/r/49514/diff/
> 
> 
> Testing
> ---
> 
> Unit testst passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Review Request 49470: Provide an option not to log ambari-agent command output for user custom script action


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49470/
---

Review request for Ambari, Alejandro Fernandez and Nahappan Somasundaram.


Bugs: AMBARI-17507
https://issues.apache.org/jira/browse/AMBARI-17507


Repository: ambari


Description
---

Provide an option not to log ambari-agent command output for user custom script 
action


Diffs
-

  ambari-agent/src/main/python/ambari_agent/ActionQueue.py 60c72af 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 4f2da54 
  
ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
 bdb5fb1 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
 8bb6225 
  
ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
 4aa5ea4 

Diff: https://reviews.apache.org/r/49470/diff/


Testing
---

Ran tests manually - results are also posted at 
https://issues.apache.org/jira/browse/AMBARI-17507


Thanks,

Sumit Mohanty

Re: Review Request 49514: Support Storm 1.0 in Ambari Metrics for Storm

2016-07-01 Thread Sriharsha Chintalapani



> On July 1, 2016, 5:09 p.m., Sid Wagle wrote:
> > ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml,
> >  line 40
> > 
> >
> > Does the parallelism = 1, mean 1 bolt getting all metric reports in a 
> > topology. Is it counter-intuitive for performance, something we should 
> > check with storm group.
> 
> Jungtaek Lim wrote:
> Its default value has been 1, so setting it explicitly means letting 
> users to modify it easily.

this is tricky config. We should expose this in ambari config page otherwise 
most users wouldn't notice that a metrics bolt running and can cause perf 
degradation if they go with 1 parallelism to prod.


- Sriharsha


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49514/#review140384
---


On July 1, 2016, 3:32 p.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49514/
> ---
> 
> (Updated July 1, 2016, 3:32 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Jungtaek Lim, and Sid Wagle.
> 
> 
> Bugs: AMBARI-17080
> https://issues.apache.org/jira/browse/AMBARI-17080
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Support 2 storm sinks.
> 
> 
> Diffs
> -
> 
>   ambari-metrics/ambari-metrics-assembly/pom.xml 5e2d819 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink-windows.xml 
> e82d2d4 
>   ambari-metrics/ambari-metrics-assembly/src/main/assembly/sink.xml 4a3b7c5 
>   ambari-metrics/ambari-metrics-storm-sink-legacy/pom.xml PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/assemblies/empty.xml 
> PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink-legacy/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  PRE-CREATION 
>   ambari-metrics/ambari-metrics-storm-sink/pom.xml 1591d39 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/conf/storm-metrics2.properties.j2
>  4553224 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/NumberUtil.java
>  PRE-CREATION 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsReporter.java
>  ab5f1e4 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/main/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSink.java
>  6ab12e1 
>   
> ambari-metrics/ambari-metrics-storm-sink/src/test/java/org/apache/hadoop/metrics2/sink/storm/StormTimelineMetricsSinkTest.java
>  e582a95 
>   ambari-metrics/pom.xml 7221ab5 
>   
> ambari-server/src/main/resources/common-services/AMBARI_METRICS/0.1.0/configuration/storm-site.xml
>  0d029e8 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  073bb1c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  2d50767 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/ui_server.py
>  6551067 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/templates/storm-metrics2.properties.j2
>  9acf173 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/STORM/configuration/storm-site.xml
>  f3bbce8 
> 
> Diff: https://reviews.apache.org/r/49514/diff/
> 
> 
> Testing
> ---
> 
> Unit testst passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 49470: Provide an option not to log ambari-agent command output for user custom script action

2016-07-01 Thread Nahappan Somasundaram


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49470/#review140470
---


Ship it!




Ship It!

- Nahappan Somasundaram


On July 1, 2016, 5:15 p.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/49470/
> ---
> 
> (Updated July 1, 2016, 5:15 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Nahappan Somasundaram, and 
> Swapan Shridhar.
> 
> 
> Bugs: AMBARI-17507
> https://issues.apache.org/jira/browse/AMBARI-17507
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Provide an option not to log ambari-agent command output for user custom 
> script action
> 
> 
> Diffs
> -
> 
>   ambari-agent/src/main/python/ambari_agent/ActionQueue.py 60c72af 
>   ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 4f2da54 
>   
> ambari-server/src/main/java/org/apache/ambari/server/agent/ExecutionCommand.java
>  bdb5fb1 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariCustomCommandExecutionHelper.java
>  8bb6225 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/AmbariManagementControllerTest.java
>  4aa5ea4 
> 
> Diff: https://reviews.apache.org/r/49470/diff/
> 
> 
> Testing
> ---
> 
> Ran tests manually - results are also posted at 
> https://issues.apache.org/jira/browse/AMBARI-17507
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Review Request 49538: AMBARI-17540. Add a validation check for HiveServer Interactive if total remaining capacity in cluster is less than 512 MB. Also following fixes : (1). Visibility to FALSE for 'L

2016-07-01 Thread Swapan Shridhar


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/49538/
---

Review request for Ambari, Alejandro Fernandez and Sumit Mohanty.


Bugs: AMBARI-17540
https://issues.apache.org/jira/browse/AMBARI-17540


Repository: ambari


Description
---

Added the following :

- Validation check for HiveServer Interactive, if total remaining capacity in 
cluster is less than 512 MB (a safeguard for Service Checks). 
- Fix for setting visibility to FALSE for 'LLAP queue capacity' slider if 
'llap' queue is in STOPPED state.
- security_status fn. for Hive Server Interactive updated.


Diffs
-

  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/package/scripts/hive_server_interactive.py
 8fb91d6 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/stack_advisor.py 
9426571 
  ambari-server/src/test/python/stacks/2.5/common/test_stack_advisor.py a5bfefc 

Diff: https://reviews.apache.org/r/49538/diff/


Testing
---

- Added Python UT.
- Python UT passes.


Thanks,

Swapan Shridhar

Re: Review Request 49538: AMBARI-17540. Add a validation check for HiveServer Interactive if total remaining capacity in cluster is less than 512 MB. Also following fixes : (1). Visibility to FALSE fo