Re: Review Request 57350: Kerberos identity reference not working for ranger-audit property in hbase

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57350/#review168100
---


Ship it!




Ship It!

- Sebastian Toader


On March 7, 2017, 1:13 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57350/
> ---
> 
> (Updated March 7, 2017, 1:13 a.m.)
> 
> 
> Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene 
> Chekanskiy, Laszlo Puskas, Mugdha Varadkar, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-20335
> https://issues.apache.org/jira/browse/AMBARI-20335
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> From stack 2.5 onwards 
> `xasecure.audit.jaas.Client.option.principal/ranger-hbase-audit` needs to 
> have principal value available under 
> `hbase.master.kerberos.principal/hbase-site`
> 
> To achieve that added below block of code under hbase 
> [kerberos.json|https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json]
> ```
> {
>   "name": "/HBASE/HBASE_MASTER/hbase_master_hbase",
>   "principal": {
> "configuration": 
> "ranger-hbase-audit/xasecure.audit.jaas.Client.option.principal"
>   },
>   "keytab": {
> "configuration": 
> "ranger-hbase-audit/xasecure.audit.jaas.Client.option.keyTab"
>   }
> }
> ```
> 
> But on test cluster, 
> `xasecure.audit.jaas.Client.option.principal/ranger-hbase-audit` property is 
> not showing the expected value. It is showing the principal/keytab values of 
> `ams_hbase_master_hbase` identity. 
> 
> Because of wrong reference of principal audit to solr is not working in 
> kerberos environment, as security.json have below entry instead of 
> `hb...@example.com`
> ```
> "amshb...@example.com":[
> "ranger_audit_user",
> "dev"]
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  141e9cd 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
> f510770 
>   
> ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHBASE/kerberos.json
>  b053779 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  2a59ccc 
>   
> ambari-server/src/test/resources/kerberos/test_kerberos_descriptor_2_5_infra_solr.json
>  0c2723e 
> 
> 
> Diff: https://reviews.apache.org/r/57350/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested in Ambari 2.5.0 cluster and upgrade from Ambari 2.4.2.
> 
> # Local test results:
> 
> ```
> [INFO] 
> 
> [INFO] BUILD SUCCESS
> [INFO] 
> 
> [INFO] Total time: 23:53.766s
> [INFO] Finished at: Mon Mar 06 16:55:35 EST 2017
> [INFO] Final Memory: 71M/772M
> [INFO] 
> 
> ```
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 57326: storm DRPC_SERVER kerberos configs duplicate

[wang yaoxin]

> On 三月 6, 2017, 4:47 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml
> > Lines 99-109 (patched)
> > 
> >
> > Are these _new_ properties used anywhere?
> 
> wang yaoxin wrote:
> These new properties are not used .

These new properties are not being used elsewhere


- wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57326/#review167997
---


On 三月 7, 2017, 2:59 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57326/
> ---
> 
> (Updated 三月 7, 2017, 2:59 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Robert Levas, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18892
> https://issues.apache.org/jira/browse/AMBARI-18892
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> when ambari enables kerberos,  storm service ,nimbus_keytab and 
> nimbus_principal_name will duplicate
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/STORM/0.9.1/kerberos.json 
> 20cc32d 
>   ambari-server/src/main/resources/common-services/STORM/1.0.1/kerberos.json 
> fa2f6db 
>   ambari-web/app/assets/data/stacks/HDP-2.1/service_components.json 0a8f20b 
> 
> 
> Diff: https://reviews.apache.org/r/57326/diff/2/
> 
> 
> Testing
> ---
> 
> done?
> 
> 
> File Attachments
> 
> 
> storm.png
>   
> https://reviews.apache.org/media/uploaded/files/2017/03/06/35c14040-f918-4f46-83e8-be51a01d5d53__storm.png
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 57326: storm DRPC_SERVER kerberos configs duplicate

[wang yaoxin]

> On 三月 6, 2017, 4:47 p.m., Robert Levas wrote:
> > ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml
> > Lines 99-109 (patched)
> > 
> >
> > Are these _new_ properties used anywhere?

These new properties are not used .


- wang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57326/#review167997
---


On 三月 7, 2017, 2:59 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57326/
> ---
> 
> (Updated 三月 7, 2017, 2:59 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Robert Levas, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18892
> https://issues.apache.org/jira/browse/AMBARI-18892
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> when ambari enables kerberos,  storm service ,nimbus_keytab and 
> nimbus_principal_name will duplicate
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/common-services/STORM/0.9.1/kerberos.json 
> 20cc32d 
>   ambari-server/src/main/resources/common-services/STORM/1.0.1/kerberos.json 
> fa2f6db 
>   ambari-web/app/assets/data/stacks/HDP-2.1/service_components.json 0a8f20b 
> 
> 
> Diff: https://reviews.apache.org/r/57326/diff/2/
> 
> 
> Testing
> ---
> 
> done?
> 
> 
> File Attachments
> 
> 
> storm.png
>   
> https://reviews.apache.org/media/uploaded/files/2017/03/06/35c14040-f918-4f46-83e8-be51a01d5d53__storm.png
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 57326: storm DRPC_SERVER kerberos configs duplicate

[wang yaoxin]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57326/
---

(Updated 三月 7, 2017, 2:59 a.m.)


Review request for Ambari, Alejandro Fernandez, Robert Levas, and Sid Wagle.


Bugs: AMBARI-18892
https://issues.apache.org/jira/browse/AMBARI-18892


Repository: ambari


Description
---

when ambari enables kerberos,  storm service ,nimbus_keytab and 
nimbus_principal_name will duplicate


Diffs (updated)
-

  ambari-server/src/main/resources/common-services/STORM/0.9.1/kerberos.json 
20cc32d 
  ambari-server/src/main/resources/common-services/STORM/1.0.1/kerberos.json 
fa2f6db 
  ambari-web/app/assets/data/stacks/HDP-2.1/service_components.json 0a8f20b 


Diff: https://reviews.apache.org/r/57326/diff/2/

Changes: https://reviews.apache.org/r/57326/diff/1-2/


Testing
---

done?


File Attachments


storm.png
  
https://reviews.apache.org/media/uploaded/files/2017/03/06/35c14040-f918-4f46-83e8-be51a01d5d53__storm.png


Thanks,

wang yaoxin

Re: Review Request 57346: Value for "User Limit Factor" should be float instead of integer in YARN Queue Manager.

[Anita Jebaraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57346/
---

(Updated March 7, 2017, 1:02 a.m.)


Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and Sangeeta 
Ravindran.


Bugs: AMBARI-20333
https://issues.apache.org/jira/browse/AMBARI-20333


Repository: ambari


Description
---

According to yarn documentation, 
"yarn.scheduler.capacity..user-limit-factor" should be a float:
https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Hence the user should be allowed to enter decimal values for 'User limit factor'


Diffs (updated)
-

  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/capacityInput.js
 8eb06c2 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queue.hbs 
69f5b3b 


Diff: https://reviews.apache.org/r/57346/diff/2/

Changes: https://reviews.apache.org/r/57346/diff/1-2/


Testing
---

Tested manually


File Attachments


screenshot-1.png
  
https://reviews.apache.org/media/uploaded/files/2017/03/06/f55e8f25-b625-4285-96ee-beb4fa6992a4__screenshot-1.png


Thanks,

Anita Jebaraj

Review Request 57350: Kerberos identity reference not working for ranger-audit property in hbase

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57350/
---

Review request for Ambari, Attila Magyar, Balázs Bence Sári, Eugene Chekanskiy, 
Laszlo Puskas, Mugdha Varadkar, and Sebastian Toader.


Bugs: AMBARI-20335
https://issues.apache.org/jira/browse/AMBARI-20335


Repository: ambari


Description
---

>From stack 2.5 onwards 
>`xasecure.audit.jaas.Client.option.principal/ranger-hbase-audit` needs to have 
>principal value available under `hbase.master.kerberos.principal/hbase-site`

To achieve that added below block of code under hbase 
[kerberos.json|https://github.com/apache/ambari/blob/branch-2.5/ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json]
```
{
  "name": "/HBASE/HBASE_MASTER/hbase_master_hbase",
  "principal": {
"configuration": 
"ranger-hbase-audit/xasecure.audit.jaas.Client.option.principal"
  },
  "keytab": {
"configuration": 
"ranger-hbase-audit/xasecure.audit.jaas.Client.option.keyTab"
  }
}
```

But on test cluster, 
`xasecure.audit.jaas.Client.option.principal/ranger-hbase-audit` property is 
not showing the expected value. It is showing the principal/keytab values of 
`ams_hbase_master_hbase` identity. 

Because of wrong reference of principal audit to solr is not working in 
kerberos environment, as security.json have below entry instead of 
`hb...@example.com`
```
"amshb...@example.com":[
"ranger_audit_user",
"dev"]
```


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 141e9cd 
  ambari-server/src/main/resources/stacks/HDP/2.5/services/HBASE/kerberos.json 
f510770 
  
ambari-server/src/main/resources/stacks/PERF/1.0/services/FAKEHBASE/kerberos.json
 b053779 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 2a59ccc 
  
ambari-server/src/test/resources/kerberos/test_kerberos_descriptor_2_5_infra_solr.json
 0c2723e 


Diff: https://reviews.apache.org/r/57350/diff/1/


Testing
---

Manually tested in Ambari 2.5.0 cluster and upgrade from Ambari 2.4.2.

# Local test results:

```
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 23:53.766s
[INFO] Finished at: Mon Mar 06 16:55:35 EST 2017
[INFO] Final Memory: 71M/772M
[INFO] 
```

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 57168: Include option to filter out properties from APi that returns ambari.properties file

[Anita Jebaraj]

> On March 6, 2017, 9:50 p.m., Di Li wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/controller/RootServiceResponseFactory.java
> > Lines 165 (patched)
> > 
> >
> > The for loop should not run anyway when propertiesToHideInResponse is 
> > empty.

Please review the new patch. Thank you


- Anita


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57168/#review168045
---


On March 6, 2017, 10:45 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57168/
> ---
> 
> (Updated March 6, 2017, 10:45 p.m.)
> 
> 
> Review request for Ambari, Di Li, Oleksandr Diachenko, Sangeeta Ravindran, 
> and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-20243
> https://issues.apache.org/jira/browse/AMBARI-20243
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently all the details from the ambari.properties file is being returned 
> by the API call.
> 
> Some of those information may not be utilized and hence an option can be 
> provided to filter the properties
> 
> A ambari-blacklist.properties file can be created, the properties that are 
> entered in the file, will be removed from the api call that returns the 
> ambari.properties.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  0991814 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RootServiceResponseFactory.java
>  dadcf09 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
>  51114f8 
> 
> 
> Diff: https://reviews.apache.org/r/57168/diff/5/
> 
> 
> Testing
> ---
> 
> Added 1 test case
> Ran mvn test
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 57168: Include option to filter out properties from APi that returns ambari.properties file

[Anita Jebaraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57168/
---

(Updated March 6, 2017, 10:45 p.m.)


Review request for Ambari, Di Li, Oleksandr Diachenko, Sangeeta Ravindran, and 
Vitalyi Brodetskyi.


Bugs: AMBARI-20243
https://issues.apache.org/jira/browse/AMBARI-20243


Repository: ambari


Description
---

Currently all the details from the ambari.properties file is being returned by 
the API call.

Some of those information may not be utilized and hence an option can be 
provided to filter the properties

A ambari-blacklist.properties file can be created, the properties that are 
entered in the file, will be removed from the api call that returns the 
ambari.properties.


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
 0991814 
  
ambari-server/src/main/java/org/apache/ambari/server/controller/RootServiceResponseFactory.java
 dadcf09 
  
ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
 51114f8 


Diff: https://reviews.apache.org/r/57168/diff/5/

Changes: https://reviews.apache.org/r/57168/diff/4-5/


Testing
---

Added 1 test case
Ran mvn test


Thanks,

Anita Jebaraj

Review Request 57346: Value for "User Limit Factor" should be float instead of integer in YARN Queue Manager.

[Anita Jebaraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57346/
---

Review request for Ambari, DIPAYAN BHOWMICK, Pallav Kulshreshtha, and Sangeeta 
Ravindran.


Bugs: AMBARI-20333
https://issues.apache.org/jira/browse/AMBARI-20333


Repository: ambari


Description
---

According to yarn documentation, 
"yarn.scheduler.capacity..user-limit-factor" should be a float:
https://hadoop.apache.org/docs/r2.4.1/hadoop-yarn/hadoop-yarn-site/CapacityScheduler.html

Hence the user should be allowed to enter decimal values for 'User limit factor'


Diffs
-

  
contrib/views/capacity-scheduler/src/main/resources/ui/app/components/capacityInput.js
 8eb06c2 
  
contrib/views/capacity-scheduler/src/main/resources/ui/app/templates/queue.hbs 
69f5b3b 


Diff: https://reviews.apache.org/r/57346/diff/1/


Testing
---

Tested manually


File Attachments


screenshot-1.png
  
https://reviews.apache.org/media/uploaded/files/2017/03/06/f55e8f25-b625-4285-96ee-beb4fa6992a4__screenshot-1.png


Thanks,

Anita Jebaraj

Re: Review Request 57168: Include option to filter out properties from APi that returns ambari.properties file

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57168/#review168045
---




ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
Lines 4328 (patched)


This should be an error. Use String.format instead of "+".



ambari-server/src/main/java/org/apache/ambari/server/controller/RootServiceResponseFactory.java
Lines 165 (patched)


The for loop should not run anyway when propertiesToHideInResponse is empty.


- Di Li


On March 3, 2017, 8:47 p.m., Anita Jebaraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57168/
> ---
> 
> (Updated March 3, 2017, 8:47 p.m.)
> 
> 
> Review request for Ambari, Di Li, Oleksandr Diachenko, Sangeeta Ravindran, 
> and Vitalyi Brodetskyi.
> 
> 
> Bugs: AMBARI-20243
> https://issues.apache.org/jira/browse/AMBARI-20243
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Currently all the details from the ambari.properties file is being returned 
> by the API call.
> 
> Some of those information may not be utilized and hence an option can be 
> provided to filter the properties
> 
> A ambari-blacklist.properties file can be created, the properties that are 
> entered in the file, will be removed from the api call that returns the 
> ambari.properties.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/configuration/Configuration.java
>  eaecf35 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/RootServiceResponseFactory.java
>  dadcf09 
>   
> ambari-server/src/test/java/org/apache/ambari/server/configuration/ConfigurationTest.java
>  51114f8 
> 
> 
> Diff: https://reviews.apache.org/r/57168/diff/4/
> 
> 
> Testing
> ---
> 
> Added 1 test case
> Ran mvn test
> 
> 
> Thanks,
> 
> Anita Jebaraj
> 
>

Re: Review Request 57297: Cluster deployment using blueprint with empty configuration doesn't work with stack advisor enabled

[Di Li]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57297/#review168042
---


Ship it!




Ship It!

- Di Li


On March 3, 2017, 6:59 p.m., Amruta Borkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57297/
> ---
> 
> (Updated March 3, 2017, 6:59 p.m.)
> 
> 
> Review request for Ambari, Di Li, Robert Nettleton, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-20288
> https://issues.apache.org/jira/browse/AMBARI-20288
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When a blueprint deployment is made with empty configuration for a particular 
> config-type and when stack advisor is returning a ValueAttributeInfo for for 
> that config-type with a property marked "true" for "Delete" flag
> 
> Cluster deployment gets stuck at TOPOLOGY_RESOLUTION phase showing message
> Config type not resolved yet, Blueprint deployment will wait until 
> configuration update is completed.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessor.java
>  d4880b9 
>   
> ambari-server/src/test/java/org/apache/ambari/server/controller/internal/BlueprintConfigurationProcessorTest.java
>  928d1e0 
> 
> 
> Diff: https://reviews.apache.org/r/57297/diff/1/
> 
> 
> Testing
> ---
> 
> Manual testing done.
> Modified test case to address the scenario.
> 
> 
> Thanks,
> 
> Amruta Borkar
> 
>

Re: Review Request 57263: Fix values of ATS config params for HDP stack

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57263/#review168037
---




ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml
Lines 38 (patched)


These configs are also needed for YARN 3.0.0.3.0 in common-services.


- Alejandro Fernandez


On March 3, 2017, 1:30 a.m., Sumit Mohanty wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57263/
> ---
> 
> (Updated March 3, 2017, 1:30 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez and Swapan Shridhar.
> 
> 
> Bugs: AMBARI-20284
> https://issues.apache.org/jira/browse/AMBARI-20284
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Fix values of ATS config params for HDP stack
> 
> 
> Diffs
> -
> 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml 
> 000e902 
>   
> ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml
>  7c9fb67 
>   ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml 
> 9f3f42d 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml 
> 8b00b05 
>   
> ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml
>  c2e185d 
>   ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml 
> edc227a 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
> da036cf 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
>  f232118 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 
> f0a4f05 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/YARN/configuration/yarn-site.xml
>  70a2cbe 
> 
> 
> Diff: https://reviews.apache.org/r/57263/diff/1/
> 
> 
> Testing
> ---
> 
> Ran all unit tests and manually tested through EU.
> 
> 
> Thanks,
> 
> Sumit Mohanty
> 
>

Re: Review Request 57287: LogSearch Portal UI Fails on Last Page selection if logs were deleted in the meantime

[Robert Nettleton]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57287/#review168033
---


Ship it!




Ship It!

- Robert Nettleton


On March 3, 2017, 9:23 p.m., Miklos Gergely wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57287/
> ---
> 
> (Updated March 3, 2017, 9:23 p.m.)
> 
> 
> Review request for Ambari, Oliver Szabo and Robert Nettleton.
> 
> 
> Bugs: AMBARI-20300
> https://issues.apache.org/jira/browse/AMBARI-20300
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> If logs are deleted between the last screen generation and the clicking on 
> the last page button an empty table comes up. The same happens if the user 
> clicks on a page number which is no longer valid, as there are fewer logs 
> since.
> 
> 
> Diffs
> -
> 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/converter/AbstractSearchRequestQueryConverter.java
>  d44b866 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/AuditLogsManager.java
>  2dc0ef7 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ManagerBase.java
>  89873f3 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/java/org/apache/ambari/logsearch/manager/ServiceLogsManager.java
>  f960250 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/collections/BaseCollection.js
>  c175397 
>   
> ambari-logsearch/ambari-logsearch-portal/src/main/webapp/scripts/views/common/TableLayout.js
>  1066510 
> 
> 
> Diff: https://reviews.apache.org/r/57287/diff/2/
> 
> 
> Testing
> ---
> 
> Tested on local cluster setting extra short time for the logs to expire.
> 
> 
> Thanks,
> 
> Miklos Gergely
> 
>

Re: Review Request 57268: AMBARI-20287 - Filter in Customize Services Page doesn't bring up all properties that matches

[Richard Zang]

> On March 4, 2017, 2:33 a.m., Jaimin Jetly wrote:
> > ambari-web/app/models/configs/theme/sub_section_tab.js
> > Line 84 (original), 84-87 (patched)
> > 
> >
> > Whats the reason behind making this computed property volatile ?

Adding volatile here is because I keep getting inconsistent result on this 
computed property during manual test, so add volatile to make sure the value is 
consistent even it's updated more than once in one digest loop.


- Richard


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57268/#review167925
---


On March 3, 2017, 7:34 a.m., Richard Zang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57268/
> ---
> 
> (Updated March 3, 2017, 7:34 a.m.)
> 
> 
> Review request for Ambari, Jaimin Jetly and Yusaku Sako.
> 
> 
> Bugs: AMBARI-20287
> https://issues.apache.org/jira/browse/AMBARI-20287
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Take subSectionTab into consideration on show/hide
> 
> 
> Diffs
> -
> 
>   ambari-web/app/models/configs/theme/sub_section.js 7274569 
>   ambari-web/app/models/configs/theme/sub_section_tab.js 2262882 
>   ambari-web/app/views/common/configs/service_config_layout_tab_view.js 
> 466a88d 
>   ambari-web/test/models/configs/theme/sub_section_tab_test.js 0c3b98c 
> 
> 
> Diff: https://reviews.apache.org/r/57268/diff/1/
> 
> 
> Testing
> ---
> 
> Manully tested on live cluster. All unit tests passed.
>   20560 passing (25s)
>   153 pending
> 
> 
> Thanks,
> 
> Richard Zang
> 
>

Review Request 57344: Commands timed-out on ambari host without any error logs

[Eugene Chekanskiy]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57344/
---

Review request for Ambari, Attila Doroszlai, Andrew Onischuk, Dmytro Sen, 
Robert Levas, and Sebastian Toader.


Bugs: AMBARI-20323
https://issues.apache.org/jira/browse/AMBARI-20323


Repository: ambari


Description
---

All status command execution logic moved under one file.

* added protection for hardkilling(now process tries to die gracefully)
* added protection for queue internal logic blocking(underlying pipes hard 
closed)
* added lock around queues that will prevent from using queues before new one 
will be created in case of recreating executor process


Diffs
-

  ambari-agent/src/main/python/ambari_agent/ActionQueue.py 5300b52 
  ambari-agent/src/main/python/ambari_agent/Controller.py 61a74e6 
  ambari-agent/src/main/python/ambari_agent/StatusCommandsExecutor.py 2f15770 
  ambari-agent/src/main/python/ambari_agent/main.py 3f333c4 
  ambari-agent/src/test/python/ambari_agent/TestActionQueue.py 9fefefb 
  ambari-agent/src/test/python/ambari_agent/TestController.py 663e215 
  ambari-agent/src/test/python/ambari_agent/TestMain.py 97c448b 


Diff: https://reviews.apache.org/r/57344/diff/1/


Testing
---

mvn clean test, manual testing


Thanks,

Eugene Chekanskiy

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/
---

(Updated March 6, 2017, 12:26 p.m.)


Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, and 
Sid Wagle.


Bugs: AMBARI-20309
https://issues.apache.org/jira/browse/AMBARI-20309


Repository: ambari


Description
---

HBase Master CPU Utilization Alert is in unknown state due to kinit error:

```
Execution of '/usr/bin/kinit -c 
/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
 -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
/dev/null' returned 1. kinit: Client not found in Kerberos database while 
getting initial credentials
```

This issue is also seen in /var/log/krb5kdc.log:

```
Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
{18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
/etc/security/keytabs/spnego.service.key...@example.com for 
krbtgt/example@example.com, Client not found in Kerberos database
```

#Cause
It appears that the HBASE alerts.json file 
(`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
`kerberos_keytab` and `kerberos_principal` properties.

```
  {
"name": "hbase_master_cpu",
"label": "HBase Master CPU Utilization",
"description": "This host-level alert is triggered if CPU utilization 
of the HBase Master exceeds certain warning and critical thresholds. It checks 
the HBase Master JMX Servlet for the SystemCPULoad property. The threshold 
values are in percent.",
"interval": 5,
"scope": "ANY",
"enabled": true,
"source": {
  "type": "METRIC",
  "uri": {
"http": "{{hbase-site/hbase.master.info.port}}",
"default_port": 60010,
"connection_timeout": 5.0,
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
  },
  "reporting": {
"ok": {
  "text": "{1} CPU, load {0:.1%}"
},
"warning": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 200
},
"critical": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 250
},
"units" : "%",
"type": "PERCENT"
  },
  "jmx": {
"property_list": [
  "java.lang:type=OperatingSystem/SystemCpuLoad",
  "java.lang:type=OperatingSystem/AvailableProcessors"
],
"value": "{0} * 100"
  }
}
  }
```

Notice:
```
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```

#Solution
Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
`common-services/HBASE/0.96.0.2.0/alerts.json`:

```
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 2a684dc 
  ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
1b3ae25 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 7ee66ef 


Diff: https://reviews.apache.org/r/57324/diff/2/


Testing (updated)
---

Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
2.4.2 to Ambari 2.5.0

# Local test results:

```
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 26:07.123s
[INFO] Finished at: Mon Mar 06 12:25:32 EST 2017
[INFO] Final Memory: 70M/596M
[INFO] 
```


# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review168010
---


Ship it!




Ship It!

- Jonathan Hurley


On March 6, 2017, 11:25 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 6, 2017, 11:25 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  2a684dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
> 1b3ae25 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7ee66ef 
> 
> 
> Diff: https://reviews.apache.org/r/57324/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
> 2.4.2 to Ambari 2.5.0
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 57281: Script-Based Alert Dispathers support passing more parameters to script

[yao lei]

> On 三月 6, 2017, 2:21 p.m., Jonathan Hurley wrote:
> > Ship It!
> 
> yao lei wrote:
> Hi Jonathan,
> Thank you very much.
> Would you please help me to commit the patch to trunk if you are free?
> 
> Jonathan Hurley wrote:
> Yes, I'm just running a quick test on this patch locally and I'll commit 
> it shortly once it passes...
> 
> Jonathan Hurley wrote:
> Committed. Please close the review.

Thank you again.I will close it later


- yao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57281/#review167979
---


On 三月 3, 2017, 11:56 a.m., yao lei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57281/
> ---
> 
> (Updated 三月 3, 2017, 11:56 a.m.)
> 
> 
> Review request for Ambari and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-20291
> https://issues.apache.org/jira/browse/AMBARI-20291
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Script-Based Alert Dispatcher now pass five parameters to script,including 
> alert definition name, definition label,service name, alert state, and alert 
> text.
> But if script can receive other two parameters from dispather,it will be 
> better.
> 1.hostname.
> Because hostname the alert for is not always included in alert text,although 
> it may be null like aggregate alerts.
> With it we can more quick to find the related host that occured alert.
> 2.alert timestamp.
> We may need to know the alert occurrence time ( state change time) more 
> exactly. After the alert happened,it will spend some time to schedule the 
> script to run.
> Without it,we can only regard the script start time as the alert occurrence 
> time.
> 
> We now use this feature to send alert information to mobile phone and suggest 
> also passing hostname and alert timestamp.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcher.java
>  907588d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/services/AlertNoticeDispatchService.java
>  174f31f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcherTest.java
>  9e0e406 
> 
> 
> Diff: https://reviews.apache.org/r/57281/diff/1/
> 
> 
> Testing
> ---
> 
> 1.cd ambari-server
>mvn test
> 
> 2.write a python script to receive parameters from dispatcher
> 
> 
> #!/usr/bin/python
> import sys
> import logging
> 
> def main():
> definitionName = sys.argv[1]
> definitionLabel = sys.argv[2]
> serviceName = sys.argv[3]
> alertState = sys.argv[4]
> alertText = sys.argv[5]
> alertTimestamp = sys.argv[6]
> hostname = sys.argv[7] if len(sys.argv)-1 == 7 else None
> 
> logFile='/var/log/ambari-server/log_py.log'
> 
> logging.basicConfig(filename = logFile, level = logging.DEBUG)
> logging.debug('received ' + str(len(sys.argv)-1) + ' parameters')
> for i in range(1, len(sys.argv)):
> logging.debug('parameter ' +str(i) + ':' +str(sys.argv[i]))
> 
> if __name__ == '__main__':
>main()
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/log/ambari-server/log_py.log,
> 
> 3.write a shell script to  receive parameters from dispatcher
> 
> 
> #!/bin/bash
> logFile=/var/log/ambari-server/log_sh.log
> definitionName=$1
> definitionLabel=$2
> serviceName=$3
> alertState=$4 
> alertText=$5
> alertTimestamp=$6
> hostname=$7
> 
> echo received $# parameters:  $definitionName, $definitionLabel, 
> $serviceName, $alertState ,$alertText ,$alertTimestamp, $hostname  >> $logFile
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/ambari-server/log_sh.log, we can see the expected result.
> 
> 
> Thanks,
> 
> yao lei
> 
>

Re: Review Request 57089: HDP 3.0 support for Slider with configs, kerberos, widgets, metrics, quicklinks, and themes

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57089/#review168009
---


Fix it, then Ship it!





ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-client.xml
Lines 23 (patched)


Are none of these configs active?



ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-env.xml
Lines 44 (patched)


Let's set https://reviews.apache.org/r/57089/#comment240015>

Can assume that slider_home_dir is defined since this version of slider 
does support RU.


- Alejandro Fernandez


On March 6, 2017, 11:32 a.m., Dmytro Sen wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57089/
> ---
> 
> (Updated March 6, 2017, 11:32 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Sid Wagle, and Vitalyi 
> Brodetskyi.
> 
> 
> Bugs: AMBARI-19835
> https://issues.apache.org/jira/browse/AMBARI-19835
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HDP 3.0 support for Slider with configs, kerberos, widgets, metrics, 
> quicklinks, and themes
> Flatten from HDP 2.0.6 - 2.6 into common-services, and reference in HDP 3.0
> In HDP 3.0, we have created a new stack definition that does not inherit from 
> other stacks, in order to reduce the complexity of having to analyze older 
> stacks.
> This means that we need to create a service definition (metainfo.xml, 
> configs, kerberos, widgets, metrics, quicklinks, and themes) that is 
> equivalent to what is inherit and deleted from all of the previous stacks.
> A merge needs to account for additions, overrides, and deletions.
> metainfo.xml and configs perform a merge of older versions
> kerberos.json always seems to override the previous file
> Because the bits for this service may not yet be available in the HDP 3.0 
> repo, 
> the task is to ensure that /api/v1/stacks/HDP/versions/2.6/services/SLIDER 
> (which uses inheritance) is equivalent to the flattening of 
> /api/v1/stacks/HDP/versions/3.0/services/SLIDER .
> Please take a look at how this was done for ZK, HDFS, and YARN/MR.
> This means that you will not be able to actually install the service for now, 
> but can still perform validation during the Install Wizard that the correct 
> components and configs show up.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-client.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-env.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-log4j.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/kerberos.json
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/metainfo.xml
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/files/hbaseSmokeVerify.sh
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/__init__.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_linux.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_windows.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/service_check.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/templates/storm-slider-env.sh.j2
>  PRE-CREATION 
>   
> ambari-server/src/main/resources/stacks/HDP/3.0/services/SLIDER/metainfo.xml 
> PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/57089/diff/2/
> 
> 
> Testing
> ---
> 
> Unit tests passed
> 
> 
> Thanks,
> 
> Dmytro Sen
> 
>

Re: Review Request 57339: Server startup script keeps waiting even if DB consistency has failed

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57339/#review168003
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 6, 2017, 2:54 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57339/
> ---
> 
> (Updated March 6, 2017, 2:54 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Attila Magyar, Andrew Onischuk, 
> Dmytro Sen, Laszlo Puskas, Nate Cole, Oliver Szabo, Robert Levas, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-20319
> https://issues.apache.org/jira/browse/AMBARI-20319
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When DB check failed during ambari start up, the script waited some 50 
> seconds for the UI before timeout occured, even though the Java process had 
> exited. This is fixed. Also added an extra pintStackTrace() in cases when 
> unexpected error occured during startup. This is needed as testing showed 
> that logs are not always flushed on system exit and importan diagnostic 
> information was lost.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  9540ca3 
>   ambari-server/src/main/python/ambari_server/utils.py 6408285 
>   ambari-server/src/main/python/ambari_server_main.py 0cd19cc 
> 
> 
> Diff: https://reviews.apache.org/r/57339/diff/1/
> 
> 
> Testing
> ---
> 
> - Tested manually.
> - Run all python unit tests in ambari-server, all passed
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 57281: Script-Based Alert Dispathers support passing more parameters to script

[Jonathan Hurley]

> On March 6, 2017, 9:21 a.m., Jonathan Hurley wrote:
> > Ship It!
> 
> yao lei wrote:
> Hi Jonathan,
> Thank you very much.
> Would you please help me to commit the patch to trunk if you are free?
> 
> Jonathan Hurley wrote:
> Yes, I'm just running a quick test on this patch locally and I'll commit 
> it shortly once it passes...

Committed. Please close the review.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57281/#review167979
---


On March 3, 2017, 6:56 a.m., yao lei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57281/
> ---
> 
> (Updated March 3, 2017, 6:56 a.m.)
> 
> 
> Review request for Ambari and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-20291
> https://issues.apache.org/jira/browse/AMBARI-20291
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Script-Based Alert Dispatcher now pass five parameters to script,including 
> alert definition name, definition label,service name, alert state, and alert 
> text.
> But if script can receive other two parameters from dispather,it will be 
> better.
> 1.hostname.
> Because hostname the alert for is not always included in alert text,although 
> it may be null like aggregate alerts.
> With it we can more quick to find the related host that occured alert.
> 2.alert timestamp.
> We may need to know the alert occurrence time ( state change time) more 
> exactly. After the alert happened,it will spend some time to schedule the 
> script to run.
> Without it,we can only regard the script start time as the alert occurrence 
> time.
> 
> We now use this feature to send alert information to mobile phone and suggest 
> also passing hostname and alert timestamp.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcher.java
>  907588d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/services/AlertNoticeDispatchService.java
>  174f31f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcherTest.java
>  9e0e406 
> 
> 
> Diff: https://reviews.apache.org/r/57281/diff/1/
> 
> 
> Testing
> ---
> 
> 1.cd ambari-server
>mvn test
> 
> 2.write a python script to receive parameters from dispatcher
> 
> 
> #!/usr/bin/python
> import sys
> import logging
> 
> def main():
> definitionName = sys.argv[1]
> definitionLabel = sys.argv[2]
> serviceName = sys.argv[3]
> alertState = sys.argv[4]
> alertText = sys.argv[5]
> alertTimestamp = sys.argv[6]
> hostname = sys.argv[7] if len(sys.argv)-1 == 7 else None
> 
> logFile='/var/log/ambari-server/log_py.log'
> 
> logging.basicConfig(filename = logFile, level = logging.DEBUG)
> logging.debug('received ' + str(len(sys.argv)-1) + ' parameters')
> for i in range(1, len(sys.argv)):
> logging.debug('parameter ' +str(i) + ':' +str(sys.argv[i]))
> 
> if __name__ == '__main__':
>main()
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/log/ambari-server/log_py.log,
> 
> 3.write a shell script to  receive parameters from dispatcher
> 
> 
> #!/bin/bash
> logFile=/var/log/ambari-server/log_sh.log
> definitionName=$1
> definitionLabel=$2
> serviceName=$3
> alertState=$4 
> alertText=$5
> alertTimestamp=$6
> hostname=$7
> 
> echo received $# parameters:  $definitionName, $definitionLabel, 
> $serviceName, $alertState ,$alertText ,$alertTimestamp, $hostname  >> $logFile
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/ambari-server/log_sh.log, we can see the expected result.
> 
> 
> Thanks,
> 
> yao lei
> 
>

Re: Review Request 57337: AMBARI-20317 Update stack advisor logic for getting enable atlas hook flag value

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57337/#review168007
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 6, 2017, 1:31 p.m., Mugdha Varadkar wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57337/
> ---
> 
> (Updated March 6, 2017, 1:31 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Sumit 
> Mohanty.
> 
> 
> Bugs: AMBARI-20317
> https://issues.apache.org/jira/browse/AMBARI-20317
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Stack advisor recommendation for atlas hooks not working as expected due to 
> AMBARI-20304 changes.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
>  5d87c4d 
>   ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
> f18bfe9 
> 
> 
> Diff: https://reviews.apache.org/r/57337/diff/1/
> 
> 
> Testing
> ---
> 
> test_recommendHiveConfigurations_with_atlas 
> (test_stack_advisor.TestHDP23StackAdvisor) ... ok
> test_recommendSqoopConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> test_recommendStormConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
> ... ok
> 
> 
> Thanks,
> 
> Mugdha Varadkar
> 
>

Re: Review Request 57295: Atlas service check fails during EU on wire encrypted cluster

[Dmytro Grinenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57295/#review168006
---


Ship it!




Ship It!

- Dmytro Grinenko


On March 6, 2017, 2:57 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57295/
> ---
> 
> (Updated March 6, 2017, 2:57 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro 
> Lisnichenko, and Nate Cole.
> 
> 
> Bugs: AMBARI-20308
> https://issues.apache.org/jira/browse/AMBARI-20308
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade to HDP 2.6, Atlas service checks are failing. This is 
> because in ATLAS-1427, `TLSv1.2` was added as the default protocol and 
> `TLSv1, TLSv1.1` were excluded. Some versions of cURL do not support TLSv1.2.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
> e5f07ba 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  2dbc468 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
>  0d2f1bc 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> a102d13 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 
> f0a4f05 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/application-properties.xml
>  47e1fb5 
> 
> 
> Diff: https://reviews.apache.org/r/57295/diff/2/
> 
> 
> Testing
> ---
> 
> Performed an EU to HDP 2.6 for Atlas and verified the property.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 57295: Atlas service check fails during EU on wire encrypted cluster

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57295/#review168005
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 6, 2017, 2:57 p.m., Jonathan Hurley wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57295/
> ---
> 
> (Updated March 6, 2017, 2:57 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Dmytro Grinenko, Dmitro 
> Lisnichenko, and Nate Cole.
> 
> 
> Bugs: AMBARI-20308
> https://issues.apache.org/jira/browse/AMBARI-20308
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> During an upgrade to HDP 2.6, Atlas service checks are failing. This is 
> because in ATLAS-1427, `TLSv1.2` was added as the default protocol and 
> `TLSv1, TLSv1.1` were excluded. Some versions of cURL do not support TLSv1.2.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
>  PRE-CREATION 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
> e5f07ba 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
>  2dbc468 
>   
> ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
>  0d2f1bc 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
> a102d13 
>   ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 
> f0a4f05 
>   
> ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/application-properties.xml
>  47e1fb5 
> 
> 
> Diff: https://reviews.apache.org/r/57295/diff/2/
> 
> 
> Testing
> ---
> 
> Performed an EU to HDP 2.6 for Atlas and verified the property.
> 
> 
> Thanks,
> 
> Jonathan Hurley
> 
>

Re: Review Request 57075: hive-site.xml, hbase-site.xml, etc. are not found in class path for Zeppelin

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57075/#review168002
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 1, 2017, 7:35 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57075/
> ---
> 
> (Updated March 1, 2017, 7:35 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-20200
> https://issues.apache.org/jira/browse/AMBARI-20200
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> hive-site.xml, hbase-site.xml, etc. are not found in class path for 
> Zeppelin's interpreter.
> 
> As a result of which JDBC:phoenix  on kerberos mode doesn't work.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml
>  677158c 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  8a1fad6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  16a2782 
> 
> 
> Diff: https://reviews.apache.org/r/57075/diff/6/
> 
> 
> Testing
> ---
> 
> Manually on CentOS6
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review168001
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 6, 2017, 4:25 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 6, 2017, 4:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  2a684dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
> 1b3ae25 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7ee66ef 
> 
> 
> Diff: https://reviews.apache.org/r/57324/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
> 2.4.2 to Ambari 2.5.0
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 57292: Services shows as Restart required after upgrading from Ambari-2.4.x to 2.5.0.

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57292/#review168000
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 6, 2017, 3:56 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57292/
> ---
> 
> (Updated March 6, 2017, 3:56 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-20304
> https://issues.apache.org/jira/browse/AMBARI-20304
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Properties are added during upgrade to 2.5
> * falcon.atlas.hook
> * hive.atlas.hook
> * sqoop.atlas.hook
> * storm.atlas.hook
> * yarn.nodemanager.log.retain-seconds
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml
>  5663f57286 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
>  12135805b5 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml
>  508cfab65c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml
>  3b814a9b54 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-site.xml
>  2810a230df 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/configuration/yarn-site.xml
>  7053473059 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6.GlusterFS/services/YARN/configuration/yarn-site.xml
>  26aac90975 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/services/YARN/configuration/yarn-site.xml
>  26aac90975 
> 
> 
> Diff: https://reviews.apache.org/r/57292/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 57326: storm DRPC_SERVER kerberos configs duplicate

[Alejandro Fernandez]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57326/#review167999
---




ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml
Lines 101 (patched)


Set  unless they need to be added to 
earlier stacks.


- Alejandro Fernandez


On March 6, 2017, 6:29 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57326/
> ---
> 
> (Updated March 6, 2017, 6:29 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Robert Levas, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18892
> https://issues.apache.org/jira/browse/AMBARI-18892
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> when ambari enables kerberos,  storm service ,nimbus_keytab and 
> nimbus_principal_name will duplicate
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml
>  4cfe3d5 
>   ambari-server/src/main/resources/common-services/STORM/0.9.1/kerberos.json 
> 20cc32d 
>   ambari-server/src/main/resources/common-services/STORM/1.0.1/kerberos.json 
> fa2f6db 
>   ambari-web/app/assets/data/stacks/HDP-2.1/service_components.json 0a8f20b 
> 
> 
> Diff: https://reviews.apache.org/r/57326/diff/1/
> 
> 
> Testing
> ---
> 
> done?
> 
> 
> File Attachments
> 
> 
> storm.png
>   
> https://reviews.apache.org/media/uploaded/files/2017/03/06/35c14040-f918-4f46-83e8-be51a01d5d53__storm.png
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 57292: Services shows as Restart required after upgrading from Ambari-2.4.x to 2.5.0.

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57292/#review167998
---


Ship it!




Ship It!

- Sumit Mohanty


On March 6, 2017, 3:56 p.m., Dmitro Lisnichenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57292/
> ---
> 
> (Updated March 6, 2017, 3:56 p.m.)
> 
> 
> Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-20304
> https://issues.apache.org/jira/browse/AMBARI-20304
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Properties are added during upgrade to 2.5
> * falcon.atlas.hook
> * hive.atlas.hook
> * sqoop.atlas.hook
> * storm.atlas.hook
> * yarn.nodemanager.log.retain-seconds
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml
>  5663f57286 
>   
> ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
>  12135805b5 
>   
> ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml
>  508cfab65c 
>   
> ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml
>  3b814a9b54 
>   
> ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-site.xml
>  2810a230df 
>   
> ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/configuration/yarn-site.xml
>  7053473059 
>   
> ambari-server/src/main/resources/stacks/HDP/2.0.6.GlusterFS/services/YARN/configuration/yarn-site.xml
>  26aac90975 
>   
> ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/services/YARN/configuration/yarn-site.xml
>  26aac90975 
> 
> 
> Diff: https://reviews.apache.org/r/57292/diff/2/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Dmitro Lisnichenko
> 
>

Re: Review Request 57326: storm DRPC_SERVER kerberos configs duplicate

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57326/#review167997
---




ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml
Lines 99-109 (patched)


Are these _new_ properties used anywhere?



ambari-server/src/main/resources/common-services/STORM/0.9.1/kerberos.json
Lines 92-113 (original), 92-113 (patched)


This Kerberos identity descriptor should really be a reference to the 
duplicated on in the `NIMBUS` component descriptor.  This can be done by 
setting a `reference` attrute like this:

```
  {
"name": "drpc_server",
"reference": "/STORM/NIMBUS/nimbus_server"
  }
```

Since I do not see any usage of `storm-env/numbus_drpc_keytab`, this change 
is potentially causing data loss.



ambari-server/src/main/resources/common-services/STORM/1.0.1/kerberos.json
Lines 122-141 (original), 122-141 (patched)


Same as above, related to the _reference_.


- Robert Levas


On March 6, 2017, 1:29 a.m., wang yaoxin wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57326/
> ---
> 
> (Updated March 6, 2017, 1:29 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Robert Levas, and Sid Wagle.
> 
> 
> Bugs: AMBARI-18892
> https://issues.apache.org/jira/browse/AMBARI-18892
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> when ambari enables kerberos,  storm service ,nimbus_keytab and 
> nimbus_principal_name will duplicate
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/configuration/storm-env.xml
>  4cfe3d5 
>   ambari-server/src/main/resources/common-services/STORM/0.9.1/kerberos.json 
> 20cc32d 
>   ambari-server/src/main/resources/common-services/STORM/1.0.1/kerberos.json 
> fa2f6db 
>   ambari-web/app/assets/data/stacks/HDP-2.1/service_components.json 0a8f20b 
> 
> 
> Diff: https://reviews.apache.org/r/57326/diff/1/
> 
> 
> Testing
> ---
> 
> done?
> 
> 
> File Attachments
> 
> 
> storm.png
>   
> https://reviews.apache.org/media/uploaded/files/2017/03/06/35c14040-f918-4f46-83e8-be51a01d5d53__storm.png
> 
> 
> Thanks,
> 
> wang yaoxin
> 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/
---

(Updated March 6, 2017, 11:25 a.m.)


Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, and 
Sid Wagle.


Changes
---

Changed code that fixes the existing JSON string to manipiulate JSON data 
rather than replace string data.


Bugs: AMBARI-20309
https://issues.apache.org/jira/browse/AMBARI-20309


Repository: ambari


Description
---

HBase Master CPU Utilization Alert is in unknown state due to kinit error:

```
Execution of '/usr/bin/kinit -c 
/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
 -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
/dev/null' returned 1. kinit: Client not found in Kerberos database while 
getting initial credentials
```

This issue is also seen in /var/log/krb5kdc.log:

```
Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
{18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
/etc/security/keytabs/spnego.service.key...@example.com for 
krbtgt/example@example.com, Client not found in Kerberos database
```

#Cause
It appears that the HBASE alerts.json file 
(`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
`kerberos_keytab` and `kerberos_principal` properties.

```
  {
"name": "hbase_master_cpu",
"label": "HBase Master CPU Utilization",
"description": "This host-level alert is triggered if CPU utilization 
of the HBase Master exceeds certain warning and critical thresholds. It checks 
the HBase Master JMX Servlet for the SystemCPULoad property. The threshold 
values are in percent.",
"interval": 5,
"scope": "ANY",
"enabled": true,
"source": {
  "type": "METRIC",
  "uri": {
"http": "{{hbase-site/hbase.master.info.port}}",
"default_port": 60010,
"connection_timeout": 5.0,
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
  },
  "reporting": {
"ok": {
  "text": "{1} CPU, load {0:.1%}"
},
"warning": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 200
},
"critical": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 250
},
"units" : "%",
"type": "PERCENT"
  },
  "jmx": {
"property_list": [
  "java.lang:type=OperatingSystem/SystemCpuLoad",
  "java.lang:type=OperatingSystem/AvailableProcessors"
],
"value": "{0} * 100"
  }
}
  }
```

Notice:
```
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```

#Solution
Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
`common-services/HBASE/0.96.0.2.0/alerts.json`:

```
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 2a684dc 
  ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
1b3ae25 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 7ee66ef 


Diff: https://reviews.apache.org/r/57324/diff/2/

Changes: https://reviews.apache.org/r/57324/diff/1-2/


Testing
---

Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
2.4.2 to Ambari 2.5.0

# Local test results: PASSED

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Robert Levas]

> On March 6, 2017, 9:16 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
> > Lines 220-230 (patched)
> > 
> >
> > That's one way of doing it :)
> > 
> > I prefer the more direct JSON approach, similar to:
> > 
> > ```
> > String source = atlasMetadataServerWebUI.getSource();
> > JsonObject sourceJson = new 
> > JsonParser().parse(source).getAsJsonObject();
> > 
> > JsonObject uriJson = sourceJson.get("uri").getAsJsonObject();
> > uriJson.remove("kerberos_keytab");
> > uriJson.remove("kerberos_principal");
> > uriJson.addProperty("kerberos_keytab", 
> > "{{cluster-env/smokeuser_keytab}}");
> > uriJson.addProperty("kerberos_principal", 
> > "{{cluster-env/smokeuser_principal_name}}");
> > ```
> > 
> > As long as you don't see a way for this to corrupt the JSON, then it's 
> > fine.
> 
> Robert Levas wrote:
> I though about this, but this way we only change the particilar issue at 
> hand.  I am not sure if a user would have changed/fixed this manually already 
> and I didn't want to blindly undo it.   However, it we think that it unlikely 
> that the original value would have been changed, I am open to go the explicit 
> JSON route.
> 
> Jonathan Hurley wrote:
> Totally makes sense ... however, perhaps you could verify the JSON after 
> saving it to ensure that it can be de-serialized from the DB? Just to be safe 
> since we're doing a string replace on a structured syntax.

I went with your approach with some verification that Ambari only updates the 
JSON if the expected data exists to begin with. See rev 2.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167976
---


On March 5, 2017, 4:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 4:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
>   

Re: Review Request 57281: Script-Based Alert Dispathers support passing more parameters to script

[Jonathan Hurley]

> On March 6, 2017, 9:21 a.m., Jonathan Hurley wrote:
> > Ship It!
> 
> yao lei wrote:
> Hi Jonathan,
> Thank you very much.
> Would you please help me to commit the patch to trunk if you are free?

Yes, I'm just running a quick test on this patch locally and I'll commit it 
shortly once it passes...


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57281/#review167979
---


On March 3, 2017, 6:56 a.m., yao lei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57281/
> ---
> 
> (Updated March 3, 2017, 6:56 a.m.)
> 
> 
> Review request for Ambari and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-20291
> https://issues.apache.org/jira/browse/AMBARI-20291
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Script-Based Alert Dispatcher now pass five parameters to script,including 
> alert definition name, definition label,service name, alert state, and alert 
> text.
> But if script can receive other two parameters from dispather,it will be 
> better.
> 1.hostname.
> Because hostname the alert for is not always included in alert text,although 
> it may be null like aggregate alerts.
> With it we can more quick to find the related host that occured alert.
> 2.alert timestamp.
> We may need to know the alert occurrence time ( state change time) more 
> exactly. After the alert happened,it will spend some time to schedule the 
> script to run.
> Without it,we can only regard the script start time as the alert occurrence 
> time.
> 
> We now use this feature to send alert information to mobile phone and suggest 
> also passing hostname and alert timestamp.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcher.java
>  907588d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/services/AlertNoticeDispatchService.java
>  174f31f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcherTest.java
>  9e0e406 
> 
> 
> Diff: https://reviews.apache.org/r/57281/diff/1/
> 
> 
> Testing
> ---
> 
> 1.cd ambari-server
>mvn test
> 
> 2.write a python script to receive parameters from dispatcher
> 
> 
> #!/usr/bin/python
> import sys
> import logging
> 
> def main():
> definitionName = sys.argv[1]
> definitionLabel = sys.argv[2]
> serviceName = sys.argv[3]
> alertState = sys.argv[4]
> alertText = sys.argv[5]
> alertTimestamp = sys.argv[6]
> hostname = sys.argv[7] if len(sys.argv)-1 == 7 else None
> 
> logFile='/var/log/ambari-server/log_py.log'
> 
> logging.basicConfig(filename = logFile, level = logging.DEBUG)
> logging.debug('received ' + str(len(sys.argv)-1) + ' parameters')
> for i in range(1, len(sys.argv)):
> logging.debug('parameter ' +str(i) + ':' +str(sys.argv[i]))
> 
> if __name__ == '__main__':
>main()
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/log/ambari-server/log_py.log,
> 
> 3.write a shell script to  receive parameters from dispatcher
> 
> 
> #!/bin/bash
> logFile=/var/log/ambari-server/log_sh.log
> definitionName=$1
> definitionLabel=$2
> serviceName=$3
> alertState=$4 
> alertText=$5
> alertTimestamp=$6
> hostname=$7
> 
> echo received $# parameters:  $definitionName, $definitionLabel, 
> $serviceName, $alertState ,$alertText ,$alertTimestamp, $hostname  >> $logFile
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/ambari-server/log_sh.log, we can see the expected result.
> 
> 
> Thanks,
> 
> yao lei
> 
>

Re: Review Request 57292: Services shows as Restart required after upgrading from Ambari-2.4.x to 2.5.0.

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57292/
---

(Updated March 6, 2017, 5:56 p.m.)


Review request for Ambari, Jonathan Hurley, Nate Cole, and Sumit Mohanty.


Changes
---

Disabled auto-add of yarn.nodemanager.log.retain-seconds property


Bugs: AMBARI-20304
https://issues.apache.org/jira/browse/AMBARI-20304


Repository: ambari


Description (updated)
---

Properties are added during upgrade to 2.5
* falcon.atlas.hook
* hive.atlas.hook
* sqoop.atlas.hook
* storm.atlas.hook
* yarn.nodemanager.log.retain-seconds


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/FALCON/0.5.0.2.1/configuration/falcon-env.xml
 5663f57286 
  
ambari-server/src/main/resources/common-services/HIVE/0.12.0.2.0/configuration/hive-env.xml
 12135805b5 
  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-env.xml
 508cfab65c 
  
ambari-server/src/main/resources/common-services/STORM/0.10.0/configuration/storm-env.xml
 3b814a9b54 
  
ambari-server/src/main/resources/common-services/YARN/2.1.0.2.0/configuration/yarn-site.xml
 2810a230df 
  
ambari-server/src/main/resources/stacks/BIGTOP/0.8/services/YARN/configuration/yarn-site.xml
 7053473059 
  
ambari-server/src/main/resources/stacks/HDP/2.0.6.GlusterFS/services/YARN/configuration/yarn-site.xml
 26aac90975 
  
ambari-server/src/main/resources/stacks/HDP/2.1.GlusterFS/services/YARN/configuration/yarn-site.xml
 26aac90975 


Diff: https://reviews.apache.org/r/57292/diff/2/

Changes: https://reviews.apache.org/r/57292/diff/1-2/


Testing
---

mvn clean test


Thanks,

Dmitro Lisnichenko

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Jonathan Hurley]

> On March 6, 2017, 9:16 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
> > Lines 220-230 (patched)
> > 
> >
> > That's one way of doing it :)
> > 
> > I prefer the more direct JSON approach, similar to:
> > 
> > ```
> > String source = atlasMetadataServerWebUI.getSource();
> > JsonObject sourceJson = new 
> > JsonParser().parse(source).getAsJsonObject();
> > 
> > JsonObject uriJson = sourceJson.get("uri").getAsJsonObject();
> > uriJson.remove("kerberos_keytab");
> > uriJson.remove("kerberos_principal");
> > uriJson.addProperty("kerberos_keytab", 
> > "{{cluster-env/smokeuser_keytab}}");
> > uriJson.addProperty("kerberos_principal", 
> > "{{cluster-env/smokeuser_principal_name}}");
> > ```
> > 
> > As long as you don't see a way for this to corrupt the JSON, then it's 
> > fine.
> 
> Robert Levas wrote:
> I though about this, but this way we only change the particilar issue at 
> hand.  I am not sure if a user would have changed/fixed this manually already 
> and I didn't want to blindly undo it.   However, it we think that it unlikely 
> that the original value would have been changed, I am open to go the explicit 
> JSON route.

Totally makes sense ... however, perhaps you could verify the JSON after saving 
it to ensure that it can be de-serialized from the DB? Just to be safe since 
we're doing a string replace on a structured syntax.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167976
---


On March 5, 2017, 4:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 4:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
>   

Re: Review Request 57075: hive-site.xml, hbase-site.xml, etc. are not found in class path for Zeppelin

[Sumit Mohanty]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57075/#review167988
---


Ship it!




Ship It!

- Sumit Mohanty


On March 1, 2017, 7:35 a.m., Prabhjyot Singh wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57075/
> ---
> 
> (Updated March 1, 2017, 7:35 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Alejandro Fernandez, DIPAYAN 
> BHOWMICK, Jayush Luniya, Prabhjyot Singh, Rohit Choudhary, Renjith Kamath, 
> and Sumit Mohanty.
> 
> 
> Bugs: AMBARI-20200
> https://issues.apache.org/jira/browse/AMBARI-20200
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> hive-site.xml, hbase-site.xml, etc. are not found in class path for 
> Zeppelin's interpreter.
> 
> As a result of which JDBC:phoenix  on kerberos mode doesn't work.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/configuration/zeppelin-env.xml
>  677158c 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/master.py
>  8a1fad6 
>   
> ambari-server/src/main/resources/common-services/ZEPPELIN/0.6.0.2.5/package/scripts/params.py
>  16a2782 
> 
> 
> Diff: https://reviews.apache.org/r/57075/diff/6/
> 
> 
> Testing
> ---
> 
> Manually on CentOS6
> 
> 
> Thanks,
> 
> Prabhjyot Singh
> 
>

Re: Review Request 57339: Server startup script keeps waiting even if DB consistency has failed

[Robert Levas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57339/#review167986
---


Ship it!




Ship It!

- Robert Levas


On March 6, 2017, 9:54 a.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57339/
> ---
> 
> (Updated March 6, 2017, 9:54 a.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Attila Magyar, Andrew Onischuk, 
> Dmytro Sen, Laszlo Puskas, Nate Cole, Oliver Szabo, Robert Levas, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-20319
> https://issues.apache.org/jira/browse/AMBARI-20319
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When DB check failed during ambari start up, the script waited some 50 
> seconds for the UI before timeout occured, even though the Java process had 
> exited. This is fixed. Also added an extra pintStackTrace() in cases when 
> unexpected error occured during startup. This is needed as testing showed 
> that logs are not always flushed on system exit and importan diagnostic 
> information was lost.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  9540ca3 
>   ambari-server/src/main/python/ambari_server/utils.py 6408285 
>   ambari-server/src/main/python/ambari_server_main.py 0cd19cc 
> 
> 
> Diff: https://reviews.apache.org/r/57339/diff/1/
> 
> 
> Testing
> ---
> 
> - Tested manually.
> - Run all python unit tests in ambari-server, all passed
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167985
---


Ship it!




Ship It!

- Laszlo Puskas


On March 5, 2017, 9:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 9:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  2a684dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
> 1b3ae25 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7ee66ef 
> 
> 
> Diff: https://reviews.apache.org/r/57324/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
> 2.4.2 to Ambari 2.5.0
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 57339: Server startup script keeps waiting even if DB consistency has failed

[Laszlo Puskas]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57339/#review167984
---


Ship it!




Ship It!

- Laszlo Puskas


On March 6, 2017, 2:54 p.m., Balázs Bence Sári wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57339/
> ---
> 
> (Updated March 6, 2017, 2:54 p.m.)
> 
> 
> Review request for Ambari, Attila Doroszlai, Attila Magyar, Andrew Onischuk, 
> Dmytro Sen, Laszlo Puskas, Nate Cole, Oliver Szabo, Robert Levas, Sandor 
> Magyari, and Sebastian Toader.
> 
> 
> Bugs: AMBARI-20319
> https://issues.apache.org/jira/browse/AMBARI-20319
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> When DB check failed during ambari start up, the script waited some 50 
> seconds for the UI before timeout occured, even though the Java process had 
> exited. This is fixed. Also added an extra pintStackTrace() in cases when 
> unexpected error occured during startup. This is needed as testing showed 
> that logs are not always flushed on system exit and importan diagnostic 
> information was lost.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
>  9540ca3 
>   ambari-server/src/main/python/ambari_server/utils.py 6408285 
>   ambari-server/src/main/python/ambari_server_main.py 0cd19cc 
> 
> 
> Diff: https://reviews.apache.org/r/57339/diff/1/
> 
> 
> Testing
> ---
> 
> - Tested manually.
> - Run all python unit tests in ambari-server, all passed
> 
> 
> Thanks,
> 
> Balázs Bence Sári
> 
>

Re: Review Request 57295: Atlas service check fails during EU on wire encrypted cluster

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57295/
---

(Updated March 6, 2017, 9:57 a.m.)


Review request for Ambari, Dmitro Lisnichenko and Nate Cole.


Changes
---

It turns out that this Atlas change was backported into a maintenance branch, 
and is therefore present starting in HDP 2.5.4 ... so we need to do a bit more 
here on upgrade to make sure it gets set.


Bugs: AMBARI-20308
https://issues.apache.org/jira/browse/AMBARI-20308


Repository: ambari


Description
---

During an upgrade to HDP 2.6, Atlas service checks are failing. This is because 
in ATLAS-1427, `TLSv1.2` was added as the default protocol and `TLSv1, TLSv1.1` 
were excluded. Some versions of cURL do not support TLSv1.2.


Diffs (updated)
-

  
ambari-server/src/main/resources/stacks/HDP/2.5/services/ATLAS/configuration/application-properties.xml
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml 
e5f07ba 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.5.xml
 2dbc468 
  
ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
 0d2f1bc 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.5.xml 
a102d13 
  ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml 
f0a4f05 
  
ambari-server/src/main/resources/stacks/HDP/2.6/services/ATLAS/configuration/application-properties.xml
 47e1fb5 


Diff: https://reviews.apache.org/r/57295/diff/2/

Changes: https://reviews.apache.org/r/57295/diff/1-2/


Testing
---

Performed an EU to HDP 2.6 for Atlas and verified the property.


Thanks,

Jonathan Hurley

Review Request 57339: Server startup script keeps waiting even if DB consistency has failed

[Balázs Bence Sári]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57339/
---

Review request for Ambari, Attila Doroszlai, Attila Magyar, Andrew Onischuk, 
Dmytro Sen, Laszlo Puskas, Nate Cole, Oliver Szabo, Robert Levas, Sandor 
Magyari, and Sebastian Toader.


Bugs: AMBARI-20319
https://issues.apache.org/jira/browse/AMBARI-20319


Repository: ambari


Description
---

When DB check failed during ambari start up, the script waited some 50 seconds 
for the UI before timeout occured, even though the Java process had exited. 
This is fixed. Also added an extra pintStackTrace() in cases when unexpected 
error occured during startup. This is needed as testing showed that logs are 
not always flushed on system exit and importan diagnostic information was lost.


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/controller/AmbariServer.java
 9540ca3 
  ambari-server/src/main/python/ambari_server/utils.py 6408285 
  ambari-server/src/main/python/ambari_server_main.py 0cd19cc 


Diff: https://reviews.apache.org/r/57339/diff/1/


Testing
---

- Tested manually.
- Run all python unit tests in ambari-server, all passed


Thanks,

Balázs Bence Sári

Re: Review Request 57281: Script-Based Alert Dispathers support passing more parameters to script

[yao lei]

> On 三月 6, 2017, 2:21 p.m., Jonathan Hurley wrote:
> > Ship It!

Hi Jonathan,
Thank you very much.
Would you please help me to commit the patch to trunk if you are free?


- yao


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57281/#review167979
---


On 三月 3, 2017, 11:56 a.m., yao lei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57281/
> ---
> 
> (Updated 三月 3, 2017, 11:56 a.m.)
> 
> 
> Review request for Ambari and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-20291
> https://issues.apache.org/jira/browse/AMBARI-20291
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Script-Based Alert Dispatcher now pass five parameters to script,including 
> alert definition name, definition label,service name, alert state, and alert 
> text.
> But if script can receive other two parameters from dispather,it will be 
> better.
> 1.hostname.
> Because hostname the alert for is not always included in alert text,although 
> it may be null like aggregate alerts.
> With it we can more quick to find the related host that occured alert.
> 2.alert timestamp.
> We may need to know the alert occurrence time ( state change time) more 
> exactly. After the alert happened,it will spend some time to schedule the 
> script to run.
> Without it,we can only regard the script start time as the alert occurrence 
> time.
> 
> We now use this feature to send alert information to mobile phone and suggest 
> also passing hostname and alert timestamp.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcher.java
>  907588d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/services/AlertNoticeDispatchService.java
>  174f31f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcherTest.java
>  9e0e406 
> 
> 
> Diff: https://reviews.apache.org/r/57281/diff/1/
> 
> 
> Testing
> ---
> 
> 1.cd ambari-server
>mvn test
> 
> 2.write a python script to receive parameters from dispatcher
> 
> 
> #!/usr/bin/python
> import sys
> import logging
> 
> def main():
> definitionName = sys.argv[1]
> definitionLabel = sys.argv[2]
> serviceName = sys.argv[3]
> alertState = sys.argv[4]
> alertText = sys.argv[5]
> alertTimestamp = sys.argv[6]
> hostname = sys.argv[7] if len(sys.argv)-1 == 7 else None
> 
> logFile='/var/log/ambari-server/log_py.log'
> 
> logging.basicConfig(filename = logFile, level = logging.DEBUG)
> logging.debug('received ' + str(len(sys.argv)-1) + ' parameters')
> for i in range(1, len(sys.argv)):
> logging.debug('parameter ' +str(i) + ':' +str(sys.argv[i]))
> 
> if __name__ == '__main__':
>main()
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/log/ambari-server/log_py.log,
> 
> 3.write a shell script to  receive parameters from dispatcher
> 
> 
> #!/bin/bash
> logFile=/var/log/ambari-server/log_sh.log
> definitionName=$1
> definitionLabel=$2
> serviceName=$3
> alertState=$4 
> alertText=$5
> alertTimestamp=$6
> hostname=$7
> 
> echo received $# parameters:  $definitionName, $definitionLabel, 
> $serviceName, $alertState ,$alertText ,$alertTimestamp, $hostname  >> $logFile
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/ambari-server/log_sh.log, we can see the expected result.
> 
> 
> Thanks,
> 
> yao lei
> 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Robert Levas]

> On March 6, 2017, 9:16 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
> > Lines 220-230 (patched)
> > 
> >
> > That's one way of doing it :)
> > 
> > I prefer the more direct JSON approach, similar to:
> > 
> > ```
> > String source = atlasMetadataServerWebUI.getSource();
> > JsonObject sourceJson = new 
> > JsonParser().parse(source).getAsJsonObject();
> > 
> > JsonObject uriJson = sourceJson.get("uri").getAsJsonObject();
> > uriJson.remove("kerberos_keytab");
> > uriJson.remove("kerberos_principal");
> > uriJson.addProperty("kerberos_keytab", 
> > "{{cluster-env/smokeuser_keytab}}");
> > uriJson.addProperty("kerberos_principal", 
> > "{{cluster-env/smokeuser_principal_name}}");
> > ```
> > 
> > As long as you don't see a way for this to corrupt the JSON, then it's 
> > fine.

I though about this, but this way we only change the particilar issue at hand.  
I am not sure if a user would have changed/fixed this manually already and I 
didn't want to blindly undo it.   However, it we think that it unlikely that 
the original value would have been changed, I am open to go the explicit JSON 
route.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167976
---


On March 5, 2017, 4:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 4:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> 

Re: Review Request 57281: Script-Based Alert Dispathers support passing more parameters to script

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57281/#review167979
---


Ship it!




Ship It!

- Jonathan Hurley


On March 3, 2017, 6:56 a.m., yao lei wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57281/
> ---
> 
> (Updated March 3, 2017, 6:56 a.m.)
> 
> 
> Review request for Ambari and Jonathan Hurley.
> 
> 
> Bugs: AMBARI-20291
> https://issues.apache.org/jira/browse/AMBARI-20291
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Script-Based Alert Dispatcher now pass five parameters to script,including 
> alert definition name, definition label,service name, alert state, and alert 
> text.
> But if script can receive other two parameters from dispather,it will be 
> better.
> 1.hostname.
> Because hostname the alert for is not always included in alert text,although 
> it may be null like aggregate alerts.
> With it we can more quick to find the related host that occured alert.
> 2.alert timestamp.
> We may need to know the alert occurrence time ( state change time) more 
> exactly. After the alert happened,it will spend some time to schedule the 
> script to run.
> Without it,we can only regard the script start time as the alert occurrence 
> time.
> 
> We now use this feature to send alert information to mobile phone and suggest 
> also passing hostname and alert timestamp.
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcher.java
>  907588d 
>   
> ambari-server/src/main/java/org/apache/ambari/server/state/services/AlertNoticeDispatchService.java
>  174f31f 
>   
> ambari-server/src/test/java/org/apache/ambari/server/notifications/dispatchers/AlertScriptDispatcherTest.java
>  9e0e406 
> 
> 
> Diff: https://reviews.apache.org/r/57281/diff/1/
> 
> 
> Testing
> ---
> 
> 1.cd ambari-server
>mvn test
> 
> 2.write a python script to receive parameters from dispatcher
> 
> 
> #!/usr/bin/python
> import sys
> import logging
> 
> def main():
> definitionName = sys.argv[1]
> definitionLabel = sys.argv[2]
> serviceName = sys.argv[3]
> alertState = sys.argv[4]
> alertText = sys.argv[5]
> alertTimestamp = sys.argv[6]
> hostname = sys.argv[7] if len(sys.argv)-1 == 7 else None
> 
> logFile='/var/log/ambari-server/log_py.log'
> 
> logging.basicConfig(filename = logFile, level = logging.DEBUG)
> logging.debug('received ' + str(len(sys.argv)-1) + ' parameters')
> for i in range(1, len(sys.argv)):
> logging.debug('parameter ' +str(i) + ':' +str(sys.argv[i]))
> 
> if __name__ == '__main__':
>main()
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/log/ambari-server/log_py.log,
> 
> 3.write a shell script to  receive parameters from dispatcher
> 
> 
> #!/bin/bash
> logFile=/var/log/ambari-server/log_sh.log
> definitionName=$1
> definitionLabel=$2
> serviceName=$3
> alertState=$4 
> alertText=$5
> alertTimestamp=$6
> hostname=$7
> 
> echo received $# parameters:  $definitionName, $definitionLabel, 
> $serviceName, $alertState ,$alertText ,$alertTimestamp, $hostname  >> $logFile
> 
> 
> Stop and start any service like HDFS , we can see the expected result in 
> /var/ambari-server/log_sh.log, we can see the expected result.
> 
> 
> Thanks,
> 
> yao lei
> 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Jonathan Hurley]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167976
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
Lines 220-230 (patched)


That's one way of doing it :)

I prefer the more direct JSON approach, similar to:

```
String source = atlasMetadataServerWebUI.getSource();
JsonObject sourceJson = new 
JsonParser().parse(source).getAsJsonObject();

JsonObject uriJson = sourceJson.get("uri").getAsJsonObject();
uriJson.remove("kerberos_keytab");
uriJson.remove("kerberos_principal");
uriJson.addProperty("kerberos_keytab", 
"{{cluster-env/smokeuser_keytab}}");
uriJson.addProperty("kerberos_principal", 
"{{cluster-env/smokeuser_principal_name}}");
```

As long as you don't see a way for this to corrupt the JSON, then it's fine.


- Jonathan Hurley


On March 5, 2017, 4:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 4:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> 

Re: Review Request 57338: ulimit config missing for storm service

[Dmitro Lisnichenko]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57338/#review167975
---


Ship it!




Ship It!

- Dmitro Lisnichenko


On March 6, 2017, 3:26 p.m., Andrew Onischuk wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57338/
> ---
> 
> (Updated March 6, 2017, 3:26 p.m.)
> 
> 
> Review request for Ambari and Dmitro Lisnichenko.
> 
> 
> Bugs: AMBARI-20318
> https://issues.apache.org/jira/browse/AMBARI-20318
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
>   
> INFO  
> Cluster Machines.  
> 
>   
> Ambari:   
> Ambari Login credentials: Okta credentials  
> Host Login Credentials: Okta credentials  
> 
> 
> On host hcube2-1n02.eng.hortonworks.com which runs the storm service doesn't
> have ulimit config for storm service for e.x Nimbus
> 
> Below is the current ulimit config available
> 
> 
> 
> 
> (docker)[root@hcube2-1n02 ~]# cd /etc/security/limits.d/
> 90-nproc.conf   hbase.conf  hdfs.conf   hive.conf   
> kafka.conf  mapreduce.conf  oozie.conf  yarn.conf
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/params_linux.py
>  a176456 
>   
> ambari-server/src/main/resources/common-services/STORM/0.9.1/package/scripts/storm.py
>  f02ced4 
> 
> 
> Diff: https://reviews.apache.org/r/57338/diff/1/
> 
> 
> Testing
> ---
> 
> mvn clean test
> 
> 
> Thanks,
> 
> Andrew Onischuk
> 
>

Review Request 57337: AMBARI-20317 Update stack advisor logic for getting enable atlas hook flag value

[Mugdha Varadkar]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57337/
---

Review request for Ambari, Alejandro Fernandez, Gautam Borad, and Sumit Mohanty.


Bugs: AMBARI-20317
https://issues.apache.org/jira/browse/AMBARI-20317


Repository: ambari


Description
---

Stack advisor recommendation for atlas hooks not working as expected due to 
AMBARI-20304 changes.


Diffs
-

  
ambari-server/src/main/resources/common-services/SQOOP/1.4.4.2.0/configuration/sqoop-site.xml
 5d87c4d 
  ambari-server/src/main/resources/stacks/HDP/2.3/services/stack_advisor.py 
f18bfe9 


Diff: https://reviews.apache.org/r/57337/diff/1/


Testing
---

test_recommendHiveConfigurations_with_atlas 
(test_stack_advisor.TestHDP23StackAdvisor) ... ok
test_recommendSqoopConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ok
test_recommendStormConfigurations (test_stack_advisor.TestHDP23StackAdvisor) 
... ok


Thanks,

Mugdha Varadkar

Re: Review Request 57334: Move breadcrumbs to the separated view

[Denys Buzhor]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57334/#review167972
---


Ship it!




Ship It!

- Denys Buzhor


On March 6, 2017, 1 p.m., Oleg Nechiporenko wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57334/
> ---
> 
> (Updated March 6, 2017, 1 p.m.)
> 
> 
> Review request for Ambari and Denys Buzhor.
> 
> 
> Bugs: ambari-20136
> https://issues.apache.org/jira/browse/ambari-20136
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> Breadcrumbs items should be defined in the routes.
> 
> 
> Diffs
> -
> 
>   ambari-web/app/assets/test/tests.js 308edcd 
>   ambari-web/app/routes/main.js 4460442 
>   ambari-web/app/routes/view.js 50d8f4b 
>   ambari-web/app/routes/views.js 88b1251 
>   ambari-web/app/templates/application.hbs 9d6db78 
>   ambari-web/app/templates/common/breadcrumbs.hbs PRE-CREATION 
>   ambari-web/app/views.js 7ec59f7 
>   ambari-web/app/views/application.js ef9df6a 
>   ambari-web/app/views/common/breadcrumbs_view.js PRE-CREATION 
>   ambari-web/test/controllers/main/service/item_test.js e33ecaa 
>   ambari-web/test/views/common/breadcrumbs_view_test.js PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/57334/diff/1/
> 
> 
> Testing
> ---
> 
> 20576 passing (21s)
>   153 pending
> 
> 
> Thanks,
> 
> Oleg Nechiporenko
> 
>

Re: Review Request 57089: HDP 3.0 support for Slider with configs, kerberos, widgets, metrics, quicklinks, and themes

[Dmytro Sen]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57089/
---

(Updated Март 6, 2017, 11:32 д.п.)


Review request for Ambari, Alejandro Fernandez, Sid Wagle, and Vitalyi 
Brodetskyi.


Bugs: AMBARI-19835
https://issues.apache.org/jira/browse/AMBARI-19835


Repository: ambari


Description
---

HDP 3.0 support for Slider with configs, kerberos, widgets, metrics, 
quicklinks, and themes
Flatten from HDP 2.0.6 - 2.6 into common-services, and reference in HDP 3.0
In HDP 3.0, we have created a new stack definition that does not inherit from 
other stacks, in order to reduce the complexity of having to analyze older 
stacks.
This means that we need to create a service definition (metainfo.xml, configs, 
kerberos, widgets, metrics, quicklinks, and themes) that is equivalent to what 
is inherit and deleted from all of the previous stacks.
A merge needs to account for additions, overrides, and deletions.
metainfo.xml and configs perform a merge of older versions
kerberos.json always seems to override the previous file
Because the bits for this service may not yet be available in the HDP 3.0 repo, 
the task is to ensure that /api/v1/stacks/HDP/versions/2.6/services/SLIDER 
(which uses inheritance) is equivalent to the flattening of 
/api/v1/stacks/HDP/versions/3.0/services/SLIDER .
Please take a look at how this was done for ZK, HDFS, and YARN/MR.
This means that you will not be able to actually install the service for now, 
but can still perform validation during the Install Wizard that the correct 
components and configs show up.


Diffs (updated)
-

  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-client.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-env.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/configuration/slider-log4j.xml
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/kerberos.json
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/metainfo.xml 
PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/files/hbaseSmokeVerify.sh
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/__init__.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_linux.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/params_windows.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/service_check.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/scripts/slider_client.py
 PRE-CREATION 
  
ambari-server/src/main/resources/common-services/SLIDER/0.91.0.3.0/package/templates/storm-slider-env.sh.j2
 PRE-CREATION 
  ambari-server/src/main/resources/stacks/HDP/3.0/services/SLIDER/metainfo.xml 
PRE-CREATION 


Diff: https://reviews.apache.org/r/57089/diff/2/

Changes: https://reviews.apache.org/r/57089/diff/1-2/


Testing
---

Unit tests passed


Thanks,

Dmytro Sen

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

[Sebastian Toader]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167961
---


Ship it!




Ship It!

- Sebastian Toader


On March 5, 2017, 10:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 10:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  2a684dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
> 1b3ae25 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7ee66ef 
> 
> 
> Diff: https://reviews.apache.org/r/57324/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
> 2.4.2 to Ambari 2.5.0
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>