Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Robert Levas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/
---

(Updated March 6, 2017, 12:26 p.m.)


Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, and 
Sid Wagle.


Bugs: AMBARI-20309
https://issues.apache.org/jira/browse/AMBARI-20309


Repository: ambari


Description
---

HBase Master CPU Utilization Alert is in unknown state due to kinit error:

```
Execution of '/usr/bin/kinit -c 
/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
 -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
/dev/null' returned 1. kinit: Client not found in Kerberos database while 
getting initial credentials
```

This issue is also seen in /var/log/krb5kdc.log:

```
Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
{18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
/etc/security/keytabs/spnego.service.key...@example.com for 
krbtgt/example@example.com, Client not found in Kerberos database
```

#Cause
It appears that the HBASE alerts.json file 
(`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
`kerberos_keytab` and `kerberos_principal` properties.

```
  {
"name": "hbase_master_cpu",
"label": "HBase Master CPU Utilization",
"description": "This host-level alert is triggered if CPU utilization 
of the HBase Master exceeds certain warning and critical thresholds. It checks 
the HBase Master JMX Servlet for the SystemCPULoad property. The threshold 
values are in percent.",
"interval": 5,
"scope": "ANY",
"enabled": true,
"source": {
  "type": "METRIC",
  "uri": {
"http": "{{hbase-site/hbase.master.info.port}}",
"default_port": 60010,
"connection_timeout": 5.0,
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
  },
  "reporting": {
"ok": {
  "text": "{1} CPU, load {0:.1%}"
},
"warning": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 200
},
"critical": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 250
},
"units" : "%",
"type": "PERCENT"
  },
  "jmx": {
"property_list": [
  "java.lang:type=OperatingSystem/SystemCpuLoad",
  "java.lang:type=OperatingSystem/AvailableProcessors"
],
"value": "{0} * 100"
  }
}
  }
```

Notice:
```
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```

#Solution
Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
`common-services/HBASE/0.96.0.2.0/alerts.json`:

```
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 2a684dc 
  ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
1b3ae25 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 7ee66ef 


Diff: https://reviews.apache.org/r/57324/diff/2/


Testing (updated)
---

Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
2.4.2 to Ambari 2.5.0

# Local test results:

```
[INFO] 
[INFO] BUILD SUCCESS
[INFO] 
[INFO] Total time: 26:07.123s
[INFO] Finished at: Mon Mar 06 12:25:32 EST 2017
[INFO] Final Memory: 70M/596M
[INFO] 
```


# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review168010
---


Ship it!




Ship It!

- Jonathan Hurley


On March 6, 2017, 11:25 a.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 6, 2017, 11:25 a.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  2a684dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
> 1b3ae25 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7ee66ef 
> 
> 
> Diff: https://reviews.apache.org/r/57324/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
> 2.4.2 to Ambari 2.5.0
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Alejandro Fernandez 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review168001
---


Ship it!




Ship It!

- Alejandro Fernandez


On March 6, 2017, 4:25 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 6, 2017, 4:25 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  2a684dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
> 1b3ae25 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7ee66ef 
> 
> 
> Diff: https://reviews.apache.org/r/57324/diff/2/
> 
> 
> Testing
> ---
> 
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
> 2.4.2 to Ambari 2.5.0
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Robert Levas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/
---

(Updated March 6, 2017, 11:25 a.m.)


Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, and 
Sid Wagle.


Changes
---

Changed code that fixes the existing JSON string to manipiulate JSON data 
rather than replace string data.


Bugs: AMBARI-20309
https://issues.apache.org/jira/browse/AMBARI-20309


Repository: ambari


Description
---

HBase Master CPU Utilization Alert is in unknown state due to kinit error:

```
Execution of '/usr/bin/kinit -c 
/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
 -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
/dev/null' returned 1. kinit: Client not found in Kerberos database while 
getting initial credentials
```

This issue is also seen in /var/log/krb5kdc.log:

```
Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
{18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
/etc/security/keytabs/spnego.service.key...@example.com for 
krbtgt/example@example.com, Client not found in Kerberos database
```

#Cause
It appears that the HBASE alerts.json file 
(`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
`kerberos_keytab` and `kerberos_principal` properties.

```
  {
"name": "hbase_master_cpu",
"label": "HBase Master CPU Utilization",
"description": "This host-level alert is triggered if CPU utilization 
of the HBase Master exceeds certain warning and critical thresholds. It checks 
the HBase Master JMX Servlet for the SystemCPULoad property. The threshold 
values are in percent.",
"interval": 5,
"scope": "ANY",
"enabled": true,
"source": {
  "type": "METRIC",
  "uri": {
"http": "{{hbase-site/hbase.master.info.port}}",
"default_port": 60010,
"connection_timeout": 5.0,
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
  },
  "reporting": {
"ok": {
  "text": "{1} CPU, load {0:.1%}"
},
"warning": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 200
},
"critical": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 250
},
"units" : "%",
"type": "PERCENT"
  },
  "jmx": {
"property_list": [
  "java.lang:type=OperatingSystem/SystemCpuLoad",
  "java.lang:type=OperatingSystem/AvailableProcessors"
],
"value": "{0} * 100"
  }
}
  }
```

Notice:
```
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```

#Solution
Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
`common-services/HBASE/0.96.0.2.0/alerts.json`:

```
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```


Diffs (updated)
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 2a684dc 
  ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
1b3ae25 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 7ee66ef 


Diff: https://reviews.apache.org/r/57324/diff/2/

Changes: https://reviews.apache.org/r/57324/diff/1-2/


Testing
---

Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
2.4.2 to Ambari 2.5.0

# Local test results: PASSED

# Jenkins test results: PENDING


Thanks,

Robert Levas

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Robert Levas 


> On March 6, 2017, 9:16 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
> > Lines 220-230 (patched)
> > 
> >
> > That's one way of doing it :)
> > 
> > I prefer the more direct JSON approach, similar to:
> > 
> > ```
> > String source = atlasMetadataServerWebUI.getSource();
> > JsonObject sourceJson = new 
> > JsonParser().parse(source).getAsJsonObject();
> > 
> > JsonObject uriJson = sourceJson.get("uri").getAsJsonObject();
> > uriJson.remove("kerberos_keytab");
> > uriJson.remove("kerberos_principal");
> > uriJson.addProperty("kerberos_keytab", 
> > "{{cluster-env/smokeuser_keytab}}");
> > uriJson.addProperty("kerberos_principal", 
> > "{{cluster-env/smokeuser_principal_name}}");
> > ```
> > 
> > As long as you don't see a way for this to corrupt the JSON, then it's 
> > fine.
> 
> Robert Levas wrote:
> I though about this, but this way we only change the particilar issue at 
> hand.  I am not sure if a user would have changed/fixed this manually already 
> and I didn't want to blindly undo it.   However, it we think that it unlikely 
> that the original value would have been changed, I am open to go the explicit 
> JSON route.
> 
> Jonathan Hurley wrote:
> Totally makes sense ... however, perhaps you could verify the JSON after 
> saving it to ensure that it can be de-serialized from the DB? Just to be safe 
> since we're doing a string replace on a structured syntax.

I went with your approach with some verification that Ambari only updates the 
JSON if the expected data exists to begin with. See rev 2.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167976
---


On March 5, 2017, 4:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 4:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Jonathan Hurley 


> On March 6, 2017, 9:16 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
> > Lines 220-230 (patched)
> > 
> >
> > That's one way of doing it :)
> > 
> > I prefer the more direct JSON approach, similar to:
> > 
> > ```
> > String source = atlasMetadataServerWebUI.getSource();
> > JsonObject sourceJson = new 
> > JsonParser().parse(source).getAsJsonObject();
> > 
> > JsonObject uriJson = sourceJson.get("uri").getAsJsonObject();
> > uriJson.remove("kerberos_keytab");
> > uriJson.remove("kerberos_principal");
> > uriJson.addProperty("kerberos_keytab", 
> > "{{cluster-env/smokeuser_keytab}}");
> > uriJson.addProperty("kerberos_principal", 
> > "{{cluster-env/smokeuser_principal_name}}");
> > ```
> > 
> > As long as you don't see a way for this to corrupt the JSON, then it's 
> > fine.
> 
> Robert Levas wrote:
> I though about this, but this way we only change the particilar issue at 
> hand.  I am not sure if a user would have changed/fixed this manually already 
> and I didn't want to blindly undo it.   However, it we think that it unlikely 
> that the original value would have been changed, I am open to go the explicit 
> JSON route.

Totally makes sense ... however, perhaps you could verify the JSON after saving 
it to ensure that it can be de-serialized from the DB? Just to be safe since 
we're doing a string replace on a structured syntax.


- Jonathan


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167976
---


On March 5, 2017, 4:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 4:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Laszlo Puskas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167985
---


Ship it!




Ship It!

- Laszlo Puskas


On March 5, 2017, 9:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 9:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  2a684dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
> 1b3ae25 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7ee66ef 
> 
> 
> Diff: https://reviews.apache.org/r/57324/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
> 2.4.2 to Ambari 2.5.0
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Robert Levas 


> On March 6, 2017, 9:16 a.m., Jonathan Hurley wrote:
> > ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
> > Lines 220-230 (patched)
> > 
> >
> > That's one way of doing it :)
> > 
> > I prefer the more direct JSON approach, similar to:
> > 
> > ```
> > String source = atlasMetadataServerWebUI.getSource();
> > JsonObject sourceJson = new 
> > JsonParser().parse(source).getAsJsonObject();
> > 
> > JsonObject uriJson = sourceJson.get("uri").getAsJsonObject();
> > uriJson.remove("kerberos_keytab");
> > uriJson.remove("kerberos_principal");
> > uriJson.addProperty("kerberos_keytab", 
> > "{{cluster-env/smokeuser_keytab}}");
> > uriJson.addProperty("kerberos_principal", 
> > "{{cluster-env/smokeuser_principal_name}}");
> > ```
> > 
> > As long as you don't see a way for this to corrupt the JSON, then it's 
> > fine.

I though about this, but this way we only change the particilar issue at hand.  
I am not sure if a user would have changed/fixed this manually already and I 
didn't want to blindly undo it.   However, it we think that it unlikely that 
the original value would have been changed, I am open to go the explicit JSON 
route.


- Robert


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167976
---


On March 5, 2017, 4:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 4:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Jonathan Hurley 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167976
---


Ship it!





ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
Lines 220-230 (patched)


That's one way of doing it :)

I prefer the more direct JSON approach, similar to:

```
String source = atlasMetadataServerWebUI.getSource();
JsonObject sourceJson = new 
JsonParser().parse(source).getAsJsonObject();

JsonObject uriJson = sourceJson.get("uri").getAsJsonObject();
uriJson.remove("kerberos_keytab");
uriJson.remove("kerberos_principal");
uriJson.addProperty("kerberos_keytab", 
"{{cluster-env/smokeuser_keytab}}");
uriJson.addProperty("kerberos_principal", 
"{{cluster-env/smokeuser_principal_name}}");
```

As long as you don't see a way for this to corrupt the JSON, then it's fine.


- Jonathan Hurley


On March 5, 2017, 4:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 4:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
>

Re: Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-06 Thread Sebastian Toader 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/#review167961
---


Ship it!




Ship It!

- Sebastian Toader


On March 5, 2017, 10:38 p.m., Robert Levas wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57324/
> ---
> 
> (Updated March 5, 2017, 10:38 p.m.)
> 
> 
> Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
> Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, 
> and Sid Wagle.
> 
> 
> Bugs: AMBARI-20309
> https://issues.apache.org/jira/browse/AMBARI-20309
> 
> 
> Repository: ambari
> 
> 
> Description
> ---
> 
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> 
> ```
> Execution of '/usr/bin/kinit -c 
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
>  -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
> /dev/null' returned 1. kinit: Client not found in Kerberos database while 
> getting initial credentials
> ```
> 
> This issue is also seen in /var/log/krb5kdc.log:
> 
> ```
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
> /etc/security/keytabs/spnego.service.key...@example.com for 
> krbtgt/example@example.com, Client not found in Kerberos database
> ```
> 
> #Cause
> It appears that the HBASE alerts.json file 
> (`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
> `kerberos_keytab` and `kerberos_principal` properties.
> 
> ```
>   {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization 
> of the HBase Master exceeds certain warning and critical thresholds. It 
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The 
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
>   "type": "METRIC",
>   "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
>   },
>   "reporting": {
> "ok": {
>   "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 200
> },
> "critical": {
>   "text": "{1} CPU, load {0:.1%}",
>   "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
>   },
>   "jmx": {
> "property_list": [
>   "java.lang:type=OperatingSystem/SystemCpuLoad",
>   "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
>   }
> }
>   }
> ```
> 
> Notice:
> ```
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> #Solution
> Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
> `common-services/HBASE/0.96.0.2.0/alerts.json`:
> 
> ```
> "kerberos_principal": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab": 
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> ```
> 
> 
> Diffs
> -
> 
>   
> ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
>  2a684dc 
>   
> ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
> 1b3ae25 
>   
> ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
>  7ee66ef 
> 
> 
> Diff: https://reviews.apache.org/r/57324/diff/1/
> 
> 
> Testing
> ---
> 
> Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
> 2.4.2 to Ambari 2.5.0
> 
> # Local test results: PASSED
> 
> # Jenkins test results: PENDING
> 
> 
> Thanks,
> 
> Robert Levas
> 
>

Review Request 57324: HBase Master CPU Utilization Alert is in unknown state due to kinit error

2017-03-05 Thread Robert Levas 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57324/
---

Review request for Ambari, Alejandro Fernandez, Attila Magyar, Balázs Bence 
Sári, Eugene Chekanskiy, Jonathan Hurley, Laszlo Puskas, Sebastian Toader, and 
Sid Wagle.


Bugs: AMBARI-20309
https://issues.apache.org/jira/browse/AMBARI-20309


Repository: ambari


Description
---

HBase Master CPU Utilization Alert is in unknown state due to kinit error:

```
Execution of '/usr/bin/kinit -c 
/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
 -kt HTTP/_h...@example.com /etc/security/keytabs/spnego.service.keytab > 
/dev/null' returned 1. kinit: Client not found in Kerberos database while 
getting initial credentials
```

This issue is also seen in /var/log/krb5kdc.log:

```
Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes 
{18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND: 
/etc/security/keytabs/spnego.service.key...@example.com for 
krbtgt/example@example.com, Client not found in Kerberos database
```

#Cause
It appears that the HBASE alerts.json file 
(`common-services/HBASE/0.96.0.2.0/alerts.json`) has swapped values for the 
`kerberos_keytab` and `kerberos_principal` properties.

```
  {
"name": "hbase_master_cpu",
"label": "HBase Master CPU Utilization",
"description": "This host-level alert is triggered if CPU utilization 
of the HBase Master exceeds certain warning and critical thresholds. It checks 
the HBase Master JMX Servlet for the SystemCPULoad property. The threshold 
values are in percent.",
"interval": 5,
"scope": "ANY",
"enabled": true,
"source": {
  "type": "METRIC",
  "uri": {
"http": "{{hbase-site/hbase.master.info.port}}",
"default_port": 60010,
"connection_timeout": 5.0,
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
  },
  "reporting": {
"ok": {
  "text": "{1} CPU, load {0:.1%}"
},
"warning": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 200
},
"critical": {
  "text": "{1} CPU, load {0:.1%}",
  "value": 250
},
"units" : "%",
"type": "PERCENT"
  },
  "jmx": {
"property_list": [
  "java.lang:type=OperatingSystem/SystemCpuLoad",
  "java.lang:type=OperatingSystem/AvailableProcessors"
],
"value": "{0} * 100"
  }
}
  }
```

Notice:
```
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```

#Solution
Fix values for the `kerberos_keytab` and `kerberos_principal` properties in 
`common-services/HBASE/0.96.0.2.0/alerts.json`:

```
"kerberos_principal": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_keytab": 
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
```


Diffs
-

  
ambari-server/src/main/java/org/apache/ambari/server/upgrade/UpgradeCatalog250.java
 2a684dc 
  ambari-server/src/main/resources/common-services/HBASE/0.96.0.2.0/alerts.json 
1b3ae25 
  
ambari-server/src/test/java/org/apache/ambari/server/upgrade/UpgradeCatalog250Test.java
 7ee66ef 


Diff: https://reviews.apache.org/r/57324/diff/1/


Testing
---

Manually tested in new Ambari 2.5.0 cluster and upgrade scenario from Ambari 
2.4.2 to Ambari 2.5.0

# Local test results: PASSED

# Jenkins test results: PENDING


Thanks,

Robert Levas