Robert Levas created AMBARI-20309:
-------------------------------------
Summary: HBase Master CPU Utilization Alert is in unknown state
due to kinit error
Key: AMBARI-20309
URL: https://issues.apache.org/jira/browse/AMBARI-20309
Project: Ambari
Issue Type: Bug
Components: ambari-server
Affects Versions: 2.5.0
Reporter: Robert Levas
Assignee: Robert Levas
Fix For: 2.5.0
HBase Master CPU Utilization Alert is in unknown state due to kinit error:
{noformat}
Execution of '/usr/bin/kinit -c
/var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
-kt HTTP/[email protected] /etc/security/keytabs/spnego.service.keytab >
/dev/null' returned 1. kinit: Client not found in Kerberos database while
getting initial credentials
{noformat}
This issue is also seen in /var/log/krb5kdc.log:
{noformat}
Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes
{18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND:
/etc/security/keytabs/[email protected] for
krbtgt/[email protected], Client not found in Kerberos database
{noformat}
*Cause*
It appears that the HBASE alerts.json file
({{common-services/HBASE/0.96.0.2.0/alerts.json}}) has swapped values for the
{{kerberos_keytab}} and {{kerberos_principal}} properties.
{code}
{
"name": "hbase_master_cpu",
"label": "HBase Master CPU Utilization",
"description": "This host-level alert is triggered if CPU utilization
of the HBase Master exceeds certain warning and critical thresholds. It checks
the HBase Master JMX Servlet for the SystemCPULoad property. The threshold
values are in percent.",
"interval": 5,
"scope": "ANY",
"enabled": true,
"source": {
"type": "METRIC",
"uri": {
"http": "{{hbase-site/hbase.master.info.port}}",
"default_port": 60010,
"connection_timeout": 5.0,
"kerberos_keytab":
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal":
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
},
"reporting": {
"ok": {
"text": "{1} CPU, load {0:.1%}"
},
"warning": {
"text": "{1} CPU, load {0:.1%}",
"value": 200
},
"critical": {
"text": "{1} CPU, load {0:.1%}",
"value": 250
},
"units" : "%",
"type": "PERCENT"
},
"jmx": {
"property_list": [
"java.lang:type=OperatingSystem/SystemCpuLoad",
"java.lang:type=OperatingSystem/AvailableProcessors"
],
"value": "{0} * 100"
}
}
}
{code}
Notice:
{code}
"kerberos_keytab":
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_principal":
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
{code}
*Solution*
Fix values for the {{kerberos_keytab}} and {{kerberos_principal}} properties in
{{common-services/HBASE/0.96.0.2.0/alerts.json}}:
{code}
"kerberos_principal":
"{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
"kerberos_keytab":
"{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
