[
https://issues.apache.org/jira/browse/AMBARI-20309?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Robert Levas updated AMBARI-20309:
----------------------------------
Status: Patch Available (was: In Progress)
> HBase Master CPU Utilization Alert is in unknown state due to kinit error
> -------------------------------------------------------------------------
>
> Key: AMBARI-20309
> URL: https://issues.apache.org/jira/browse/AMBARI-20309
> Project: Ambari
> Issue Type: Bug
> Components: ambari-server
> Affects Versions: 2.5.0
> Reporter: Robert Levas
> Assignee: Robert Levas
> Labels: alerts, kerberos
> Fix For: 2.5.0
>
> Attachments: AMBARI-20309_branch-2.5_01.patch,
> AMBARI-20309_trunk_01.patch
>
>
> HBase Master CPU Utilization Alert is in unknown state due to kinit error:
> {noformat}
> Execution of '/usr/bin/kinit -c
> /var/lib/ambari-agent/tmp/curl_krb_cache/metric_alert_ambari-qa_cc_56787c2122a8214ca9775f3433361f8b
> -kt HTTP/[email protected] /etc/security/keytabs/spnego.service.keytab >
> /dev/null' returned 1. kinit: Client not found in Kerberos database while
> getting initial credentials
> {noformat}
> This issue is also seen in /var/log/krb5kdc.log:
> {noformat}
> Mar 03 16:43:06 c6401.ambari.apache.org krb5kdc[4749](info): AS_REQ (4 etypes
> {18 17 16 23}) 192.168.64.101: CLIENT_NOT_FOUND:
> /etc/security/keytabs/[email protected] for
> krbtgt/[email protected], Client not found in Kerberos database
> {noformat}
> *Cause*
> It appears that the HBASE alerts.json file
> ({{common-services/HBASE/0.96.0.2.0/alerts.json}}) has swapped values for the
> {{kerberos_keytab}} and {{kerberos_principal}} properties.
> {code}
> {
> "name": "hbase_master_cpu",
> "label": "HBase Master CPU Utilization",
> "description": "This host-level alert is triggered if CPU utilization
> of the HBase Master exceeds certain warning and critical thresholds. It
> checks the HBase Master JMX Servlet for the SystemCPULoad property. The
> threshold values are in percent.",
> "interval": 5,
> "scope": "ANY",
> "enabled": true,
> "source": {
> "type": "METRIC",
> "uri": {
> "http": "{{hbase-site/hbase.master.info.port}}",
> "default_port": 60010,
> "connection_timeout": 5.0,
> "kerberos_keytab":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> },
> "reporting": {
> "ok": {
> "text": "{1} CPU, load {0:.1%}"
> },
> "warning": {
> "text": "{1} CPU, load {0:.1%}",
> "value": 200
> },
> "critical": {
> "text": "{1} CPU, load {0:.1%}",
> "value": 250
> },
> "units" : "%",
> "type": "PERCENT"
> },
> "jmx": {
> "property_list": [
> "java.lang:type=OperatingSystem/SystemCpuLoad",
> "java.lang:type=OperatingSystem/AvailableProcessors"
> ],
> "value": "{0} * 100"
> }
> }
> }
> {code}
> Notice:
> {code}
> "kerberos_keytab":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_principal":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> {code}
> *Solution*
> Fix values for the {{kerberos_keytab}} and {{kerberos_principal}} properties
> in {{common-services/HBASE/0.96.0.2.0/alerts.json}}:
> {code}
> "kerberos_principal":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.principal}}",
> "kerberos_keytab":
> "{{hbase-site/hbase.security.authentication.spnego.kerberos.keytab}}"
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
