Michael Ho created KUDU-2545:
--------------------------------
Summary: KinitContext::DoRenewal() is unable to recover after
credentials cache gets deleted
Key: KUDU-2545
URL: https://issues.apache.org/jira/browse/KUDU-2545
Project: Kudu
Issue Type: Bug
Components: security
Affects Versions: 1.7.0, 1.6.0, 1.5.0, 1.4.0, 1.3.0
Reporter: Michael Ho
Impala recently switched over to using the Kerberos implementation in Kudu for
kinit and acquiring a new the TGT periodically.
A user ran into a situation in which {{krb5_get_init_creds_keytab()}} called
from {{KinitContext::DoRenewal()}} failed because of IO failure to the
credentials cache. Apparently, the credentials cache get deleted afterwards.
It's unclear if it was done by some clean up code in Kerberos library or
something external to Kerberos which deleted that credentials cache. The IO
failure to store credentials suggested the credentials cache still existed when
the failure occurred.
On the next call to {{KinitContext::DoRenewal()}}, the credentials cache was
gone so it failed right away on the call to {{krb5_cc_start_seq_get()}} a the
beginning of the function. This failure kept happening until the user restarted
the Impala service, at which point the credentials cache got recreated. It
seems that {{KinitContext::DoRenewal()}} *cannot recover at all* once the
credentials cache disappears somehow. It seems that the code could be more
robust if it handles the failure by reverting to calling
{{KinitContext::Kinit()}} if the credentials cache disappeared somehow.
{noformat}
| W0815 10:04:01.132095 144773 init.cc:180] Kerberos reacquire error: : Runtime
error: Reacquire error: unable to login from keytab: Failed to store
credentials: Credentials cache I/O operation failed (filename:
/tmp/krb5cc_impala_internal)
| W0815 10:05:37.133210 144773 init.cc:180] Kerberos reacquire error: :
Runtime error: Failed to peek into ccache: No credentials cache found
(filename: /tmp/krb5cc_impala_internal)
| W0815 10:10:01.133746 144773 init.cc:180] Kerberos reacquire error: :
Runtime error: Failed to peek into ccache: No credentials cache found
(filename: /tmp/krb5cc_impala_internal)
| W0815 10:18:26.134222 144773 init.cc:180] Kerberos reacquire error: :
Runtime error: Failed to peek into ccache: No credentials cache found
(filename: /tmp/krb5cc_impala_internal)
| W0815 10:41:30.134889 144773 init.cc:180] Kerberos reacquire error: :
Runtime error: Failed to peek into ccache: No credentials cache found
(filename: /tmp/krb5cc_impala_internal)
| W0815 10:59:01.135460 144773 init.cc:180] Kerberos reacquire error: :
Runtime error: Failed to peek into ccache: No credentials cache found
(filename: /tmp/krb5cc_impala_internal)
| W0815 12:24:18.135974 144773 init.cc:180] Kerberos reacquire error: :
Runtime error: Failed to peek into ccache: No credentials cache found
(filename: /tmp/krb5cc_impala_internal)
| W0815 13:49:35.136660 144773 init.cc:180] Kerberos reacquire error: :
Runtime error: Failed to peek into ccache: No credentials cache found
(filename: /tmp/krb5cc_impala_internal)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
