Your principal isn't getting cached for some reason. The same on my machine looks like:
$ klist /tmp/krb5cc_impala_internal Ticket cache: FILE:/tmp/krb5cc_impala_internal Default principal: impala/[email protected] Valid starting Expires Service principal 01/08/2018 15:49:30 01/09/2018 15:49:30 krbtgt/[email protected] renew until 01/15/2018 15:49:30 01/08/2018 15:49:30 01/09/2018 15:49:30 impala/localhost@ renew until 01/15/2018 15:49:30 01/08/2018 15:49:30 01/09/2018 15:49:30 impala/[email protected] renew until 01/15/2018 15:49:30 Did you check if your user has appropriate permissions on the file? Looks like only the MiniKDC was able to write to it, and maybe your 'kinit' couldn't and silently failed (which should be a bug if it didn't throw an error) ? On Mon, Jan 8, 2018 at 3:40 PM, Philip Zeyliger <[email protected]> wrote: > Hi Sailiesh, > > Is this what you'd expect? > > $klist /tmp/krb5cc_impala_internal > Ticket cache: FILE:/tmp/krb5cc_impala_internal > Default principal: impala/[email protected] > > Valid starting Expires Service principal > 01/08/2018 15:39:23 01/09/2018 15:39:23 krbtgt/[email protected] > renew until 01/15/2018 15:39:23 > > Thanks! > > > On Mon, Jan 8, 2018 at 12:20 PM, Sailesh Mukil <[email protected]> > wrote: > > > Can you run the test again, and klist the contents of the credential > cache > > and post the error logs again? Looks like "impala/localhost" might not be > > stored as expected in the cache on your machine. > > > > On Wed, Dec 13, 2017 at 2:47 PM, Philip Zeyliger <[email protected]> > > wrote: > > > > > The KDC in this case is the "minikdc" from > > > https://github.com/apache/impala/blob/master/be/src/ > > > kudu/security/test/mini_kdc.cc. > > > I see evidence of it, and have been able to look at its configuration > by, > > > um, adding --gtest_break_on_failure. (The feature actually doesn't > work, > > > presumably because of an interaction with breakpad, but a temporary > > > directory is left on my filesystem, so that's nice.) > > > > > > -- Philip > > > > > > On Tue, Dec 12, 2017 at 4:08 PM, Evo Eftimov <[email protected]> > > > wrote: > > > > > > > Is your cluster Kerberized at all, especially the Impala daemon - it > > > > doesn’t seem to be enrolled in the KDC at all > > > > > > > > You / your personal account/principal is definitely enrolled though > > > > > > > > And there is definetly a KDC in your environment > > > > > > > > -----Original Message----- > > > > From: Philip Zeyliger [mailto:[email protected]] > > > > Sent: Tuesday, December 12, 2017 11:26 PM > > > > To: [email protected] > > > > Subject: thrift-server-test > > > > > > > > Hi folks, > > > > > > > > I've been running into issues with thrift-server-test and Kerberos. > > Below > > > > is an excerpt of "KRB5_TRACE=/dev/stderr be/build/debug/rpc/thrift- > > > server-test"; > > > > both SslConnectivity/1 and > > > > SslConnectivity/2 fail the same way. > > > > > > > > I'm running Ubuntu16.04. I've seen this both on my host, as well as > > > inside > > > > of an Ubuntu 16.04 Docker container. > > > > > > > > Does this ring any bells? > > > > > > > > Thanks! > > > > > > > > -- Philip > > > > > > > > > > > > [ RUN ] KerberosOnAndOff/ThriftKerberizedParamsTest. > > > SslConnectivity/2 > > > > Loading random data > > > > Initializing database '7abf-cef9-113e-eae3/krb5kdc/principal' for > > realm > > > ' > > > > KRBTEST.COM', > > > > master key name 'K/[email protected]' > > > > [31585] 1513120922.459517: Retrieving K/[email protected] from > > > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) > > with > > > > result: 0/Success > > > > [31586] 1513120922.472314: Retrieving K/[email protected] from > > > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) > > with > > > > result: 0/Success > > > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): > > > setting > > > > up network... > > > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): > > > > listening on fd 11: udp 0.0.0.0.51781 (pktinfo) > > > > krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked Dec 12 15:22:02 > > > > philip-dev.gce.cloudera.com krb5kdc[31586](info): listening on fd > 12: > > > udp > > > > ::.51781 (pktinfo) Dec 12 15:22:02 philip-dev.gce.cloudera.com > > > > krb5kdc[31586](info): set up 2 sockets Dec 12 15:22:02 > > > > philip-dev.gce.cloudera.com krb5kdc[31586](info): > > > > commencing operation > > > > krb5kdc: starting... > > > > Authenticating as principal philip/[email protected] with password. > > > > [31589] 1513120922.498913: Retrieving K/[email protected] from > > > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) > > with > > > > result: 0/Success > > > > WARNING: no policy specified for impala/[email protected]; > > > defaulting > > > > to no policy Principal "impala/[email protected]" created. > > > > Authenticating as principal philip/[email protected] with password. > > > > [31590] 1513120922.508777: Retrieving K/[email protected] from > > > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) > > with > > > > result: 0/Success > > > > Entry for principal impala/localhost with kvno 2, encryption type > > > > aes256-cts-hmac-sha1-96 added to keytab > > > > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab. > > > > Entry for principal impala/localhost with kvno 2, encryption type > > > > aes128-cts-hmac-sha1-96 added to keytab > > > > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab. > > > > Entry for principal impala/localhost with kvno 2, encryption type > > > > des3-cbc-sha1 added to keytab > > > > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab. > > > > Entry for principal impala/localhost with kvno 2, encryption type > > > > arcfour-hmac added to keytab WRFILE:7abf-cef9-113e-eae3/ > > > > krb5kdc/impala_localhost.keytab. > > > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): > > AS_REQ > > > > (6 etypes {18 17 16 23 25 26}) 127.0.0.1: ISSUE: authtime > 1513120922, > > > > etypes > > > > {rep=18 tkt=18 ses=18}, impala/[email protected] for krbtgt/ > > > > [email protected] [31476] 1513120922.532304: ccselect can't > find > > > > appropriate cache for server principal impala@localhost [31476] > > > > 1513120922.532347: Getting credentials impala/[email protected] > > > > -> impala@localhost using ccache FILE:/tmp/krb5cc_impala_internal > > > > [31476] 1513120922.532382: Retrieving impala/[email protected] > -> > > > > impala@localhost from FILE:/tmp/krb5cc_impala_internal with result: > > > > -1765328243/Matching credential not found [31476] 1513120922.532407: > > > > Retrieving impala/[email protected] -> > krbtgt/localhost@localhost > > > > from FILE:/tmp/krb5cc_impala_internal with > > > > result: -1765328243/Matching credential not found [31476] > > > > 1513120922.532433: Retrieving impala/[email protected] -> > krbtgt/ > > > > [email protected] from FILE:/tmp/krb5cc_impala_internal with > > > > result: 0/Success > > > > [31476] 1513120922.532441: Starting with TGT for client realm: > impala/ > > > > [email protected] -> krbtgt/[email protected] [31476] > > > > 1513120922.532467: Retrieving impala/[email protected] -> > > > > krbtgt/localhost@localhost from FILE:/tmp/krb5cc_impala_internal > with > > > > result: -1765328243/Matching credential not found [31476] > > > > 1513120922.532475: Requesting TGT krbtgt/[email protected] using > > TGT > > > > krbtgt/[email protected] [31476] 1513120922.532491: Generated > > > > subkey for TGS request: aes256-cts/005D [31476] 1513120922.532524: > > etypes > > > > requested in TGS request: aes256-cts, aes128-cts, des3-cbc-sha1, > > > rc4-hmac, > > > > camellia128-cts, camellia256-cts [31476] 1513120922.532574: Encoding > > > > request body and padata into FAST request [31476] 1513120922.532616: > > > > Sending request (951 bytes) to KRBTEST.COM [31476] > 1513120922.532630: > > > > Resolving hostname 127.0.0.1 [31476] 1513120922.532648: Sending > initial > > > UDP > > > > request to dgram > > > > 127.0.0.1:51781 > > > > [31586] 1513120922.532790: AP-REQ ticket: impala/ > [email protected] > > > -> > > > > krbtgt/[email protected], session key aes256-cts/580F [31586] > > > > 1513120922.532814: Negotiated enctype based on authenticator: > > > > aes256-cts > > > > [31586] 1513120922.532820: Authenticator contains subkey: > > aes256-cts/005D > > > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): > > > TGS_REQ > > > > (6 etypes {18 17 16 23 25 26}) 127.0.0.1: UNKNOWN_SERVER: authtime > 0, > > > > impala/[email protected] for krbtgt/[email protected], > Server > > > not > > > > found in Kerberos database [31476] 1513120922.533028: Received answer > > > (491 > > > > bytes) from dgram > > > > 127.0.0.1:51781 > > > > [31476] 1513120922.533044: Response was not from master KDC [31476] > > > > 1513120922.533053: Decoding FAST response [31476] 1513120922.533081: > > TGS > > > > request result: -1765328377/Server krbtgt/ [email protected] not > > > > found in Kerberos database > > > > /home/philip/src/impala/be/src/rpc/thrift-server-test.cc:153: > Failure > > > > Value of: status_.ok() > > > > Actual: false > > > > Expected: true > > > > Error: Couldn't open transport for localhost:62119 (SASL(-1): generic > > > > failure: GSSAPI Error: Unspecified GSS failure. Minor code may > provide > > > > more information (Server krbtgt/[email protected] not found in > > > > Kerberos > > > > database)) > > > > > > > > [ FAILED ] KerberosOnAndOff/ThriftKerberizedParamsTest. > > > > SslConnectivity/2, > > > > where GetParam() = 2 (100 ms) > > > > > > > > > > > > > >
