Can you run the test again, and klist the contents of the credential cache and post the error logs again? Looks like "impala/localhost" might not be stored as expected in the cache on your machine.
On Wed, Dec 13, 2017 at 2:47 PM, Philip Zeyliger <[email protected]> wrote: > The KDC in this case is the "minikdc" from > https://github.com/apache/impala/blob/master/be/src/ > kudu/security/test/mini_kdc.cc. > I see evidence of it, and have been able to look at its configuration by, > um, adding --gtest_break_on_failure. (The feature actually doesn't work, > presumably because of an interaction with breakpad, but a temporary > directory is left on my filesystem, so that's nice.) > > -- Philip > > On Tue, Dec 12, 2017 at 4:08 PM, Evo Eftimov <[email protected]> > wrote: > > > Is your cluster Kerberized at all, especially the Impala daemon - it > > doesn’t seem to be enrolled in the KDC at all > > > > You / your personal account/principal is definitely enrolled though > > > > And there is definetly a KDC in your environment > > > > -----Original Message----- > > From: Philip Zeyliger [mailto:[email protected]] > > Sent: Tuesday, December 12, 2017 11:26 PM > > To: [email protected] > > Subject: thrift-server-test > > > > Hi folks, > > > > I've been running into issues with thrift-server-test and Kerberos. Below > > is an excerpt of "KRB5_TRACE=/dev/stderr be/build/debug/rpc/thrift- > server-test"; > > both SslConnectivity/1 and > > SslConnectivity/2 fail the same way. > > > > I'm running Ubuntu16.04. I've seen this both on my host, as well as > inside > > of an Ubuntu 16.04 Docker container. > > > > Does this ring any bells? > > > > Thanks! > > > > -- Philip > > > > > > [ RUN ] KerberosOnAndOff/ThriftKerberizedParamsTest. > SslConnectivity/2 > > Loading random data > > Initializing database '7abf-cef9-113e-eae3/krb5kdc/principal' for realm > ' > > KRBTEST.COM', > > master key name 'K/[email protected]' > > [31585] 1513120922.459517: Retrieving K/[email protected] from > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) with > > result: 0/Success > > [31586] 1513120922.472314: Retrieving K/[email protected] from > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) with > > result: 0/Success > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): > setting > > up network... > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): > > listening on fd 11: udp 0.0.0.0.51781 (pktinfo) > > krb5kdc: setsockopt(12,IPV6_V6ONLY,1) worked Dec 12 15:22:02 > > philip-dev.gce.cloudera.com krb5kdc[31586](info): listening on fd 12: > udp > > ::.51781 (pktinfo) Dec 12 15:22:02 philip-dev.gce.cloudera.com > > krb5kdc[31586](info): set up 2 sockets Dec 12 15:22:02 > > philip-dev.gce.cloudera.com krb5kdc[31586](info): > > commencing operation > > krb5kdc: starting... > > Authenticating as principal philip/[email protected] with password. > > [31589] 1513120922.498913: Retrieving K/[email protected] from > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) with > > result: 0/Success > > WARNING: no policy specified for impala/[email protected]; > defaulting > > to no policy Principal "impala/[email protected]" created. > > Authenticating as principal philip/[email protected] with password. > > [31590] 1513120922.508777: Retrieving K/[email protected] from > > FILE:7abf-cef9-113e-eae3/krb5kdc/.k5.KRBTEST.COM (vno 0, enctype 0) with > > result: 0/Success > > Entry for principal impala/localhost with kvno 2, encryption type > > aes256-cts-hmac-sha1-96 added to keytab > > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab. > > Entry for principal impala/localhost with kvno 2, encryption type > > aes128-cts-hmac-sha1-96 added to keytab > > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab. > > Entry for principal impala/localhost with kvno 2, encryption type > > des3-cbc-sha1 added to keytab > > WRFILE:7abf-cef9-113e-eae3/krb5kdc/impala_localhost.keytab. > > Entry for principal impala/localhost with kvno 2, encryption type > > arcfour-hmac added to keytab WRFILE:7abf-cef9-113e-eae3/ > > krb5kdc/impala_localhost.keytab. > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): AS_REQ > > (6 etypes {18 17 16 23 25 26}) 127.0.0.1: ISSUE: authtime 1513120922, > > etypes > > {rep=18 tkt=18 ses=18}, impala/[email protected] for krbtgt/ > > [email protected] [31476] 1513120922.532304: ccselect can't find > > appropriate cache for server principal impala@localhost [31476] > > 1513120922.532347: Getting credentials impala/[email protected] > > -> impala@localhost using ccache FILE:/tmp/krb5cc_impala_internal > > [31476] 1513120922.532382: Retrieving impala/[email protected] -> > > impala@localhost from FILE:/tmp/krb5cc_impala_internal with result: > > -1765328243/Matching credential not found [31476] 1513120922.532407: > > Retrieving impala/[email protected] -> krbtgt/localhost@localhost > > from FILE:/tmp/krb5cc_impala_internal with > > result: -1765328243/Matching credential not found [31476] > > 1513120922.532433: Retrieving impala/[email protected] -> krbtgt/ > > [email protected] from FILE:/tmp/krb5cc_impala_internal with > > result: 0/Success > > [31476] 1513120922.532441: Starting with TGT for client realm: impala/ > > [email protected] -> krbtgt/[email protected] [31476] > > 1513120922.532467: Retrieving impala/[email protected] -> > > krbtgt/localhost@localhost from FILE:/tmp/krb5cc_impala_internal with > > result: -1765328243/Matching credential not found [31476] > > 1513120922.532475: Requesting TGT krbtgt/[email protected] using TGT > > krbtgt/[email protected] [31476] 1513120922.532491: Generated > > subkey for TGS request: aes256-cts/005D [31476] 1513120922.532524: etypes > > requested in TGS request: aes256-cts, aes128-cts, des3-cbc-sha1, > rc4-hmac, > > camellia128-cts, camellia256-cts [31476] 1513120922.532574: Encoding > > request body and padata into FAST request [31476] 1513120922.532616: > > Sending request (951 bytes) to KRBTEST.COM [31476] 1513120922.532630: > > Resolving hostname 127.0.0.1 [31476] 1513120922.532648: Sending initial > UDP > > request to dgram > > 127.0.0.1:51781 > > [31586] 1513120922.532790: AP-REQ ticket: impala/[email protected] > -> > > krbtgt/[email protected], session key aes256-cts/580F [31586] > > 1513120922.532814: Negotiated enctype based on authenticator: > > aes256-cts > > [31586] 1513120922.532820: Authenticator contains subkey: aes256-cts/005D > > Dec 12 15:22:02 philip-dev.gce.cloudera.com krb5kdc[31586](info): > TGS_REQ > > (6 etypes {18 17 16 23 25 26}) 127.0.0.1: UNKNOWN_SERVER: authtime 0, > > impala/[email protected] for krbtgt/[email protected], Server > not > > found in Kerberos database [31476] 1513120922.533028: Received answer > (491 > > bytes) from dgram > > 127.0.0.1:51781 > > [31476] 1513120922.533044: Response was not from master KDC [31476] > > 1513120922.533053: Decoding FAST response [31476] 1513120922.533081: TGS > > request result: -1765328377/Server krbtgt/ [email protected] not > > found in Kerberos database > > /home/philip/src/impala/be/src/rpc/thrift-server-test.cc:153: Failure > > Value of: status_.ok() > > Actual: false > > Expected: true > > Error: Couldn't open transport for localhost:62119 (SASL(-1): generic > > failure: GSSAPI Error: Unspecified GSS failure. Minor code may provide > > more information (Server krbtgt/[email protected] not found in > > Kerberos > > database)) > > > > [ FAILED ] KerberosOnAndOff/ThriftKerberizedParamsTest. > > SslConnectivity/2, > > where GetParam() = 2 (100 ms) > > > > >
