erictcgs commented on issue #22911: [SPARK-25815][k8s] Support kerberos in client mode, keytab-based token renewal. URL: https://github.com/apache/spark/pull/22911#issuecomment-447513508 Update - my setAuthenticationMethod change didn't work - after calling that I see that the executor reports it's trying to use TOKEN method, but it's still getting rejected with an error that it can't login via TOKEN, KERBEROS: ``` 18/12/14 23:26:01 DEBUG UserGroupInformation: PrivilegedActionException as:user (auth:TOKEN) cause:org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] 18/12/14 23:26:01 DEBUG UserGroupInformation: PrivilegedAction as:user (auth:TOKEN) from:org.apache.hadoop.ipc.Client$Connection.handleSaslConnectionFailure(Client.java:651) 18/12/14 23:26:01 WARN Client: Exception encountered while connecting to the server : org.apache.hadoop.security.AccessControlException: Client cannot authenticate via:[TOKEN, KERBEROS] ``` As a test I tried mounting in my keytab and hadoop_conf_dir into the executors and setting KRB5CCNAME env var (via pod template) - this results in the main user being kerberized, then I commented out the "runassparkuser" section of the coarsescheduler, and now the executors are able to successfully login to HDFS and grab the parquet data. It looks like there's some incompatibility in my environment with how the delegation tokens are being used - I think this shouldn't block this merge if it's working for others - is there any other venue people would suggest I continue trying to debug?
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
