erictcgs commented on issue #22911: [SPARK-25815][k8s] Support kerberos in client mode, keytab-based token renewal. URL: https://github.com/apache/spark/pull/22911#issuecomment-447508472 Continuing to dig on this - I do have that "Updating delegation tokens" line, but later find that the UserGroupInformation is attempting to access HDFS using auth:SIMPLE. It looks like the executor is running everything in the context of [runAsSparkUser](https://github.com/vanzin/spark/blob/ccb39560298a5e54f144b8ba2a43d950289ccf34/core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala#L188), which ends up creating a new UserGroupInformation user with [createRemoteUser](https://github.com/vanzin/spark/blob/ccb39560298a5e54f144b8ba2a43d950289ccf34/core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala#L68), which defaults to [auth:simple](https://github.com/apache/hadoop/blob/cd915e1e8d9d0131462a0b7301586c175728a282/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L1276) I see how [addCurrentUserCredentials](https://github.com/vanzin/spark/blob/ccb39560298a5e54f144b8ba2a43d950289ccf34/core/src/main/scala/org/apache/spark/deploy/SparkHadoopUtil.scala#L125) in Spark is calling [addCredentials](https://github.com/apache/hadoop/blob/cd915e1e8d9d0131462a0b7301586c175728a282/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java#L1575) in hadoop, but I don't see anywhere that's changing the authMethod of the sparkuser from SIMPLE to TOKEN - shouldn't this be somewhere? My working hypothesis is that my executor is trying to run with sparkuser credentials, which have the delegation token present, but authmethod set to simple. Currently modifying SparkHadoopUtil.addCurrentUserCredentials to call setAuthenticationMethod(TOKEN) when auth method was SIMPLE (will post with results), but would love to hear if this is known how SIMPLE is supposed to change to TOKEN
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
