LucaCanali commented on a change in pull request #23525: [SPARK-26595][core] Allow credential renewal based on kerberos ticket cache. URL: https://github.com/apache/spark/pull/23525#discussion_r248966327
########## File path: core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala ########## @@ -97,28 +106,37 @@ private[spark] class HadoopDelegationTokenManager( ThreadUtils.newDaemonSingleThreadScheduledExecutor("Credential Renewal Thread") val ugi = UserGroupInformation.getCurrentUser() - if (ugi.isFromKeytab()) { + val tgtRenewalTask = if (ugi.isFromKeytab()) { // In Hadoop 2.x, renewal of the keytab-based login seems to be automatic, but in Hadoop 3.x, // it is configurable (see hadoop.kerberos.keytab.login.autorenewal.enabled, added in // HADOOP-9567). This task will make sure that the user stays logged in regardless of that // configuration's value. Note that checkTGTAndReloginFromKeytab() is a no-op if the TGT does // not need to be renewed yet. - val tgtRenewalTask = new Runnable() { + new Runnable() { override def run(): Unit = { ugi.checkTGTAndReloginFromKeytab() Review comment: I should clarify that the warning messages I reported are for the case where I use the TGT and --conf spark.kerberos.renewal.credentials=ccache rather than using keytab, apologies for the possible confusion this may have generated. Looking now at the code for UserGroupInformatio.reloginFromTicketCache I can see that it calls hasSufficientTimeElapsed which is responsible for generating the warning message in question, when users are trying to renew at a rate higher than a certain frequency. As you pointed out, with hadoop.kerberos.min.seconds.before.relogin set to default value of 60 we are OK as it matches the default for spark.kerberos.relogin.period, (but this requires HADOOP-7930, e.g. Hadoop version >= 2.8)). On a related topic, I can see that checkTGTAndReloginFromKeytab has a "silent" way of checking if the rate of request for renewal is higher than the threshold, so no warning are generated in this case. Does this make sense and is it reproducible in you Hadoop 2.7 environment too? ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org