LucaCanali commented on a change in pull request #23525: [SPARK-26595][core]
Allow credential renewal based on kerberos ticket cache.
URL: https://github.com/apache/spark/pull/23525#discussion_r248966327
##########
File path:
core/src/main/scala/org/apache/spark/deploy/security/HadoopDelegationTokenManager.scala
##########
@@ -97,28 +106,37 @@ private[spark] class HadoopDelegationTokenManager(
ThreadUtils.newDaemonSingleThreadScheduledExecutor("Credential Renewal
Thread")
val ugi = UserGroupInformation.getCurrentUser()
- if (ugi.isFromKeytab()) {
+ val tgtRenewalTask = if (ugi.isFromKeytab()) {
// In Hadoop 2.x, renewal of the keytab-based login seems to be
automatic, but in Hadoop 3.x,
// it is configurable (see
hadoop.kerberos.keytab.login.autorenewal.enabled, added in
// HADOOP-9567). This task will make sure that the user stays logged in
regardless of that
// configuration's value. Note that checkTGTAndReloginFromKeytab() is a
no-op if the TGT does
// not need to be renewed yet.
- val tgtRenewalTask = new Runnable() {
+ new Runnable() {
override def run(): Unit = {
ugi.checkTGTAndReloginFromKeytab()
Review comment:
I should clarify that the warning messages I reported are for the case where
I use the TGT and --conf spark.kerberos.renewal.credentials=ccache rather than
using keytab, apologies for the possible confusion this may have generated.
Looking now at the code for UserGroupInformatio.reloginFromTicketCache I can
see that it calls hasSufficientTimeElapsed which is responsible for generating
the warning message in question, when users are trying to renew at a rate
higher than a certain frequency. As you pointed out, with
hadoop.kerberos.min.seconds.before.relogin set to default value of 60 we are OK
as it matches the default for spark.kerberos.relogin.period, (but this requires
HADOOP-7930, e.g. Hadoop version >= 2.8)).
On a related topic, I can see that checkTGTAndReloginFromKeytab has a
"silent" way of checking if the rate of request for renewal is higher than the
threshold, so no warning are generated in this case.
Does this make sense and is it reproducible in you Hadoop 2.7 environment
too?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
