vinodkc commented on a change in pull request #32894:
URL: https://github.com/apache/spark/pull/32894#discussion_r650834371
##########
File path:
core/src/main/scala/org/apache/spark/deploy/security/HBaseDelegationTokenProvider.scala
##########
@@ -97,7 +97,8 @@ private[security] class HBaseDelegationTokenProvider
creds.addToken(token.getService, token)
} catch {
case NonFatal(e) =>
- logWarning(s"Failed to get token from service $serviceName", e)
+ logWarning(s"Failed to get token from service $serviceName due to $e.
" +
+ s"If HBase is not used, set spark.security.credentials.hbase.enabled
to false")
Review comment:
In a multi-tenant cluster with HBase service, which is down for some
planned maintenance, even if some teams are not using HBase in their business
application, still they are seeing an exception stack trace of more than 100+
line. So it is good to suppress full stack trace and multiple HBase connection
failure errors.
> Code-wise, we'll have to fix other instances too together:
1)
>
core/src/main/scala/org/apache/spark/deploy/security/HBaseDelegationTokenProvider.scala
Done
2)
>
core/src/main/scala/org/apache/spark/deploy/security/HadoopFSDelegationTokenProvider.scala:
logWarning(s"Failed to get token from service $serviceName", e)
In HadoopFSDelegationTokenProvider.scala, if DT is not generated,
application submit will fail fast, hence adding the suggestion to disable HDFS
DT generation won't be helpful
3)
>
external/kafka-0-10-token-provider/src/main/scala/org/apache/spark/kafka010/KafkaDelegationTokenProvider.scala:
logWarning(s"Failed to get token from service: $serviceName " +
Done
4)
>
sql/hive/src/main/scala/org/apache/spark/sql/hive/security/HiveDelegationTokenProvider.scala:
logWarning(s"Failed to get token from service $serviceName", e)
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
