gaborgsomogyi commented on a change in pull request #24305: [SPARK-27294][SS]
Add multi-cluster Kafka delegation token
URL: https://github.com/apache/spark/pull/24305#discussion_r281519837
##########
File path:
external/kafka-0-10-token-provider/src/main/scala/org/apache/spark/kafka010/KafkaTokenUtil.scala
##########
@@ -223,14 +236,27 @@ private[spark] object KafkaTokenUtil extends Logging {
}
}
- def isTokenAvailable(): Boolean = {
- UserGroupInformation.getCurrentUser().getCredentials.getToken(
- KafkaTokenUtil.TOKEN_SERVICE) != null
+ def findMatchingToken(
+ sparkConf: SparkConf,
+ bootStrapServers: String): Option[KafkaTokenClusterConf] = {
+ val tokens =
UserGroupInformation.getCurrentUser().getCredentials.getAllTokens.asScala
+ val clusterConfigs = tokens
+ .filter(_.getService().toString().startsWith(TOKEN_SERVICE_PREFIX))
+ .map { token =>
+ KafkaTokenSparkConf.getClusterConfig(sparkConf,
getClusterIdentifier(token.getService()))
+ }
+ .filter { clusterConfig =>
+ val pattern = Pattern.compile(clusterConfig.targetServersRegex)
+
Utils.stringToSeq(bootStrapServers).exists(pattern.matcher(_).matches())
Review comment:
The most simple single cluster scenario works without any regex config by
the user.
In the multi-cluster case even if providing a proper regex is an extra step
I would keep the implementation as it is. As a user I would personally burn
more time on understanding such fallback scenarios than writing a regex. If
hostname pattern changes somehow then it has to be re-understand again (in
general not big fan of fallbacks until it's absolutely not required).
If you think we can adapt the code because both solutions are good.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
