Github user arunmahadevan commented on a diff in the pull request:
https://github.com/apache/spark/pull/22598#discussion_r228320944
--- Diff:
core/src/main/scala/org/apache/spark/internal/config/package.scala ---
@@ -647,4 +647,42 @@ package object config {
.stringConf
.toSequence
.createWithDefault(Nil)
+
+ private[spark] val KAFKA_DELEGATION_TOKEN_ENABLED =
+ ConfigBuilder("spark.kafka.delegation.token.enabled")
+ .doc("Set to 'true' for obtaining delegation token from kafka.")
+ .booleanConf
+ .createWithDefault(false)
+
+ private[spark] val KAFKA_BOOTSTRAP_SERVERS =
+ ConfigBuilder("spark.kafka.bootstrap.servers")
--- End diff --
Given the defaults the tokens fetch would be attempted if only
`spark.kafka.bootstrap.servers` is defined right ? And the spark-sql-kafka
libraries needs to be in the class path as well ?
Better mention these in the docs. And make the
`spark.security.credentials.kafka.enabled` default to false if it makes sense.
Also consider prefixing all the configs with
`spark.security.credentials.kafka` instead of `spark.kafka` (like
`spark.security.credentials.kafka.bootstrap.servers`) to make it explicit that
these are security related settings required for fetching kafka delegation
tokens.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
