anishshri-db commented on code in PR #41791:
URL: https://github.com/apache/spark/pull/41791#discussion_r1251218995
##########
docs/structured-streaming-kafka-integration.md:
##########
@@ -1171,3 +1171,14 @@ This can be done several ways. One possibility is to
provide additional JVM para
--driver-java-options
"-Djava.security.auth.login.config=/path/to/custom_jaas.conf" \
--conf
spark.executor.extraJavaOptions=-Djava.security.auth.login.config=/path/to/custom_jaas.conf
\
...
+
+### IAM authentication
+
+Managed Streaming for Kafka (MSK) is a fully managed service offered by Amazon
Web Services (AWS) that simplifies the deployment, management, and scaling of
Apache Kafka clusters.
+One way to connect to MSK is by using IAM authentication. To facilitate this,
Spark already ships with the [aws msk
library](https://github.com/aws/aws-msk-iam-auth) by default so no additional
dependencies are required.
+To use IAM authentication, you can set the kafka config as in the [library
documentation](https://github.com/aws/aws-msk-iam-auth). For example:
+
+ --conf spark.kafka.clusters.${cluster}.kafka.security.protocol=SASL_SSL
Review Comment:
I believe the method described below applies to `spark-submit` ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
