gaborgsomogyi commented on a change in pull request #29729:
URL: https://github.com/apache/spark/pull/29729#discussion_r499549277
##########
File path: docs/ss-migration-guide.md
##########
@@ -26,6 +26,19 @@ Note that this migration guide describes the items specific
to Structured Stream
Many items of SQL migration can be applied when migrating Structured Streaming
to higher versions.
Please refer [Migration Guide: SQL, Datasets and
DataFrame](sql-migration-guide.html).
+## Upgrading from Structured Streaming 3.0 to 3.1
+
+- In Spark 3.0 and below, secure Kafka processing needed the following ACLs
from driver perspective:
+ * Topic resource describe operation
+ * Topic resource read operation
+ * Group resource read operation
+
+ Since Spark 3.1, offsets are obtained with `AdminClient` instead of
`KafkaConsumer` and now the following ACLs needed from driver perspective:
+ * Topic resource describe operation
+
+ Since `AdminClient` in driver is not connecting to consumer group,
`group.id` based authorization will not work anymore (executors never done
group based authorization).
Review comment:
Some of the users authorizing based on `group.id` which will not work
with this PR (topic based authorization is a good alternative). Nothing more,
nothing less. If you can help how to make it more descriptive it's welcome.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
