tdas commented on a change in pull request #23301: [SPARK-26350][SS]Allow to override group id of the Kafka consumer URL: https://github.com/apache/spark/pull/23301#discussion_r247069632
########## File path: docs/structured-streaming-kafka-integration.md ########## @@ -379,7 +379,25 @@ The following configurations are optional: <td>string</td> <td>spark-kafka-source</td> <td>streaming and batch</td> - <td>Prefix of consumer group identifiers (`group.id`) that are generated by structured streaming queries</td> + <td>Prefix of consumer group identifiers (`group.id`) that are generated by structured streaming + queries. If "kafka.group.id" is set, this option will be ignored. </td> +</tr> +<tr> + <td>kafka.group.id</td> + <td>string</td> + <td>none</td> + <td>streaming and batch</td> + <td>The Kafka group id to use in Kafka consumer while reading from Kafka. Use this with caution. + By default, each query generates a unique group id for reading data. This ensures that each Kafka + source has its own consumer group that does not face interference from any other consumer, and + therefore can read all of the partitions of its subscribed topics. In some scenarios (for example, + Kafka group-based authorization), you may want to use a specific authorized group id to read data. + You can optionally set the group ID. However, do this with extreme caution as it can cause Review comment: nit: ID -> id ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
