HeartSaVioR commented on issue #24613: [SPARK-27549][SS] Add support for committing kafka offsets per batch for supporting external tooling URL: https://github.com/apache/spark/pull/24613#issuecomment-533997056 FYI: I just took a step on this and crafted a project for enabling this feature via library: https://github.com/HeartSaVioR/spark-sql-kafka-offset-committer Only tested in unsecured Kafka cluster, but as it runs in driver and all parameters for Kafka are provided it would run in secured Kafka cluster as well.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
