HeartSaVioR edited a comment on issue #24613: [SPARK-27549][SS] Add support for committing kafka offsets per batch for supporting external tooling URL: https://github.com/apache/spark/pull/24613#issuecomment-494322232 Ah, my bad. I'm kinda lost about the details and missed your key point of approach. As we don't use the generated gid to consumers running in executors, sounds good to me. EDIT: I just saw the updated comment, and feel more safer to use `query ID + sequence` than `Allow user pass a gId per source`. We still need to guide to end users that they should be fully aware of impact of conflicted gid. If we can provide the way to avoid it at all, I feel it's better to do so.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
