gabrywu commented on issue #28985: URL: https://github.com/apache/beam/issues/28985#issuecomment-1764018722
As of now, BeamKafkaTable doesn't support commitOffsetsInFinalize (default value is false), in each checkpoint & savepoint kafka reader will not commit offset. If commitOffsetInFinalize = false & enable.auto.commit = true, related consumer commits offset automatically. When job restarts from a savepoint, it will choose the automatic-commit offset as the first offset, and ignore offsets from savepoints If commitOffsetInFinalize = true & enable.auto.commit = false. kafka reader commits offset in each checkpoint & savepoint, and can restart from a specific savepoint. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
