[jira] [Commented] (KAFKA-3850) WorkerSinkTask should retry commits if woken up during rebalance or shutdown
[ https://issues.apache.org/jira/browse/KAFKA-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15338822#comment-15338822 ] ASF GitHub Bot commented on KAFKA-3850: --- Github user asfgit closed the pull request at: https://github.com/apache/kafka/pull/1511 > WorkerSinkTask should retry commits if woken up during rebalance or shutdown > > > Key: KAFKA-3850 > URL: https://issues.apache.org/jira/browse/KAFKA-3850 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.0.0 >Reporter: Jason Gustafson >Assignee: Jason Gustafson > Fix For: 0.10.0.1 > > > We use consumer.wakeup() to interrupt long polls when we need to pause/resume > partitions and when we shutdown sink tasks. The resulting {{WakeupException}} > could be raised from the synchronous commit which we use in between > rebalances and on shutdown. Since we don't currently catch this exception, we > can fail to commit offsets, which typically results in duplicates. To fix > this problem, we should catch the exception, retry the commit, and then > rethrow it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3850) WorkerSinkTask should retry commits if woken up during rebalance or shutdown
[ https://issues.apache.org/jira/browse/KAFKA-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332964#comment-15332964 ] ASF GitHub Bot commented on KAFKA-3850: --- GitHub user hachikuji reopened a pull request: https://github.com/apache/kafka/pull/1511 KAFKA-3850: WorkerSinkTask commit prior to rebalance should be retried on wakeup You can merge this pull request into a Git repository by running: $ git pull https://github.com/hachikuji/kafka retry-commit-on-wakeup-in-sinks Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/1511.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1511 commit 0e2e4568325a52611f67e172688e2ade86e8bdf5 Author: Jason GustafsonDate: 2016-06-16T00:40:32Z KAFKA-??: WorkerSinkTask commit prior to rebalance should be retried on wakeup > WorkerSinkTask should retry commits if woken up during rebalance or shutdown > > > Key: KAFKA-3850 > URL: https://issues.apache.org/jira/browse/KAFKA-3850 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.0.0 >Reporter: Jason Gustafson >Assignee: Jason Gustafson > Fix For: 0.10.0.1 > > > We use consumer.wakeup() to interrupt long polls when we need to pause/resume > partitions and when we shutdown sink tasks. The resulting {{WakeupException}} > could be raised from the synchronous commit which we use in between > rebalances and on shutdown. Since we don't currently catch this exception, we > can fail to commit offsets, which typically results in duplicates. To fix > this problem, we should catch the exception, retry the commit, and then > rethrow it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (KAFKA-3850) WorkerSinkTask should retry commits if woken up during rebalance or shutdown
[ https://issues.apache.org/jira/browse/KAFKA-3850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15332963#comment-15332963 ] ASF GitHub Bot commented on KAFKA-3850: --- Github user hachikuji closed the pull request at: https://github.com/apache/kafka/pull/1511 > WorkerSinkTask should retry commits if woken up during rebalance or shutdown > > > Key: KAFKA-3850 > URL: https://issues.apache.org/jira/browse/KAFKA-3850 > Project: Kafka > Issue Type: Bug > Components: KafkaConnect >Affects Versions: 0.10.0.0 >Reporter: Jason Gustafson >Assignee: Jason Gustafson > Fix For: 0.10.0.1 > > > We use consumer.wakeup() to interrupt long polls when we need to pause/resume > partitions and when we shutdown sink tasks. The resulting {{WakeupException}} > could be raised from the synchronous commit which we use in between > rebalances and on shutdown. Since we don't currently catch this exception, we > can fail to commit offsets, which typically results in duplicates. To fix > this problem, we should catch the exception, retry the commit, and then > rethrow it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)