GitHub user rhauch opened a pull request: https://github.com/apache/kafka/pull/3672
KAFKA-5731 Corrected how the sink task worker updates the last committed offsets (0.11.0) Prior to this change, it was possible for the synchronous consumer commit request to be handled before previously-submitted asynchronous commit requests. If that happened, the out-of-order handlers improperly set the last committed offsets, which then became inconsistent with the offsets the connector task is working with. This change ensures that the last committed offsets are updated only for the most recent commit request, even if the consumer reorders the calls to the callbacks. **This is for the `0.11.0` branch; see #3662 for the equivalent and already-approved PR for `trunk`.** You can merge this pull request into a Git repository by running: $ git pull https://github.com/rhauch/kafka kafka-5731-0.11.0 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/kafka/pull/3672.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3672 ---- commit 526dbcb776effcc1661e51293a6d03256b19d0a6 Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-12T00:42:06Z KAFKA-5731 Corrected how the sink task worker updates the last committed offsets Prior to this change, it was possible for the synchronous consumer commit request to be handled before previously-submitted asynchronous commit requests. If that happened, the out-of-order handlers improperly set the last committed offsets, which then became inconsistent with the offsets the connector task is working with. This change ensures that the last committed offsets are updated only for the most recent commit request, even if the consumer reorders the calls to the callbacks. # Conflicts: # connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java commit 80965cb5f8771e63b0dad095287cb9a29dea47f6 Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-14T19:11:08Z KAFKA-5731 Corrected mock consumer behavior during rebalance Corrects the test case added in the previous commit to properly revoke the existing partition assignments before adding new partition assigments. commit 37687c544513566c7d728273137a12751702ad41 Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-14T19:11:45Z KAFKA-5731 Added expected call that was missing in another test commit bfac0688ab64935bc4ac9c11e0a6251ca03e1043 Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-14T22:24:35Z KAFKA-5731 Improved log messages related to offset commits # Conflicts: # connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java commit bbc316dfa4353a7e914d11ceaca2b60c1bdaf291 Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-15T14:47:05Z KAFKA-5731 More cleanup of log messages related to offset commits commit 00b17ebbb5effb7f8aa171ea69b0227c7b009e97 Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-15T16:21:52Z KAFKA-5731 More improvements to the log messages in WorkerSinkTask # Conflicts: # connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java commit 3a5c31f1b912e06aded4b74524c73dfe1033e76c Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-15T16:31:28Z KAFKA-5731 Removed unnecessary log message commit 8b91f93e8c4c7b6b8e1aa6721f86ff01f8ecf40e Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-15T17:54:16Z KAFKA-5731 Additional tweaks to debug and trace log messages to ensure clarity and usefulness # Conflicts: # connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java commit bea03f69055524e9005302200f3a560a9cad2c3f Author: Randall Hauch <rha...@gmail.com> Date: 2017-08-15T19:30:09Z KAFKA-5731 Use the correct value in trace messages ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---