suzhiking opened a new pull request, #23873: URL: https://github.com/apache/kafka/pull/23873
In a streams group, one process must never hold the same task twice, for example as an active task on one member and as a standby task on another. The reconciler checks this only when it grants a task to a member. A task that the member already holds and still has in its target assignment is kept without the check. A static member that rejoins under a new member ID gets a copy of its previous member, including the previous member's assigned tasks. If the member rejoins from a different process, the copied tasks move to the new process with it, and none of them is checked. When another member on the new process already holds one of those tasks, that process now holds the task twice. A Kafka Streams client gets here when an instance restarts under a new process ID, for example because its topology has no persistent stores: if one stream thread rejoins and is given the standby of a task before the thread that ran the task as active rejoins, both threads are told to run the task. With this change, when a static member rejoins from a different process, the new member keeps only the copied tasks that no member holds on the new process, in any role. The dropped tasks stay in the target assignment, so the reconciler grants them again later with its usual checks. Until then, a dropped task has no owner on the new process instead of two. When the process is unchanged, or no copied task is held on the new process, the member keeps all its tasks as before. `testStaticMemberRejoinFromNewProcessDropsTaskHeldThereAsStandby` drives the restart above with the sticky assignor and one standby replica. The second thread rejoins first and gets the standby. The first thread then rejoins and gets no active task, and the new process holds the task only as a standby. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
