lucasbru opened a new pull request, #23675: URL: https://github.com/apache/kafka/pull/23675
Stacked on #23620; only the last commit is new here. This adds three warm-up task lifecycle tests for situations that only arise with real clients and brokers, on top of the harness from #23620. The first fails the group coordinator over while a warm-up task is in flight. The intermediate assignment and the reported offsets are held in memory only, so the new coordinator derives the intermediate assignment again from the persisted assignments, and the warm-up task has to survive that without being closed and restored again, and be promoted with nothing left to restore. The second has the active owner leave while the warm-up task is still behind, so the task is handed to the warm-up task's member right away, and the active task has to continue restoring from exactly where the warm-up task stopped rather than from the start. The third runs a member with two stream threads that each warm up and take over a task, and then removes one thread, after which the remaining thread restores the moved task from scratch, which is the current beha viour for in-memory stores until KAFKA-21090. The multi-threaded test tracks when each task's store is opened and closed and on which thread, and checks that a process never holds a task on two threads at once. The published thread metadata cannot be used for this, since a thread only refreshes it when it starts running again, so a shutting-down thread keeps listing tasks it has already closed, and a thread that finished restoring a task since it last started running does not list it yet. The cluster runs three brokers with replicated internal topics so that the coordinator can fail over. The failover test picks an application ID whose coordinator is not on the broker that also runs the controller, so that restarting it does not take the metadata quorum down. EmbeddedKafkaCluster gains a way to restart a broker and to tell which nodes run the controller. The tests passed repeated local runs, and the owner-leaves test fails when the reconciler routes a warm-up promotion through revocation instead of promoting the warm-up task in place. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
