varshini03 opened a new pull request, #23236:
URL: https://github.com/apache/kafka/pull/23236
This is a follow-up to the stale PR #22071, rewritten to address the open
reviewer comments before submitting.
### Problem
`VerifiableProducer.start()` returns as soon as background threads are
launched, before producer nodes have connected to Kafka. Tests begin
immediately, causing sporadic failures. The Java `VerifiableProducer` already
emits `{"name": "startup_complete"}` before the send loop - the Python service
simply never handled it.
### Fix
Three additions to `verifiable_producer.py`:
1. **`startup_complete` event handling in `_worker()`** - when the event
arrives, calls `_handle_startup_complete(node)` which adds the node to
`_started_nodes_set` under the existing lock
2. **`started_nodes()`** — returns a defensive copy of the set of nodes that
have started; uses a plain `set` consistent with `clean_shutdown_nodes` already
in the class (no `event_handlers` structure needed since the producer has no
per-node state machine unlike the consumer - addressing @lianetm's question on
#22071)
3. **`start()` override** - calls `wait_until()` with
`_propagate_exceptions()` in the lambda so a worker crash surfaces immediately
rather than waiting the full timeout (directly addressing @chickenchickenlove's
feedback on #22071)
### Why a new PR instead of updating #22071
Starting fresh allowed addressing all open reviewer comments upfront.
### Testing
Ran the verifiable producer sanity tests locally using the Docker-based
ducker-ak setup.
<img width="682" height="244" alt="image"
src="https://github.com/user-attachments/assets/7e8c44eb-ae8d-4e49-842d-445bac6600d3";
/>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
