Pankraz76 commented on PR #23947: URL: https://github.com/apache/kafka/pull/23947#issuecomment-6385631913
More detail on the retry loop mentioned above, in case it helps the review: While a marker waits in the unknown-broker queue (leader endpoint is `Node.noNode()`), the `TxnMarkerSenderThread` re-checks it in a tight loop until the partition has a leader again: - `generateRequests()` drains the unknown-broker queue and calls `addTxnMarkersToBrokerQueue()` for each parked marker. - The leader is still unavailable, so the marker is parked again. - `addTxnMarkersToBrokerQueue()` always ends with `wakeup()`, so the next `pollOnce(Long.MAX_VALUE)` returns immediately and the thread loops straight back. With one parked marker I measured about 30,000 `getPartitionLeaderEndpoint` calls per second (0 when nothing is parked). It does not affect correctness: the marker is sent as soon as the partition has a leader, which the integration test in this PR verifies. This already happens on trunk when a leader has no endpoint for the inter-broker listener, and it was the behavior for offline partitions before KAFKA-17573. So I kept it out of this fix. A possible follow-up would be to call `wakeup()` only when markers reach a real broker queue, and retry parked markers after a backoff (e.g. `retry.backoff.ms`) or on a metadata update. Happy to file a JIRA for that if reviewers agree. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
