linzch3 opened a new pull request, #9280: URL: https://github.com/apache/storm/pull/9280
Addresses the docs part of #9078. The design questions there (signalling the drop back to the spout, dead-lettering) are left alone. ## What is the purpose of the change `docs/storm-kafka-client.md` says nothing about what happens when a tuple from the Kafka spout is never acked, for example because it always fails to deserialize on the receiving worker. In at-least-once mode that quietly stops commits for the partition and lag keeps climbing, and with the default retry settings it never recovers. I added a short "Records that are never acked" section that explains: - why commits stop and when polling of the partition stops (`max.uncommitted.offsets`) - that the default retry service retries forever (`DEFAULT_MAX_RETRIES` is `Integer.MAX_VALUE`) - the current mitigation: a `KafkaSpoutRetryExponentialBackoff` with a finite `maxRetries`, plus `KafkaTupleListener.onMaxRetryReached` to log or keep track of the record before the spout acks it - the trade-offs: each retry waits for the message timeout, and a finite `maxRetries` also gives up on tuples that fail for temporary reasons Docs only; no code changes. ## How was the change tested I checked the description against `KafkaSpout.fail`, `KafkaSpoutConfig` (defaults and builder methods) and `KafkaTupleListener` on master, and that the example uses the actual constructor and method names. I also read the rendered Markdown. Happy to adjust the wording or move it somewhere else in the doc if you'd prefer. An AI assistant helped draft this change; I reviewed it and take responsibility for it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
