linzch3 opened a new pull request, #9280:
URL: https://github.com/apache/storm/pull/9280

   Addresses the docs part of #9078. The design questions there (signalling the 
drop back to the spout, dead-lettering) are left alone.
   
   ## What is the purpose of the change
   
   `docs/storm-kafka-client.md` says nothing about what happens when a tuple 
from the Kafka spout is never acked, for example because it always fails to 
deserialize on the receiving worker. In at-least-once mode that quietly stops 
commits for the partition and lag keeps climbing, and with the default retry 
settings it never recovers.
   
   I added a short "Records that are never acked" section that explains:
   - why commits stop and when polling of the partition stops 
(`max.uncommitted.offsets`)
   - that the default retry service retries forever (`DEFAULT_MAX_RETRIES` is 
`Integer.MAX_VALUE`)
   - the current mitigation: a `KafkaSpoutRetryExponentialBackoff` with a 
finite `maxRetries`, plus `KafkaTupleListener.onMaxRetryReached` to log or keep 
track of the record before the spout acks it
   - the trade-offs: each retry waits for the message timeout, and a finite 
`maxRetries` also gives up on tuples that fail for temporary reasons
   
   Docs only; no code changes.
   
   ## How was the change tested
   
   I checked the description against `KafkaSpout.fail`, `KafkaSpoutConfig` 
(defaults and builder methods) and `KafkaTupleListener` on master, and that the 
example uses the actual constructor and method names. I also read the rendered 
Markdown.
   
   Happy to adjust the wording or move it somewhere else in the doc if you'd 
prefer.
   
   An AI assistant helped draft this change; I reviewed it and take 
responsibility for it.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to