reiabreu commented on issue #9078:
URL: https://github.com/apache/storm/issues/9078#issuecomment-6712719128

   *Generated with the help of an LLM.*
   
   **Taking this out of the 3.2.0 milestone; it stays open.**
   
   #9093 relates to this issue but does not fix it. It only logs the retry 
give-up at ERROR instead of debug, and adds javadoc to 
`KafkaSpoutRetryExponentialBackoff` and `KafkaTupleListener.onMaxRetryReached`. 
The retry, ack and commit behavior is unchanged.
   
   What is still true on master:
   - `KafkaSpoutConfig.DEFAULT_MAX_RETRIES` is `Integer.MAX_VALUE`. With 
defaults the spout never gives up, so the new ERROR log never fires.
   - Offsets behind an undecodable record cannot commit. Polling for that 
partition stops at `maxUncommittedOffsets` (default 10,000,000), so the visible 
symptom is climbing lag.
   - The receiving worker cannot tell the spout that it dropped a message as 
undecodable. There is no dead-letter handling, and `docs/storm-kafka-client.md` 
has no guidance on this case.
   
   The design questions in the description are still open: whether the receiver 
should signal the drop to the spout, and whether giving up should dead-letter 
the raw record instead of acking it silently.
   
   Partial mitigation for now: set a finite `maxRetries` and use 
`KafkaTupleListener.onMaxRetryReached` to keep the record. Its `msgId` gives 
the topic, partition and offset.
   
   No milestone for now.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to