kriti-sc opened a new pull request, #3954: URL: https://github.com/apache/iggy/pull/3954
Closes #2928, #2927 Related: #3203 ## Rationale Sinks must have an option to decide exactly when it is safe to advance the offset, as only the sink knows when a write is truly durable from the target's perspective. ## What changed? This PR adds an `offset_commit` setting to sink configs: - `after_polling` (default) - existing behaviour of offset being committed when messages are polled. Nothing changes for existing configs. - `after_consuming` - the offset is stored only once the sink reports that it wrote the batch. The offset commit decision hinges upon the i32 returned by the sink. This also required binding the i32 returned by the sink, which was previous discarded. Now, the sink exits if i32 != 0, regardless of offset commit configuration, which is the correct behaviour as a sink returns error for failures lying outside its purview. This PR also introduces integration tests that inject failures into a running sink and verify the desired commit behaviour. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
