avirajkhare00 opened a new issue, #2940: URL: https://github.com/apache/iggy/issues/2940
While reviewing the MongoDB sink follow-up, I ended up at a broader connector-runtime question that likely applies beyond MongoDB. Some sinks can partially commit a batch in the external system and still return an error. That creates a hard tradeoff between: - not reporting full success when only part of the batch was written - not getting stuck replaying already-written messages forever - keeping progress and restart behavior understandable A few design questions seem worth discussing more generally: - should sinks be able to report partial success to the runtime - should runtime progress be able to advance partially - should deterministic external write identity be a standard requirement for sinks - when is duplicate detection enough, and when do we need a stronger contract I am not proposing a concrete runtime change in this issue yet. I have a small PR separately to align the MongoDB sink README and test naming with the current merged behavior. This issue is about the larger contract and design question behind partial writes and replay-safe progress. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
