Abacn commented on PR #30218: URL: https://github.com/apache/beam/pull/30218#issuecomment-2037690938
We have to recreate the consumer/session to properly handle checkpoints, otherwise there were data losses. This is due to the limitation that message acknowledge in Jms is per session. All messages that delivered by the time of acknowledging a message within a session will be marked as acknowledged. Agree with the analysis and I think the cause is same here: https://github.com/apache/beam/pull/30218/files#diff-a63812b51f93708cc60430f314b496ae1110425c6a8ae4c85e59573cfb8f0938R204-R207 The yet-finalized checkpoint hold an active session which may contain messages in its internal buffer. Those message won't get released until a checkpoint being finalized. I tested my PR with IBM MQ and it had no issue. So this is still implementation related, though. Is there a way to release the messages in internal buffer but do not acknowledge the receive message within the same session? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
