ppawel commented on PR #30218: URL: https://github.com/apache/beam/pull/30218#issuecomment-2037595552
@Abacn I am using GCP Dataflow but the test case is with DirectRunner. I am now digging deeper into this and this is easily reproducible when you force the data source splits to 1 in `UnboundedJmsSource#split`. This means that you will have only one connection/session/consumer and there will basically be a deadlock between the first (original) consumer and the one created when doing a checkpoint. I temporarily removed the code that creates the new consumer and also removed closing the consumer and session from `JmsCheckpointMark` and now it works fine as expected. So I think this is most likely related to closing/opening the session/consumer. > In the case of 50 outstanding messages, are these messages already acknowledged in the broker side (which means data loss)? Or there are still unacknowledged but do not send to other session by the broker ? They are marked as "not acknowledged" by the broker but as "received" by one consumer. The other consumer does not get those messages. I guess the broker would send those messages to the second consumer when the first one is closed but closing only happens when finalizing the checkpoint and that doesn't happen because only the new consumer (which does not have any messages) is being called in `advance` so in the end nothing is being done, it just spins around calling `receive`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
