Github user mxm commented on the pull request:
https://github.com/apache/flink/pull/1243#issuecomment-160141917
This currently only works when checkpointing is turned on. If checkpointing
is turned off, not only the number of queued messages will grow unbounded, but
also the messages will never be acknowledged. In terms of RabbitMQ that means
that messages will be kept in the broker forever and will be redelivered once
another consumer connects to the queue again.
In terms of exactly once, I'm not 100% sure how we can support it. I think
we have to default to per-message acknowledgments (instead of acknowledging all
ids <= the last processed id). Only that way we can make sure that upon failure
we acknowledge the right messages. The reason for that is that distribution of
the messages may not be the same after a redeployment of a failed job, i.e.
consumers may receive messages that have not seen the message beforehand, thus
a message may be processed more than once.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
