GitHub user suvrajit94 edited a discussion: Challenges Achieving Proper Queue Functionality with Pulsar and Flaky Message Dispatch in Key Shared Subscription
Pulsar Version: 2.9.2 I want to achieve Pulsar as a proper queue. Process individual keys in order Process. If there is a validation failure during processing, negatively ack the message to re-process it again later. I am using the Key_Shared subscription to achieve the same. I have around 10 consumers running in the pipeline. But somehow message dispatch from brokers to consumers seems flaky. Even if there is a backlog, most of the time, except for 1-2 consumers rest remain idle. Restart works. See more even distribution of polled messages across consumers for some time. But it comes back in the same state within a few hours. Before the pipeline goes to a stuck state, the last logs are always some consumer doing negative ack. Not sure how I am using negative ack is wrong or not. I am not using a retry-letter topic. Relying upon the negatively acknowledged messages to get redelivered by the broker to existing consumers. I am negatively ack up to 10 times, then handling it in the code to ack when this number is exhausted. I have disabled `deliveryAfter` on the Producer (because of the [bug](https://github.com/apache/pulsar/pull/20233) prior to 2.10.5 releases). GitHub link: https://github.com/apache/pulsar/discussions/21186 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
