addisonj commented on issue #14082:
URL: https://github.com/apache/pulsar/issues/14082#issuecomment-1029181907
@anant-ahuja This appears to be a problem with your consumers not
acknowledging all messages.
To explain, if you look at the `unackedMessages` and `availablePermits`
together indicate this problem. To explain, see this selection from the topic
stats:
```
{
...
"availablePermits" : 0,
"unackedMessages" : 3,
...
"connectedSince" : "2022-02-03T00:11:00.17Z",
"address" : "/10.32.8.9:59458"
}
```
unackedMessages indicates that Pulsar has dispatched messages to this
consumer, but the consumer has not acknowledged them. unackedMessages
themselves are not an issue, but, the avaiablePermits is zero which indicates
to Pulsar that it cannot send any more messages.
The way Pulsar works is that clients dictate when messages can be sent. The
client does this by sending the broker a "flow" command to tell the broker it
can send up to some number of messages (refereed to as permits). When a client
library processes messages and is ready for more, it sends another flow
command, which gives the broker more permits to send more messages.
You never have to worry about sending flow messges yourself, instead, the
pulsar client library does it automatically based on when you acknowledge
messages. However, if your application fails to acknowledge too many messages,
this can cause issues with the client not sending any more flow commands.
As far as how to fix this, here are a few options:
1. Find the code paths that are resulting in unprocessed messages (usually
related to error handling or deadlocks in process) and fix those by adding
error handling or a timeout of process, then either nack the messages or just
ack the message (depending on your processing semantics)
2. Use Pulsar consumer "ackTimeout" feature which will send a negative
acknowledgement (causing redelivery) if processing doesn't happen within some
period of time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
