massakam opened a new pull request #7299: URL: https://github.com/apache/pulsar/pull/7299
This is a further modification of the previously closed PR #6592. Fixes #6437 ### Motivation In a replicated subscription with no consumers connected, the number of marker messages in the backlog will continue to increase. If at least one active consumer is connected, the marker messages will be acknowledged and deleted by the dispatcher. https://github.com/apache/pulsar/blob/2d2c63e4d7fc4ce90068edf15f39de82ef738e33/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractBaseDispatcher.java#L88-L100 On the other hand, if no consumers are connected, or if they are connected but cannot receive any messages, the dispatcher does not exist or has stopped reading entries. As a result, the marker messages accumulate in the backlog without being acknoledged by anyone. ### Modifications There are four types of marker messages: - ReplicatedSubscriptionsSnapshotRequest - ReplicatedSubscriptionsSnapshotResponse - ReplicatedSubscriptionsSnapshot - ReplicatedSubscriptionsUpdate Of these, three messages, except `ReplicatedSubscriptionsSnapshot`, are not used in the local cluster. They are published to local topics for sending to remote clusters. So, modified the `ReplicatedSubscriptionsController` class to acknowledge these marker messages on all subscriptions immediately after publishing the messages to a local topic. On the other hand, `ReplicatedSubscriptionsSnapshot` is only used by dispatchers in the local cluster and does not need to be sent to remote clusters. So, `ReplicatedSubscriptionsController` only acknowledges those messages on behalf of dispatcher if dispatcher has not been initialized or if there are no available consumers. In addition, marker messages sent from remote clusters are now acknowledged by the replicator on all subscriptions. With these changes, marker messages no longer continue to accumulate in the replicated subscription backlog. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
