Hi Minjian, > Is it still need a pip if we fix the origin isue in a lightweight way which just calculated with a simple subtraction of index?
Yes, I think so too. It would be good to have a clear conclusion explaining why we chose the message index-based approach for Failover and Exclusive subscriptions. Alternatively, you can proceed with this proposal by introducing a flag that allows users to enable the max unacked messages limit, with the default set to false to prevent any breaking changes. This way, if more users later agree that the limitation should be enabled, we can consider switching the default to true in the future. It provides a smooth path to roll out the change incrementally. Regards, Penghui On Mon, Jun 16, 2025 at 12:19 AM Zixuan Liu <zix...@apache.org> wrote: > This PIP looks good to me overall. > > However, I'd like to point out a concern regarding exclusive and failover > subscriptions. Currently, the unacknowledged message count is always shown > as 0 for these types, which can be misleading. > > Failover and Exclusive subscriptions don't require a max unacknowledged > > message limit > > because they operate more simply > > While it's true that exclusive and failover subscriptions are simpler in > design, I believe we should still enforce a maximum unacknowledged message > limit for them. Enabling this limit would ensure fairness across all > consumer types and provide better consistency in system behavior. > > Unifying this behavior across all subscription types would be the right > approach. > > Thanks, > Zixuan > > 蔡敏健 <xiaocair...@gmail.com> 于2025年6月16日周一 12:26写道： > > > Thanks for detail explanation. It's really helpful. It will save lots of > > time if I have thought from the root like this. Since there isn't > > a real-world scenarios to add flow permit to exclusive or failover > > subscription, we reach an agreement that it's useless. Is it still need a > > pip if we fix the origin isue in a lightweight way which just calculated > > with a simple subtraction of index? IMO, we can reject and close this PIP > > and return back to fix the origin issue! > > > > PengHui Li <peng...@apache.org> 于2025年6月14日周六 14:06写道： > > > > > Hi Minjian, > > > > > > Thanks for sharing the proposal. Sharing my thoughts here: > > > > > > The maximum unacknowledged message limitation is crucial for consumer > > load > > > balancing > > > and managing broker resources in Pulsar's Shared and Key_Shared > > > subscriptions: > > > > > > Consumer Load Balancing: The limit prevents any single consumer from > > being > > > overwhelmed > > > with more messages than it can process. When a consumer hits this cap, > > > Pulsar stops > > > sending messages and, in a shared subscription, will dispatch new > > messages > > > to other > > > available consumers. This effectively balances the workload across the > > > consumer group, > > > preventing bottlenecks and ensuring smooth processing. > > > > > > Manages Broker Resources: Every message sent but not yet acknowledged > > > consumes broker > > > memory and CPU to track its state. By capping the number of > > unacknowledged > > > messages > > > per consumer and subscription (ack state), the broker avoids being > > swamped > > > by slow or > > > unresponsive consumers. This conserves vital resources, preventing > > > performance degradation > > > and ensuring the stability of the entire Pulsar cluster. > > > > > > Failover and Exclusive subscriptions don't require a max unacknowledged > > > message limit > > > because they operate more simply: > > > > > > One Active Consumer: Only a single consumer receives messages at any > > time. > > > This eliminates the need to load balance or protect individual > consumers > > > from being overwhelmed relative to others. > > > > > > Cumulative Acknowledgement: The consumer can acknowledge the last > message > > > it processed, > > > and Pulsar automatically marks all previous messages as acknowledged. > > This > > > is highly > > > efficient and drastically reduces the number of unacknowledged messages > > the > > > broker needs > > > to track for that single consumer. > > > > > > I agree that there could be use cases for limiting the max unacked > > messages > > > for > > > Failover or Exclusive subscriptions. If you have any real-world > scenarios > > > where > > > this limitation is required, it would be great to include them in the > > > proposal. > > > This would help users understand when and why they should use this > > feature. > > > > > > Since this proposal will change the behavior of the existing Failover > or > > > Exclusive > > > subscriptions, please consider making it disabled by default to avoid > > > breaking > > > existing users and having a flag for users to enable it. > > > > > > Regards, > > > Penghui > > > > > > On Tue, Jun 10, 2025 at 4:21 AM 蔡敏健 <xiaocair...@gmail.com> wrote: > > > > > > > Hi, Pulsar Community, > > > > > > > > I opened a new pip to enable consumer throttling and accurate > > > > unacknowledged message tracking for exclusive and failover > > subscriptions. > > > > I'm looking > > > > forward to your suggestions! > > > > > > > > link: https://github.com/apache/pulsar/pull/24400 > > > > > > > > Thanks, > > > > Minjian cai > > > > > > > > > >
