Hi @Enrico @Bo > If there is a problem we should spend time on investigating the problem and not in adding this kind of tools.
You are right. When a user encounters a problem, it often takes a while to solve the root cause. It is important to provide a tool to recover services quickly, and cmd 'unload topic' is often used to solve problems temporarily. But now a topic can be used by multiple teams, such as Business-team A to produce messages, Business-team B and C to consume messages (each using a different subscription name), and Data-team D to read messages using reader API (using a random subscription name). When there is a consumption problem in subscription B, if we do unload the topic, this will affect teams A, B, C, D, and when we provide a new API to reset subscribers, the impact can be controlled only to affect team B. The new API can also be used in this scenario: Trigger consumer rebalance. So I feel like we can add this API with relatively small changes. Thanks Yubiao Feng On Wed, Jan 11, 2023 at 5:00 PM Yubiao Feng <yubiao.f...@streamnative.io> wrote: > Hi community > > I am starting a DISCUSS for PIP-240: A new API to unload subscriptions. > > PIP issue: https://github.com/apache/pulsar/issues/19187 > > ### Motivation > > We sometimes try to unload the topic to resolve some consumption-stop > issues. But the unloading topic will also impact the producer side. > > ### Goal > > Providing a new API to unload the subscription dimension triggers > reconnection of all consumers on that subscription and reconnection is > guaranteed by the client. The API will be used in these ways: > - unload special subscription of one topic(or partitioned topic) > - unload all subscriptions of one topic(or partitioned topic) > - unload subscriptions of one topic(or partitioned topic) by regular > expression > - If a reader's subscription name is not set, a random subscription name > prefixed with 'multiTopicsReader-' or 'reader-' will be used, and users can > uninstall these subscriptions using regular expressions. > > In addition to triggering consumer disconnection, Unloading Subscribers > will restart the Dispatcher, which resets the redeliver message queue and > delayed message queue in the Broker's memory, which can help resolve issues > caused by an abnormal dispatcher state. However, the execution flow of > Unloading Subscribers does not include a restart of the Managed Cursor > related to this dispatcher; if there is a problem with the cursor, we can > only rely on the unload topic to solve it. > > Note: From the client's perspective, this connection may be shared by > consumers, producers, and transactions, so Unloading Subscribers maybe > impact the producer and transaction. > > #### These scenarios are not supported > - Functions `message-dedup`, `geo-replication,` and `shadow-topic` also > read messages from the topic, but Unloading subscribers will not support > triggering restarts of these three functions( because the cursor is used > directly to read the data in these scenarios, not the consumer or reader ). > - The Compression task(subscription name is `__compaction`) also use a > reader to read data, but Unloading Subscribers does not support it because > this task creates a new reader each time it starts. > - Do not support all topics related to Transaction features. > - `__transaction_buffer_snapshot` works with the task TB recover, and > this task will create a new reader each time they start. > - `__transaction_pending_ack` works with the task Transaction Pending > Ack Store replay, and this task will use managed cursor directly to read > data. > - `__transaction_log_xxx` works with the task Transaction Log, which > will use managed cursor directly to read data. > - `transaction_coordinator_assign` No data will be written on this topic. > > #### Special system topic supports > The system topic `__change_events` is used to support topic-level > policies, there may also be some message delivery issues in this scenario, > so Unloading Subscribers will support this topic. > > ### API Changes > > #### For persistent topic > ``` > pulsar-admin persistent unload {topic_name} -s {sub_name} > ``` > > #### For non-persistent topic > ``` > pulsar-admin non-persistent unload {topic_name} -s {sub_name} > ``` > > #### Explain the param `-s` > - set param `-s` to special sub name to unload special subscription > - set param `-s` to `**` to unload all subscriptions under this topic > - set param `-s` to `regexp` to unload a batch subscriptions under this > topic > > > Thanks > Yubiao Feng >
