Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

Hi As we discussed above, I have posted pr #14730 to solve the deleting namespace issue. Thanks Michael for your idea. And thanks to all the people joining this. Regards Jiwei Guo (Tboy) On Mon, Mar 7, 2022 at 1:45 PM guo jiwei wrote: > Hi

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

Hi Penghui, After test, we can use #12136 to stop the replicator. Regards Jiwei Guo (Tboy) On Sat, Mar 5, 2022 at 5:31 PM PengHui Li wrote: > > Great point. I was focused on deleting namespaces and missed the case > where the user wants to delete a topic from a replicated namespace. I >

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> Great point. I was focused on deleting namespaces and missed the case where the user wants to delete a topic from a replicated namespace. I agree that we should make it possible to delete these topics without removing the namespace replication. Oh, sorry. I thought again, after

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> The geo-replication's configuration can be centralized by using > one configuration store. But that doesn't change anything, we should > provide the same behavior for both centralized and decentralized > configuration store. Perhaps I misused the word decentralized. I meant that the challenge

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> I agree with you that it'd be nice to provide the same deletion behavior. However, because geo-replication's configuration is decentralized, I think namespace or topic deletion is more complicated than unreplicated deletion. Note that users cannot currently delete namespaces that are configured

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> Either replicated topic or non-replicated topic, we should provide the > same topic deletion behavior, the topic can be deleted if no active > user's producers/consumers, should not be affected by the Pulsar > internal producers/consumers. I agree with you that it'd be nice to provide the same

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

I have added the geo-replicator topic case and updated the PIP. Regards Jiwei Guo (Tboy) On Sun, Feb 27, 2022 at 1:00 PM PengHui Li wrote: > > To me, the main question is whether we create a custom error or expand > the `CommandCloseProducer` command. I lean towards adding an error > because

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> To me, the main question is whether we create a custom error or expand the `CommandCloseProducer` command. I lean towards adding an error because it will automatically be backward compatible based on the way the client determines which errors are retriable. Although, I don't have a strong

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> We need to determine the overall plan, not the implementation > at this moment. Great point, I agree. > Looks like we need an option in the > `CommandCloseProducer` to avoid the replication producer > reconnecting again. This is an important point. One of the primary requirements for

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

The replicated topic deletion also has a similar problem. If the topic auto-creation is enabled, we are not able to delete a replicated topic, because the producer used for replication will try to reconnect. Looks like we need an option in the `CommandCloseProducer` to avoid the replication

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> Penghui, are you suggesting that we implement the namespace/tenant terminated logic after completing this PIP? I'm ok with both being implemented together or separated. We need to determine the overall plan, not the implementation at this moment. > For the sake of discussion, if we implement

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

Regarding the namespace "terminated" concept, I just noticed that we already have a "deleted" field in a namespace's policies [0]. There is even a comment that says: > // set the policies to deleted so that somebody else cannot acquire this > namespace I am not familiar with this feature, but

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

Hi Dave, > automatically delete tenants and namespaces for not containing topics I don't think that is what we are discussing. I agree that the initial email says just that, though, which is why I asked above: >> When there are no user-created topics under a namespace, >> Namespace should be

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

Hi - I hope I’m understanding what’s being discussed. If we are going to automatically delete tenants and namespaces for not containing topics then we need to make both of these automatic actions configurable with a default to NOT do so. Otherwise we break existing use cases. Automatic

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> The old producer/consumer should be closed after applying the changes from > this proposal. Penghui, are you suggesting that we implement the namespace/tenant terminated logic after completing this PIP? For the sake of discussion, if we implement the namespace terminated logic first, we could

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> If we want to solve this problem, we need to add some sync resources like lock/state, I think it’s a harm for us, we don’t need to do that. I think we can make the namespace/tenants to the inactive state first so that we can avoid any new producer/consumer connect to the topic under the

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

> This is supposed to mean that the namespace should be able to be > deleted, correct? Yes, the main background is the user doesn’t have an active topic. so, they want to delete the namespace. > However, I think > we might still have a race condition that could make tenant or > namespace

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

This proposal identifies an important issue that we should definitely solve. I have some questions. > When there are no user-created topics under a namespace, > Namespace should be deleted. This is supposed to mean that the namespace should be able to be deleted, correct? > For this reason, we

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

+1 Thanks, Zike On Sat, Jan 29, 2022 at 12:30 PM guo jiwei wrote: > > Hi > The PIP link : https://github.com/apache/pulsar/issues/13989 > > Regards > Jiwei Guo (Tboy) > > > On Sat, Jan 29, 2022 at 11:46 AM mattison chao > wrote: > > > Hello everyone, > > > > I want to start a discussion

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

Hi The PIP link : https://github.com/apache/pulsar/issues/13989 Regards Jiwei Guo (Tboy) On Sat, Jan 29, 2022 at 11:46 AM mattison chao wrote: > Hello everyone, > > I want to start a discussion about PIP-139 : Support Broker send command > to real close producer/consumer. > > This is the PIP

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

+1 发自我的iPhone > 在 2022年1月29日，12:24，PengHui Li 写道： > > +1 > > Penghui > >> On Sat, Jan 29, 2022 at 11:46 AM mattison chao >> wrote: >> >> Hello everyone, >> >> I want to start a discussion about PIP-139 : Support Broker send command >> to real close producer/consumer. >> >> This is the

Re: [DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

+1 Penghui On Sat, Jan 29, 2022 at 11:46 AM mattison chao wrote: > Hello everyone, > > I want to start a discussion about PIP-139 : Support Broker send command > to real close producer/consumer. > > This is the PIP document > > https://github.com/apache/pulsar/issues/13989 < >

[DISCUSS] PIP-139 : Support Broker send command to real close producer/consumer.

Hello everyone, I want to start a discussion about PIP-139 : Support Broker send command to real close producer/consumer. This is the PIP document https://github.com/apache/pulsar/issues/13989 Please check it out and feel free to share your