zengguan opened a new issue, #17021: URL: https://github.com/apache/pulsar/issues/17021
### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Motivation In [PIP-121](https://github.com/apache/pulsar/issues/13315)，we support cluster level controlled cluster failover on client side。But I have some issue with this - when we upgrade pulsar in future，it is more difficult to upgrade client than upgrade proxy。 - client side only support Java，We need to implement other programming language such as go，python。 - cluster level auto failover is great，but i want to implement controlled in topic level。it is more flexible and smaller influences。In this way we can migrate request step by step with topic level，and Observe whether there is an impact on the task associated with the topic。 ### Solution - [Idea] We propose an idea，which proxy support multi pulsar cluster，with specified configuration multi cluster。 - [Implement] In this mode，proxy can discover multi pulsar cluster with multi zookeeper connections。when we creating topic，we can specified Mapping between topics and clusters，such as TopicA -> Cluster A, Topic B -> Cluster B, store this metadata in zookeeper。If clients send lookup request to proxy with topicA，and the proxy will respond the specified cluster url (such as Cluster A)。 - [migration] for example，in cluster A we have 10 topics reading and writing., And TopicA -> Cluster A in proxy. I send the swith request(TopicA -> Cluster B) to proxy, the proxy change the mapping metadata in zookeeper, and after then request topic A become mapping Cluster B. - [dependent] we should turn on cluster level geo-replication to ensure the topic data sync between all primary and secondary clusters. Otherwise, it may distribute the topic data into different clusters. And the consumers won’t get the whole data of the topic. ### Alternatives _No response_ ### Anything else? - current architecture situation :  - after support multi pulsar cluster architecture :  ### Are you willing to submit a PR? - [X] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
