lhotari commented on issue #23451: URL: https://github.com/apache/pulsar/issues/23451#issuecomment-2414072629
> the first place is because it got the broker (in my case pulsar-broker-35) from a LOOKUP it did earlier. If so, that's confusing because it means it thinks that broker owns the topic (in my case named xxxx-partition-1), but wouldn't that broker have to actively claim the topic somehow, in which case it should've had the correct metadata to see that there was now more than one partition? See, xxxx is originally a partitioned topic with only one partition (so with only xxxx-partition-0 as a valid topic name), but after the topic metadata update, the C++ client looked up the new partitions and was told to connect to xxxx-partition-1 on pulsar-broker-35. But pulsar-broker-35 thinks there's only one partition in the topic. How could it when it seems to have claimed xxxx-partition-1? The default lookup logic is in code in NamespaceService.findBrokerServiceUrl and searchForCandidateBroker methods: https://github.com/apache/pulsar/blob/9012422bcbaac7b38820ce545cd5a3b4f8b586d0/pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java#L473-L539 https://github.com/apache/pulsar/blob/9012422bcbaac7b38820ce545cd5a3b4f8b586d0/pulsar-broker/src/main/java/org/apache/pulsar/broker/namespace/NamespaceService.java#L573-L712 Then the client makes a lookup request (`CommandLookupTopic` in proto) with the [`authoritative` flag](https://github.com/apache/pulsar/blob/9a97c843a46e23a0811e2172991cd00a3af642c0/pulsar-common/src/main/proto/PulsarApi.proto#L436), the target broker will attempt to claim the ownership of the topic without doing lookups. That's why the solution tolerates eventual consistency. Each lookup could get redirected multiple times until it hits the target broker. The target broker will also process the lookup request and when it's `authoritative`, it will attempt to claim the ownership. The partitioned topic metadata is a completely separate concern in Pulsar. The client needs that information for subscribing to multi-partitioned topics. The broker itself doesn't need that information to serve topics since an individual partition of a multi-partitioned topic behaves completely independently. I hope this explanation makes sense. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
