darinspivey commented on issue #24879: URL: https://github.com/apache/pulsar/issues/24879#issuecomment-3453159146
>A little bit confused, so the brokerClient_connectionsPerBroker works for you or not? Sorry for the confusion. YES, this change **appears to have solved the http timeouts**, which may be the sole reason for topics being orphaned. I've chosen to definitely use connection pooling, and set that value to `15`, which should cover topics with more partitions, and *may* even be enough to prevent the race if I have partitions > 15. We can see, but as of now, I don't see orphaned topics. The race condition to update metadata (as originally posted in this report) still appears to happen, even for successful deletions. So, that may be an issue to deal with or not--you'd have to decide that. But I guess the http timeouts were the main cause of topics being orphaned? >On partitions that didn't get any data, those topics are deleted right after the subscriptions are removed. What I meant by this was the recent fix in #24733 . Since I'm now running pulsar version `4.0.7`, I see the effect of that happening, which I think is that for low-throughput topics, not all partitions will have data. The ones that don't appear to have the partition's topic be deleted when the subscriptions delete (and not wait for the rention policy time). That's great, actually--less to manage. Ignore me here if you're confused by this. I was just being verbose :) >Call chain: 1.gc process -> 2.delete partitioned-topic admin api(invoked on broker0) -> 3.delete topic admin api(invoked on broker0). >The concurrent race condition maybe another use case, so let' fix the self call problem first. BTW, I'll try to submit a PR to avoid the third admin api call if this analysis works. Since I think `brokerClient_connectionsPerBroker` had a positive effect, I'm glad you've identified another area to look at with your comment above. You think you'll have a PR to fix that flow? Thanks again for your help, we're looking good here I think! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
