lhotari opened a new pull request, #17401:
URL: https://github.com/apache/pulsar/pull/17401
### Motivation
- Fixes issue where leader broker information isn't available and this
warning gets logged:
`WARN org.apache.pulsar.broker.namespace.NamespaceService - The
information about the current leader broker wasn't available. Handling load
manager decisions in a decentralized way.`
- Possibly fixing:
- Issue #13584 is a result of brokers making decentralized decisions
without the leader broker
- Issues where the symptoms are topic redirection issues with admin api,
example #13946
The root cause of the problem is that currently leader election cache
entries expire in 10 minutes. Entries are refreshed when there's a cache hit
between 5 to 10 minutes after the entry was added to the cache or refreshed the
last time. This expiration behavior cannot be configured currently in
MetadataCache.
### Modifications
Since leader election cache expiration leads to problems, it is better to
disable expiration completely for leader election entries. The cache entry
refreshing is not changed so a missed Zookeeper change notification won't cause
the cache to serve stale values.
- change `MetadataStore`
- add support passing cache configuration for a `MetadataCache` so that
expiration can be disabled for leader election cache
- revisit expiration logic in LeaderElectionImpl. Use `cache.refresh`
instead of removing entries from the cache.
### Additional context
- The value for the leader broker is currently read intentionally in a
non-blocking way. There's an alternative fix #17254 which retrieves the value
also in this case. That approach is abandoned since the correct solution is to
not expire the Leader election cache at all. Thank you, @michaeljmarshall, for
recommending this approach where Leader election cache isn't expired at all.
- [x] `doc-not-needed`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
