David Arthur created KAFKA-7738:
-----------------------------------
Summary: Track partition leader epochs in client metadata
Key: KAFKA-7738
URL: https://issues.apache.org/jira/browse/KAFKA-7738
Project: Kafka
Issue Type: Improvement
Reporter: David Arthur
The Metadata API now exposes the current leader epoch of each partition. We can
leverage this information to be smarter in how we fetch metadata. For example,
when we receive a NOT_LEADER_FOR_PARTITION, we know to look for an epoch bump
before resuming whatever operation we were trying to do. Additionally, this
gives us a way to detect stale metadata.
This requires a little more sophistication in how we track metadata in the
client. For example, we may be not be able to assume metadata updates are
monotonic. In other words, they may mix stale metadata for one partition and
fresh metadata for another. I am not sure we have any strong guarantees on the
order in which metadata updates are seen on each broker.
It may be helpful in this context to control the topics that we fetch metadata
for at a finer granularity. Potentially we could even extend the Metadata API
to specify a subset of the partitions that the client is interested in.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
