[
https://issues.apache.org/jira/browse/KAFKA-15839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lianet Magrans updated KAFKA-15839:
-----------------------------------
Description:
With the new consumer group protocol, assignments received by the consumer
contain topic IDs instead of topic names. Topic Ids are used in the
reconciliation path, integrated using TopicIdPartition. When reconciling, topic
names are resolved via a metadata update, but they are also kept in a local
#MembershipManager cache. This local cache serves the purpose of keeping
assigned topicId-names (that might have been deleted from metadata, ex. topic
deleted).
That's just an initial step towards spreading topic IDs internally in the
consumer code. Next step to address with this task would be to include topic
IDs in the subscription state, so that assigned topicId-names can be accessed
from other components without the need of resolving names multiple times.
Note that this task aims only at spreading topic IDs internally in the
consumer, no changes to expose them at the API level.
was:
TopicIdPartition is currently used in the reconciliation path. Could be used
more, just leaving topicPartitions when necessary for the callbacks and
interaction with the subscription state that does not fully support topic ids
yet
Ensure that we properly handle topic re-creation (same name, diff topic IDs) in
the reconciliation process (assignment cache, same assignment comparison, etc.)
> Review topic ID integration in consumer subscription
> ----------------------------------------------------
>
> Key: KAFKA-15839
> URL: https://issues.apache.org/jira/browse/KAFKA-15839
> Project: Kafka
> Issue Type: Sub-task
> Components: clients, consumer
> Reporter: Lianet Magrans
> Assignee: Lianet Magrans
> Priority: Major
> Labels: kip-848, kip-848-client-support, kip-848-e2e,
> kip-848-preview
> Fix For: 3.8.0
>
>
> With the new consumer group protocol, assignments received by the consumer
> contain topic IDs instead of topic names. Topic Ids are used in the
> reconciliation path, integrated using TopicIdPartition. When reconciling,
> topic names are resolved via a metadata update, but they are also kept in a
> local #MembershipManager cache. This local cache serves the purpose of
> keeping assigned topicId-names (that might have been deleted from metadata,
> ex. topic deleted).
> That's just an initial step towards spreading topic IDs internally in the
> consumer code. Next step to address with this task would be to include topic
> IDs in the subscription state, so that assigned topicId-names can be accessed
> from other components without the need of resolving names multiple times.
> Note that this task aims only at spreading topic IDs internally in the
> consumer, no changes to expose them at the API level.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
