[
https://issues.apache.org/jira/browse/KAFKA-7831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758773#comment-16758773
]
ASF GitHub Bot commented on KAFKA-7831:
---------------------------------------
hachikuji commented on pull request #6221: KAFKA-7831; Do not modify
subscription state from background thread
URL: https://github.com/apache/kafka/pull/6221
Metadata may be updated from the background thread, so we need to protect
access to SubscriptionState. This patch restructures the metadata handling so
that we only check pattern subscriptions in the foreground. Additionally, it
improves the following:
1. SubscriptionState is now the source of truth for the topics that will be
fetched. We had a lot of messy logic previously to try and keep the the topic
set in Metadata consistent with the subscription, so this simplifies the logic.
2. The metadata needs for the producer and consumer are quite different, so
it made sense to separate the custom logic into separate extensions of
Metadata. For example, only the producer requires topic expiration.
3. We've always had an edge case in which a metadata change with an inflight
request may cause us to effectively miss an expected update. This patch
implements a separate version inside Metadata which is bumped when the needed
topics changes.
4. This patch removes the MetadataListener, which was the cause of
https://issues.apache.org/jira/browse/KAFKA-7764.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Consumer SubscriptionState missing synchronization
> --------------------------------------------------
>
> Key: KAFKA-7831
> URL: https://issues.apache.org/jira/browse/KAFKA-7831
> Project: Kafka
> Issue Type: Bug
> Reporter: Jason Gustafson
> Assignee: Jason Gustafson
> Priority: Major
>
> ConsumerCoordinator installs a Metadata.Listener in order to update pattern
> subscriptions after metadata changes. The listener is invoked from
> NetworkClient.poll, which could happen in the heartbeat thread. Currently,
> however, there is no synchronization in SubscriptionState to make this safe.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
