[jira] [Commented] (KAFKA-8813) Race condition when creating topics and changing their configuration
[
https://issues.apache.org/jira/browse/KAFKA-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952193#comment-16952193
]
ASF GitHub Bot commented on KAFKA-8813:
---
hachikuji commented on pull request #7305: KAFKA-8813: Refresh log config if
it's updated before initialization
URL: https://github.com/apache/kafka/pull/7305
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Race condition when creating topics and changing their configuration
>
>
> Key: KAFKA-8813
> URL: https://issues.apache.org/jira/browse/KAFKA-8813
> Project: Kafka
> Issue Type: Bug
>Reporter: Gwen Shapira
>Assignee: Vikas Singh
>Priority: Major
>
> In Partition.createLog we do:
> {code:java}
> val props = stateStore.fetchTopicConfig()
> val config = LogConfig.fromProps(logManager.currentDefaultConfig.originals,
> props)
> val log = logManager.getOrCreateLog(topicPartition, config, isNew,
> isFutureReplica)
> {code}
> [https://github.com/apache/kafka/blob/33d06082117d971cdcddd4f01392006b543f3c01/core/src/main/scala/kafka/cluster/Partition.scala#L314-L316|https://github.com/apache/kafka/blob/33d06082117d971cdcddd4f01392006b543f3c01/core/src/main/scala/kafka/cluster/Partition.scala#L314-L316]
> Config changes that arrive after configs are loaded from ZK, but before
> LogManager added the partition to `futureLogs` or `currentLogs` where the
> dynamic config handlers picks up topics to update their configs, will be lost.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KAFKA-8813) Race condition when creating topics and changing their configuration
[
https://issues.apache.org/jira/browse/KAFKA-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16923832#comment-16923832
]
ASF GitHub Bot commented on KAFKA-8813:
---
soondenana commented on pull request #7305: KAFKA-8813: Refresh log config if
it's updated before initializaiton
URL: https://github.com/apache/kafka/pull/7305
A partition log in initialized in following steps:
1. Fetch log config from ZK
2. Call LogManager.getOrCreateLog which creates the Log object, then
3. Registers the Log object
Step #3 enables Configuration update thread to deliver configuration
updates to the log. But if any update arrives between step #1 and #3
then that update is missed. It breaks following use case:
1. Create a topic with default configuration, and immediately after that
2. Update the configuration of topic
There is a race condition here and in random cases update made in
seocond step will get dropped.
This change fixes it by tracking updates arriving between step #1 and #3
Once a Partition is done initialzing log, it checks if it has missed any
update. If yes, then the configuration is read from ZK again.
Added unit tests to make sure a dirty configuration is refreshed. Tested
on local cluster to make sure that topic configuration and updates are
handled correctly.
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Race condition when creating topics and changing their configuration
>
>
> Key: KAFKA-8813
> URL: https://issues.apache.org/jira/browse/KAFKA-8813
> Project: Kafka
> Issue Type: Bug
>Reporter: Gwen Shapira
>Assignee: Vikas Singh
>Priority: Major
>
> In Partition.createLog we do:
> {code:java}
> val props = stateStore.fetchTopicConfig()
> val config = LogConfig.fromProps(logManager.currentDefaultConfig.originals,
> props)
> val log = logManager.getOrCreateLog(topicPartition, config, isNew,
> isFutureReplica)
> {code}
> [https://github.com/apache/kafka/blob/33d06082117d971cdcddd4f01392006b543f3c01/core/src/main/scala/kafka/cluster/Partition.scala#L314-L316|https://github.com/apache/kafka/blob/33d06082117d971cdcddd4f01392006b543f3c01/core/src/main/scala/kafka/cluster/Partition.scala#L314-L316]
> Config changes that arrive after configs are loaded from ZK, but before
> LogManager added the partition to `futureLogs` or `currentLogs` where the
> dynamic config handlers picks up topics to update their configs, will be lost.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (KAFKA-8813) Race condition when creating topics and changing their configuration
[
https://issues.apache.org/jira/browse/KAFKA-8813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916232#comment-16916232
]
Vikas Singh commented on KAFKA-8813:
I am working on a fix for this issue. Expect a PR sometime this week.
> Race condition when creating topics and changing their configuration
>
>
> Key: KAFKA-8813
> URL: https://issues.apache.org/jira/browse/KAFKA-8813
> Project: Kafka
> Issue Type: Bug
>Reporter: Gwen Shapira
>Assignee: Vikas Singh
>Priority: Major
>
> In Partition.createLog we do:
> {code:java}
> val config = LogConfig.fromProps(logManager.currentDefaultConfig.originals,
> props)
> val log = logManager.getOrCreateLog(topicPartition, config, isNew,
> isFutureReplica)
> {code}
> https://github.com/apache/kafka/blob/33d06082117d971cdcddd4f01392006b543f3c01/core/src/main/scala/kafka/cluster/Partition.scala#L315-L316
> Config changes that arrive after configs are loaded from ZK, but before
> LogManager added the partition to `futureLogs` or `currentLogs` where the
> dynamic config handlers picks up topics to update their configs, will be lost.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
