[
https://issues.apache.org/jira/browse/KAFKA-4896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16020447#comment-16020447
]
Onur Karaman commented on KAFKA-4896:
-------------------------------------
handleGroupImmigration only gets called from handling a LeaderAndIsrRequest.
Its cache load is synchronized over the ReplicaManager's replicaStateChangeLock.
handleGroupEmigration gets called in two places: when handling a
LeaderAndIsrRequest as well as from StopReplicaRequest. Its cache removal is
synchronized over the ReplicaManager's replicaStateChangeLock only in the
LeaderAndIsrRequest handling path.
I think even with today's single scheduler thread, we might have an interleaved
load and removal problem when the following are happening concurrently:
1. handleGroupImmigration is called for a partition P through
LeaderAndIsrRequest
2. handleGroupEmigration is called for a partition P through StopReplicaRequest
This should be rare. Controller does a blocking send and receive, so I think
this should only happen if:
1. one of the requests times out while the controller stays the same
2. the controller moves
[~junrao] does this edge case sound right?
> Offset loading can use more threads
> -----------------------------------
>
> Key: KAFKA-4896
> URL: https://issues.apache.org/jira/browse/KAFKA-4896
> Project: Kafka
> Issue Type: Improvement
> Components: core
> Affects Versions: 0.10.2.0
> Reporter: Jun Rao
> Assignee: Abhishek Mendhekar
> Labels: newbie
>
> Currently, in GroupMetadataManager, we have a single thread for loading the
> offset cache. We could speed it up with more threads.
> /* single-thread scheduler to handle offset/group metadata cache loading and
> unloading */
> private val scheduler = new KafkaScheduler(threads = 1, threadNamePrefix =
> "group-metadata-manager-")
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
