Hello hudeqi, I apologize if the KIP and discussion have diverged, as I've been trying to add detail rather than propose changes.
> Why can't we use the follower fetch protocol? What you've described sounds like a very reasonable implementation of CCR. I purposely have not specified any implementation details so far and have been focusing only on the user-facing semantics of the feature. You are of course welcome to add details of how the follower-fetch implementation would work to the KIP. I think maybe this wording in the KIP was ambiguous: "Are not eligible for fetch-from-follower on the source cluster". I clarified the justification for this earlier in Tom's point D3. But to make the statement itself more clear: Consumers on the source cluster will not see a cross-cluster leader or followers as valid replicas to fetch from for load-sharing or latency optimization. Consumers on the target cluster will see the cross-cluster followers as valid replicas to fetch from, as they will appear as normal replicas. This was only meant to describe the relationship of the remote replicas with Consumers, not with each other. I hope this is more clear. Greg On Sat, Oct 7, 2023 at 1:38 AM hudeqi <16120...@bjtu.edu.cn> wrote: > > Hi, Greg. > > After reading this KIP and your discussion, I feel that it is very divergent. > I can only start from one of them: > Why can't we use the follower fetch protocol? The leader of the target > cluster topic partition can be treated as the follower of the source cluster > topic partition leader, and the fetched data is directly appended to the > local log (the remote fetch thread is inherited to the follower fetch thread, > thereby retaining the offset of the log), so that consumer/ producer client > can be omitted. Of course, this is just data replication. I may have to think > more about group offset/acl/config replication. > > best, > hudeqi
