Hi Greg,
> I don't think I understand what you mean here. Are you suggesting an
alternative to the Admin API? An external project could certainly
build such a component with the Admin API.
So I was thinking of something more complex than the Admin API, an external
service that can instruct
Hello hudeqi,
I apologize if the KIP and discussion have diverged, as I've been
trying to add detail rather than propose changes.
> Why can't we use the follower fetch protocol?
What you've described sounds like a very reasonable implementation of
CCR. I purposely have not specified any
Hi, Greg.
After reading this KIP and your discussion, I feel that it is very divergent. I
can only start from one of them:
Why can't we use the follower fetch protocol? The leader of the target cluster
topic partition can be treated as the follower of the source cluster topic
partition leader,
Hey Viktor,
Please do! This is a draft, and it's open for you to edit to include
your new ideas :)
I don't think I understand what you mean here. Are you suggesting an
alternative to the Admin API? An external project could certainly
build such a component with the Admin API.
Thanks, Greg
On
Hi Greg,
Sure, I'll expand it with my thoughts. Is it fine if I add it to the KIP
and update this discussion?
Another thing that crossed my mind is that in MM2 you can handle configs
and replication flow in a central place because it is a separate component.
I think that for use-cases where
Hey Viktor,
Thanks for thinking about Tiered Storage. I'm not so familiar there,
so if you could add some of your expectations about how the two
features will interact, I would appreciate that.
It appears to me that follower-fetch-from-remote is a significant
optimization within TS, and so
Hi Greg,
Thanks for the answers. I think they all make sense.
Another point I realized last evening is that now that tiered storage (TS)
is available, it might complicate things with CCR. What I'm thinking of is
that if you have multiple clusters in multiple regions, enabling the object
Hi Viktor,
Thanks for your questions! I agree, replication is very fundamental in
Kafka, so it's been implemented in many different ways by different
people. I hope that this is the last implementation we'll need, but
every software engineer says that :)
GT-1: I think as this KIP is very focused
Hi Greg,
Seems like finding the perfect replication solution is a never ending story
for Kafka :).
Some general thoughts:
GT-1. While as you say it would be good to have some kind of built-in
replication in Kafka, we definitely need to understand the problem better
to provide a better solution.
Hey Tom,
Thanks for the high-level questions, as I am certainly approaching
this KIP differently than I've seen before.
I think that ideally this KIP will expand to include lots of
requirements and possible implementations, and that through discussion
we can narrow the scope and form a roadmap
Hi Greg,
Thanks for this KIP! It is obviously very ambitious, but it's great to have
a conversation about it.
I'll start with some general points:
Do you have a plan in mind for how to proceed with elaborating this KIP?
While I like how you're involving the community in elaborating the KIP, I
11 matches
Mail list logo
