Hi, Tom, Thanks for the KIP. It definitely addresses one of the pain points in partition reassignment. Another issue that it also addresses is the ZK node size limit when writing the reassignment JSON.
My only concern is that the KIP needs to create one watcher per reassigned partition. This could add overhead in ZK and complexity for debugging when lots of partitions are being reassigned simultaneously. We could potentially improve this by introducing a separate ZK path for change notification as we do for configs. For example, every time we change the assignment for a set of partitions, we could further write a sequential node /admin/reassignment_changes/[change_x]. That way, the controller only needs to watch the change path. Once a change is triggered, the controller can read everything under /admin/reassignments/. Jun On Wed, Dec 6, 2017 at 1:19 PM, Tom Bentley <t.j.bent...@gmail.com> wrote: > Hi, > > This is still very new, but I wanted some quick feedback on a preliminary > KIP which could, I think, help with providing an AdminClient API for > partition reassignment. > > https://cwiki.apache.org/confluence/display/KAFKA/KIP- > 236%3A+Interruptible+Partition+Reassignment > > I wasn't sure whether to start fleshing out a whole AdminClient API in this > KIP (which would make it very big, and difficult to read), or whether to > break it down into smaller KIPs (which makes it easier to read and > implement in pieces, but harder to get a high-level picture of the ultimate > destination). For now I've gone for a very small initial KIP, but I'm happy > to sketch the bigger picture here if people are interested. > > Cheers, > > Tom >
