Hi Jun, Thanks for looking at it again. Here's the new spec. (I fixed the typo in it too.) {"name": "id", "type": "string", "doc": option[UUID]}
Justine On Thu, Oct 1, 2020 at 5:03 PM Jun Rao <j...@confluent.io> wrote: > Hi, Justine, > > Thanks for the update. The KIP looks good to me now. Just a minor comment > below. > > 30. Perhaps "option[UUID]" can be put in the doc. > > Jun > > On Thu, Oct 1, 2020 at 3:28 PM Justine Olshan <jols...@confluent.io> > wrote: > > > Hi Jun, > > Thanks for the response! > > > > 30. I think I might have changed this in between. The current version > > says: {"name": > > "id", "type": "option[UUID]"}, "doc": topic id} > > I have switched to the option type to cover the migration case where a > > TopicZNode does not yet have a topic ID. > > I understand that due to json, this field is written as a string, so if I > > should move the "option[uuid]" to the doc field and the type should be > > "string" please let me know. > > > > 40. I've added a definition for UUID. > > 41,42. Fixed > > > > Thank you, > > Justine > > > > On Wed, Sep 30, 2020 at 1:15 PM Jun Rao <j...@confluent.io> wrote: > > > > > Hi, Justine, > > > > > > Thanks for the summary. Just a few minor comments blow. > > > > > > 30. {"name": "id", "type": "string", "doc": "version id"}}: The doc > > should > > > say UUID. The issue seems unfixed. > > > > > > 40. Since UUID is public facing, could you include its definition? > > > > > > 41. StopReplicaResponse still includes the topic field. > > > > > > 42. "It is unnecessary to include the name of the topic in the > following > > > Request/Response calls" It would be useful to include all modified > > requests > > > (e.g. produce) in the list below. > > > > > > Thanks, > > > > > > Jun > > > > > > > > > On Wed, Sep 30, 2020 at 10:17 AM Justine Olshan <jols...@confluent.io> > > > wrote: > > > > > > > Hello again, > > > > > > > > I've taken some time to discuss some of the remaining points brought > up > > > by > > > > the previous emails offline. Here are some of the conclusions. > > > > > > > > 1. Directory Structure: > > > > There was some talk about whether the directory structure should be > > > changed > > > > to replace all topic names with topic IDs. > > > > This will be a useful change, but will prevent downgrades. It will be > > > best > > > > to wait until a major release, likely alongside KIP-500 changes that > > will > > > > prevent downgrades. I've updated the KIP to include this change with > > the > > > > note about migration and deprecation. > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-log.dirlayout > > > > > > > > 2. Partition Metadata file > > > > There was some disagreement about the inclusion of the partition > > metadata > > > > file. > > > > This file will be needed to persist the topic ID, especially while we > > > still > > > > have the old directory structure. Even after the changes, this file > can > > > be > > > > useful for debugging and recovery. > > > > Creating a single mapping file from topic names to topic IDs was > > > > considered, but ultimately scrapped as it would not be as easy to > > > maintain. > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PartitionMetadatafile > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-PersistingTopicIDs > > > > > > > > 3. Produce Protocols > > > > After some further discussion, this replacing the topic name with > topic > > > ID > > > > in these protocols has been added to the KIP. > > > > This will cut down on the size of the protocol. Since changes to > fetch > > > are > > > > included, it does make sense to update these protocols. > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-Produce > > > > > > > > 4. KIP-500 Compatibility > > > > After some discussion with those involved with KIP-500, it seems best > > to > > > > use a sentinel topic ID for the metadata topic that is used before > the > > > > first controller election. > > > > We had an issue where this metadata topic may not be assigned an ID > > > before > > > > utilizing the Vote and Fetch protocols. It was decided to reserve a > > > unique > > > > ID for this topic to be used until the controller could give the > topic > > a > > > > unique ID. > > > > Having the topic keep the sentinel ID (not be reassigned to a unique > > ID) > > > > was considered, but it seemed like a good idea to leave the option > open > > > for > > > > the metadata topic to have a unique ID in cases where it would need > to > > be > > > > differentiated from other clusters' metadata topics. (ex. tiered > > storage) > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers#KIP516:TopicIdentifiers-CompatibilitywithKIP-500 > > > > > > > > I've also split up the KIP into sub-tasks on the JIRA. Hopefully this > > > will > > > > provide a better idea about what tasks we have, and eventually > provide > > a > > > > place to see what's done and what is left. > > > > If there is a task I am missing, please let me know! > > > > https://issues.apache.org/jira/browse/KAFKA-8872 > > > > > > > > Of course, these decisions are not set in stone, and I would love to > > hear > > > > any feedback. > > > > > > > > Thanks, > > > > Justine > > > > > > > > On Mon, Sep 28, 2020 at 11:38 AM Justine Olshan < > jols...@confluent.io> > > > > wrote: > > > > > > > > > Hello all, > > > > > > > > > > I just wanted to follow up on this discussion. Did we come to an > > > > > understanding about the directory structure? > > > > > > > > > > I think the biggest question here is what is acceptable to leave > out > > > due > > > > > to scope vs. what is considered to be too much tech debt. > > > > > This KIP is already pretty large with the number of changes, but it > > > also > > > > > makes sense to do things correctly, so I'd love to hear everyone's > > > > thoughts. > > > > > > > > > > Thanks, > > > > > Justine > > > > > > > > > > On Fri, Sep 25, 2020 at 8:19 AM Lucas Bradstreet < > lu...@confluent.io > > > > > > > > wrote: > > > > > > > > > >> Hi Ismael, > > > > >> > > > > >> If you do not store it in a metadata file or in the directory > > > structure > > > > >> would you then > > > > >> require the LeaderAndIsrRequest following startup to give you some > > > > notion > > > > >> of > > > > >> topic name in memory? We would still need this information for the > > > older > > > > >> protocols, but > > > > >> perhaps this is what's meant by tech debt. > > > > >> > > > > >> Once we're free of the old non-topicID requests then I think you > > > > wouldn't > > > > >> need to retain the topic name. > > > > >> I think the ability to easily look up topic names associated with > > > > >> partition > > > > >> directories would still be missed > > > > >> when diagnosing issues, though maybe it wouldn't be a deal breaker > > > with > > > > >> the > > > > >> right tooling. > > > > >> > > > > >> Thanks, > > > > >> > > > > >> Lucas > > > > >> > > > > >> On Fri, Sep 25, 2020 at 7:55 AM Ismael Juma <ism...@juma.me.uk> > > > wrote: > > > > >> > > > > >> > Hi Lucas, > > > > >> > > > > > >> > Why would you include the name and id? I think you'd want to > > remove > > > > the > > > > >> > name from the directory name right? Jason's suggestion was that > if > > > you > > > > >> > remove the name from the directory, then why would you need the > id > > > > name > > > > >> > mapping file? > > > > >> > > > > > >> > Ismael > > > > >> > > > > > >> > On Thu, Sep 24, 2020 at 4:24 PM Lucas Bradstreet < > > > lu...@confluent.io> > > > > >> > wrote: > > > > >> > > > > > >> > > > 2. Part of the usage of the file is to have persistent > storage > > > of > > > > >> the > > > > >> > > topic > > > > >> > > ID and use it to compare with the ID supplied in the > > LeaderAndIsr > > > > >> > Request. > > > > >> > > There is some discussion in the KIP about changes to the > > directory > > > > >> > > structure, but I believe directory changes were considered to > be > > > out > > > > >> of > > > > >> > > scope when the KIP was written. > > > > >> > > > > > > >> > > > > > > >> > > Yeah, I was hoping to get a better understanding of why it was > > > taken > > > > >> out > > > > >> > of > > > > >> > > scope. Perhaps Lucas Bradstreet might have more insight about > > the > > > > >> > decision. > > > > >> > > Basically my point is that we have to create additional > > > > infrastructure > > > > >> > here > > > > >> > > to support the name/id mapping, so I wanted to understand if > we > > > just > > > > >> > > consider this a sort of tech debt. It would be useful to cover > > how > > > > we > > > > >> > > handle the case when this file gets corrupted. Seems like we > > just > > > > >> have to > > > > >> > > trust that it matches whatever the controller tells us and > > rewrite > > > > it? > > > > >> > > > > > > >> > > > > > > >> > > Hi Jason, Justine, > > > > >> > > > > > > >> > > My thought process is that we will likely need the metadata > file > > > > >> whether > > > > >> > we > > > > >> > > rename the directories or not. > > > > >> > > Linux supports filenames of up to 255 bytes and that would not > > be > > > > >> enough > > > > >> > to > > > > >> > > support a directory name > > > > >> > > including both the name and topic ID. Given that fact, it > > seemed > > > > >> > > reasonable to add the metadata file > > > > >> > > and leave the directory rename until some time in the future > > > > (possibly > > > > >> > > never). > > > > >> > > > > > > >> > > If the file does get corrupted I think you're right. We would > > > either > > > > >> have > > > > >> > > to trust it matches what the controller tells us > > > > >> > > or error out and let an administrator resolve it by checking > > > across > > > > >> > > replicas for consistency. > > > > >> > > > > > > >> > > Lucas > > > > >> > > > > > > >> > > > > > > >> > > On Thu, Sep 24, 2020 at 3:41 PM Jason Gustafson < > > > ja...@confluent.io > > > > > > > > > >> > > wrote: > > > > >> > > > > > > >> > > > Thanks Justine. Responses below: > > > > >> > > > > > > > >> > > > > 1. Yes, the directory will still be based on the topic > > names. > > > > >> > > > LeaderAndIsrRequest is one of the few requests that will > still > > > > >> contain > > > > >> > > the > > > > >> > > > topic name. So I think we have this covered. Sorry for > > > confusion. > > > > >> > > > > > > > >> > > > Ah, you're right. My eyes passed right over the field. > > > > >> > > > > > > > >> > > > > 2. Part of the usage of the file is to have persistent > > storage > > > > of > > > > >> the > > > > >> > > > topic > > > > >> > > > ID and use it to compare with the ID supplied in the > > > LeaderAndIsr > > > > >> > > Request. > > > > >> > > > There is some discussion in the KIP about changes to the > > > directory > > > > >> > > > structure, but I believe directory changes were considered > to > > be > > > > >> out of > > > > >> > > > scope when the KIP was written. > > > > >> > > > > > > > >> > > > Yeah, I was hoping to get a better understanding of why it > was > > > > taken > > > > >> > out > > > > >> > > of > > > > >> > > > scope. Perhaps Lucas Bradstreet might have more insight > about > > > the > > > > >> > > decision. > > > > >> > > > Basically my point is that we have to create additional > > > > >> infrastructure > > > > >> > > here > > > > >> > > > to support the name/id mapping, so I wanted to understand if > > we > > > > just > > > > >> > > > consider this a sort of tech debt. It would be useful to > cover > > > how > > > > >> we > > > > >> > > > handle the case when this file gets corrupted. Seems like we > > > just > > > > >> have > > > > >> > to > > > > >> > > > trust that it matches whatever the controller tells us and > > > rewrite > > > > >> it? > > > > >> > > > > > > > >> > > > > 3. I think this is a good point, but I again I wonder > about > > > the > > > > >> scope > > > > >> > > of > > > > >> > > > the KIP. Most of the changes mentioned in the KIP are for > > > > supporting > > > > >> > > topic > > > > >> > > > deletion and I believe that is why the produce request was > > > listed > > > > >> under > > > > >> > > > future work. > > > > >> > > > > > > > >> > > > That's fair. I brought it up since `Fetch` is already > > included. > > > If > > > > >> > we've > > > > >> > > > got `Metadata` and `Fetch`, seems we may as well do > `Produce` > > > and > > > > >> save > > > > >> > an > > > > >> > > > extra kip. No strong objection though if you want to leave > it > > > out. > > > > >> > > > > > > > >> > > > > > > > >> > > > -Jason > > > > >> > > > > > > > >> > > > > > > > >> > > > On Thu, Sep 24, 2020 at 3:26 PM Justine Olshan < > > > > >> jols...@confluent.io> > > > > >> > > > wrote: > > > > >> > > > > > > > >> > > > > Hi Jason, > > > > >> > > > > > > > > >> > > > > Thanks for your comments. > > > > >> > > > > > > > > >> > > > > 1. Yes, the directory will still be based on the topic > > names. > > > > >> > > > > LeaderAndIsrRequest is one of the few requests that will > > still > > > > >> > contain > > > > >> > > > the > > > > >> > > > > topic name. So I think we have this covered. Sorry for > > > > confusion. > > > > >> > > > > > > > > >> > > > > 2. Part of the usage of the file is to have persistent > > storage > > > > of > > > > >> the > > > > >> > > > topic > > > > >> > > > > ID and use it to compare with the ID supplied in the > > > > LeaderAndIsr > > > > >> > > > Request. > > > > >> > > > > There is some discussion in the KIP about changes to the > > > > directory > > > > >> > > > > structure, but I believe directory changes were considered > > to > > > be > > > > >> out > > > > >> > of > > > > >> > > > > scope when the KIP was written. > > > > >> > > > > > > > > >> > > > > 3. I think this is a good point, but I again I wonder > about > > > the > > > > >> scope > > > > >> > > of > > > > >> > > > > the KIP. Most of the changes mentioned in the KIP are for > > > > >> supporting > > > > >> > > > topic > > > > >> > > > > deletion and I believe that is why the produce request was > > > > listed > > > > >> > under > > > > >> > > > > future work. > > > > >> > > > > > > > > >> > > > > 4. This sounds like it might be a good solution, but I > will > > > need > > > > >> to > > > > >> > > > discuss > > > > >> > > > > more with KIP-500 folks to get the details right. > > > > >> > > > > > > > > >> > > > > Thanks, > > > > >> > > > > Justine > > > > >> > > > > > > > > >> > > > > On Thu, Sep 24, 2020 at 12:30 PM Jason Gustafson < > > > > >> ja...@confluent.io > > > > >> > > > > > > >> > > > > wrote: > > > > >> > > > > > > > > >> > > > > > Hi Justine, > > > > >> > > > > > > > > > >> > > > > > Thanks for picking up this work. I have a few > > > > >> questions/comments: > > > > >> > > > > > > > > > >> > > > > > 1. It sounds like the directory structure is still going > > to > > > be > > > > >> > based > > > > >> > > on > > > > >> > > > > > topic names. Do I have that right? One complication is > > that > > > > the > > > > >> > > > > > LeaderAndIsr request does not include the topic name any > > > > longer. > > > > >> > This > > > > >> > > > > means > > > > >> > > > > > that a replica must wait for the UpdateMetadata request > to > > > > >> arrive > > > > >> > > with > > > > >> > > > > the > > > > >> > > > > > topic name mapping before it can create the directory. I > > > > wonder > > > > >> if > > > > >> > it > > > > >> > > > > would > > > > >> > > > > > be simpler to avoid assumptions on the ordering of > > > > >> UpdateMetadata > > > > >> > and > > > > >> > > > let > > > > >> > > > > > LeaderAndIsr include the topic name as well. Feels like > we > > > are > > > > >> not > > > > >> > > > saving > > > > >> > > > > > that much by excluding it. > > > > >> > > > > > > > > > >> > > > > > 2. On a related note, it seems that the reason we have > the > > > > >> > partition > > > > >> > > > > > metadata file is because we are not changing the > directory > > > > >> > structure. > > > > >> > > > We > > > > >> > > > > > need it so that we remember which directories map to > which > > > > topic > > > > >> > id. > > > > >> > > I > > > > >> > > > > > think it would be useful to add some detail about why > > > changing > > > > >> the > > > > >> > > > > > directory layout was rejected. Basically trying to > > > understand > > > > >> how > > > > >> > > much > > > > >> > > > > > effort we are saving by skipping this step if we have to > > > > >> implement > > > > >> > > this > > > > >> > > > > > additional bookkeeping. One concern I have for example > is > > > that > > > > >> the > > > > >> > > > > > partition metadata file gets corrupt and then all bets > are > > > off > > > > >> as > > > > >> > far > > > > >> > > > as > > > > >> > > > > > topic consistency. > > > > >> > > > > > > > > > >> > > > > > 3. Is it worth adding support now for topic ids in the > > > > `Produce` > > > > >> > > > request > > > > >> > > > > > now? Overhead is mentioned as one of the motivations and > > the > > > > >> best > > > > >> > > APIs > > > > >> > > > to > > > > >> > > > > > get savings from are `Produce` and `Fetch`. > > > > >> > > > > > > > > > >> > > > > > 4. When it comes to bootstrapping the metadata topic > with > > > > >> respect > > > > >> > to > > > > >> > > > > > KIP-500, one suggestion would be to reserve a sentinel > > topic > > > > ID > > > > >> > which > > > > >> > > > can > > > > >> > > > > > be used initially by new brokers. When the first > > controller > > > is > > > > >> > > elected, > > > > >> > > > > it > > > > >> > > > > > can assign a topicId to the metadata topic. This is a > bit > > > > >> similar > > > > >> > to > > > > >> > > > how > > > > >> > > > > we > > > > >> > > > > > were planning to generate the clusterId. > > > > >> > > > > > > > > > >> > > > > > Thanks, > > > > >> > > > > > Jason > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > On Thu, Sep 24, 2020 at 11:10 AM Jun Rao < > > j...@confluent.io> > > > > >> wrote: > > > > >> > > > > > > > > > >> > > > > > > Hi, Justine, > > > > >> > > > > > > > > > > >> > > > > > > Thanks for the updated KIP. A few more comments below. > > > > >> > > > > > > > > > > >> > > > > > > 30. {"name": "id", "type": "string", "doc": "version > > > id"}}: > > > > >> The > > > > >> > > doc > > > > >> > > > > > should > > > > >> > > > > > > say UUID. > > > > >> > > > > > > > > > > >> > > > > > > 31. LeaderAndIsrResponse v5 and StopReplicaResponse > v4 : > > > It > > > > >> seems > > > > >> > > > there > > > > >> > > > > > is > > > > >> > > > > > > no need to add topic_id at partitions level. > > > > >> > > > > > > > > > > >> > > > > > > 32. Regarding partition metadata file. Perhaps the key > > can > > > > be > > > > >> a > > > > >> > > > single > > > > >> > > > > > > word, sth like the following. > > > > >> > > > > > > version: 0 > > > > >> > > > > > > topic_id: 46bdb63f-9e8d-4a38-bf7b-ee4eb2a794e4 > > > > >> > > > > > > > > > > >> > > > > > > 33. Another tricky thing that I realized is how to > > support > > > > the > > > > >> > > > metadata > > > > >> > > > > > > topic introduced in KIP-595. It's a bit tricky to > > assign a > > > > >> UUID > > > > >> > to > > > > >> > > > the > > > > >> > > > > > > metadata topic since we have a chicken-and-egg > problem. > > > The > > > > >> > > > controller > > > > >> > > > > > > needs to persist the UUID in the metadata topic in > order > > > to > > > > >> > assign > > > > >> > > > one > > > > >> > > > > > > successfully, but the metadata topic is needed to > elect > > a > > > > >> > > controller > > > > >> > > > > > > (KIP-631). So, this probably needs a bit more thought. > > > > >> > > > > > > > > > > >> > > > > > > Jun > > > > >> > > > > > > > > > > >> > > > > > > On Thu, Sep 24, 2020 at 3:04 AM Ismael Juma < > > > > >> ism...@juma.me.uk> > > > > >> > > > wrote: > > > > >> > > > > > > > > > > >> > > > > > > > Also, can we provide more details on how the > Partition > > > > >> Metadata > > > > >> > > > file > > > > >> > > > > > will > > > > >> > > > > > > > be used? > > > > >> > > > > > > > > > > > >> > > > > > > > Ismael > > > > >> > > > > > > > > > > > >> > > > > > > > On Thu, Sep 24, 2020 at 3:01 AM Ismael Juma < > > > > >> ism...@juma.me.uk > > > > >> > > > > > > >> > > > > wrote: > > > > >> > > > > > > > > > > > >> > > > > > > > > Hi Justine, > > > > >> > > > > > > > > > > > > >> > > > > > > > > I think we need to update the "Rejected > > Alternatives" > > > > >> section > > > > >> > > to > > > > >> > > > > take > > > > >> > > > > > > > into > > > > >> > > > > > > > > account that the proposal now removes the topic > name > > > > from > > > > >> the > > > > >> > > > fetch > > > > >> > > > > > > > > request. Also, if we are removing it from the > Fetch > > > > >> request, > > > > >> > > does > > > > >> > > > > it > > > > >> > > > > > > make > > > > >> > > > > > > > > sense not to remove it from similar requests like > > > > >> > > > > ListOffsetRequest? > > > > >> > > > > > > > > > > > > >> > > > > > > > > Ismael > > > > >> > > > > > > > > > > > > >> > > > > > > > > On Thu, Sep 24, 2020 at 2:46 AM David Jacot < > > > > >> > > dja...@confluent.io > > > > >> > > > > > > > > >> > > > > > > wrote: > > > > >> > > > > > > > > > > > > >> > > > > > > > >> Hi Justine, > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> Thanks for the KIP. I finally had time to read it > > :). > > > > It > > > > >> is > > > > >> > a > > > > >> > > > > great > > > > >> > > > > > > > >> improvement. > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> I have a few comments/questions: > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> 1. It seems that the schema of the > > StopReplicaRequest > > > > is > > > > >> > > > slightly > > > > >> > > > > > > > >> outdated. > > > > >> > > > > > > > >> We > > > > >> > > > > > > > >> did some changes as part of KIP-570. V3 is > already > > > > >> organized > > > > >> > > by > > > > >> > > > > > > topics. > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> 2. I just want to make sure that I understand the > > > > >> > > reconciliation > > > > >> > > > > > > > >> logic correctly. When > > > > >> > > > > > > > >> an "INCREMENTAL" LeaderAndIsr Request is > received, > > > the > > > > >> > broker > > > > >> > > > will > > > > >> > > > > > > also > > > > >> > > > > > > > >> reconcile > > > > >> > > > > > > > >> when the local uuid does not match the uuid in > the > > > > >> request, > > > > >> > > > right? > > > > >> > > > > > In > > > > >> > > > > > > > this > > > > >> > > > > > > > >> case, the > > > > >> > > > > > > > >> local log is staged for deletion. > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> 3. In the documentation of the ` > > > > >> delete.stale.topic.delay.ms > > > > >> > ` > > > > >> > > > > > config, > > > > >> > > > > > > it > > > > >> > > > > > > > >> says "When a > > > > >> > > > > > > > >> FULL LeaderAndIsrRequest is received..." but I > > > suppose > > > > it > > > > >> > > > applies > > > > >> > > > > to > > > > >> > > > > > > > both > > > > >> > > > > > > > >> types. > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> Best, > > > > >> > > > > > > > >> David > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> On Thu, Sep 24, 2020 at 1:40 AM Justine Olshan < > > > > >> > > > > > jols...@confluent.io> > > > > >> > > > > > > > >> wrote: > > > > >> > > > > > > > >> > > > > >> > > > > > > > >> > Hi Jun, > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > Thanks for the comments. I apologize for some > of > > > the > > > > >> typos > > > > >> > > and > > > > >> > > > > > > > >> confusion. > > > > >> > > > > > > > >> > I’ve updated the KIP to fix some of the issues > > you > > > > >> > > mentioned. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 20.2 I’ve changed the type to String. > > > > >> > > > > > > > >> > 20.1/3 I’ve updated the TopicZNode to fix > > > formatting > > > > >> and > > > > >> > > > reflect > > > > >> > > > > > the > > > > >> > > > > > > > >> latest > > > > >> > > > > > > > >> > version before this change. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 21. You are correct and I’ve removed this line. > > > I’ve > > > > >> also > > > > >> > > > added > > > > >> > > > > a > > > > >> > > > > > > line > > > > >> > > > > > > > >> > mentioning an IBP bump is necessary for > migration > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 22. I think the wording was unclear but your > > > summary > > > > is > > > > >> > what > > > > >> > > > was > > > > >> > > > > > > > >> intended > > > > >> > > > > > > > >> > by this line. I’ve updated to make this point > > more > > > > >> clear. > > > > >> > > > > “Remove > > > > >> > > > > > > > >> deleted > > > > >> > > > > > > > >> > topics from replicas by sending > > StopReplicaRequest > > > V3 > > > > >> > before > > > > >> > > > the > > > > >> > > > > > IBP > > > > >> > > > > > > > >> bump > > > > >> > > > > > > > >> > using the old logic, and using V4 and the new > > logic > > > > >> with > > > > >> > > topic > > > > >> > > > > IDs > > > > >> > > > > > > > after > > > > >> > > > > > > > >> > the IBP bump.” > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 23. I’ve removed the unspecified type from the > > KIP > > > > and > > > > >> > > mention > > > > >> > > > > > that > > > > >> > > > > > > > IBP > > > > >> > > > > > > > >> > will be used to handle this request. “IBP will > be > > > > used > > > > >> to > > > > >> > > > > > determine > > > > >> > > > > > > > >> whether > > > > >> > > > > > > > >> > this new form of the request will be used. For > > > older > > > > >> > > requests, > > > > >> > > > > we > > > > >> > > > > > > will > > > > >> > > > > > > > >> > ignore this field and default to previous > > > behavior.” > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 24. I’ve fixed this typo. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 25. I've added a topics at a higher level for > > > > >> > > > > LeaderAndIsrResponse > > > > >> > > > > > > v5, > > > > >> > > > > > > > >> > StopReplicaResponse v4. I've also changed > > > > >> > StopReplicaRequest > > > > >> > > > v4 > > > > >> > > > > to > > > > >> > > > > > > > have > > > > >> > > > > > > > >> > topics at a higher level. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 26. I’ve updated forgotten_topics_data--added > the > > > > >> topic ID > > > > >> > > and > > > > >> > > > > > > removed > > > > >> > > > > > > > >> the > > > > >> > > > > > > > >> > topic name > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 27. I’ve decided on plain text, and I’ve added > an > > > > >> example. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > 28. I’ve added this idea to future work. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > Thanks again for taking a look, > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > Justine > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > On Wed, Sep 23, 2020 at 10:28 AM Jun Rao < > > > > >> > j...@confluent.io> > > > > >> > > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > Hi, Justine, > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > Thanks for the response. Made another pass. A > > few > > > > >> more > > > > >> > > > > comments > > > > >> > > > > > > > below. > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 20. znode schema: > > > > >> > > > > > > > >> > > 20.1 It seems that {"name": "version", > "type": > > > > "int", > > > > >> > > "id": > > > > >> > > > > > > "UUID", > > > > >> > > > > > > > >> > "doc": > > > > >> > > > > > > > >> > > "version id"} should be {"name": "version", > > > "type": > > > > >> > > "int"}, > > > > >> > > > > > > {"name": > > > > >> > > > > > > > >> > "id", > > > > >> > > > > > > > >> > > "type": "UUID", "doc": "version id"}. > > > > >> > > > > > > > >> > > 20.2 The znode format is JSON which doesn't > > have > > > > UUID > > > > >> > > type. > > > > >> > > > So > > > > >> > > > > > the > > > > >> > > > > > > > >> type > > > > >> > > > > > > > >> > > probably should be string? > > > > >> > > > > > > > >> > > 20.3 Also, the existing format used seems > > > outdated. > > > > >> It > > > > >> > > > should > > > > >> > > > > > have > > > > >> > > > > > > > the > > > > >> > > > > > > > >> > > following format. > > > > >> > > > > > > > >> > > Json.encodeAsBytes(Map( > > > > >> > > > > > > > >> > > "version" -> 2, > > > > >> > > > > > > > >> > > "partitions" -> > > > replicaAssignmentJson.asJava, > > > > >> > > > > > > > >> > > "adding_replicas" -> > > > > >> > > > > addingReplicasAssignmentJson.asJava, > > > > >> > > > > > > > >> > > "removing_replicas" -> > > > > >> > > > > > removingReplicasAssignmentJson.asJava > > > > >> > > > > > > > >> > > ).asJava) > > > > >> > > > > > > > >> > > } > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 21. Migration: The KIP says "The migration > > > process > > > > >> can > > > > >> > > take > > > > >> > > > > > place > > > > >> > > > > > > > >> without > > > > >> > > > > > > > >> > > an inter-broker protocol bump, as the format > > > stored > > > > >> in > > > > >> > > > > > > > >> > > /brokers/topics/[topic] will be compatible > with > > > > older > > > > >> > > broker > > > > >> > > > > > > > >> versions." > > > > >> > > > > > > > >> > > However, since we are bumping up the version > of > > > > >> > > inter-broker > > > > >> > > > > > > > >> requests, it > > > > >> > > > > > > > >> > > seems that we need to use IBP for migration. > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 22. The KIP says "Remove deleted topics from > > > > >> replicas by > > > > >> > > > > sending > > > > >> > > > > > > > >> > > StopReplicaRequest V3 for any topics which do > > not > > > > >> > contain > > > > >> > > a > > > > >> > > > > > topic > > > > >> > > > > > > > ID, > > > > >> > > > > > > > >> and > > > > >> > > > > > > > >> > > V4 for any topics which do contain a topic > > ID.". > > > > >> > However, > > > > >> > > if > > > > >> > > > > we > > > > >> > > > > > > use > > > > >> > > > > > > > >> IBP, > > > > >> > > > > > > > >> > it > > > > >> > > > > > > > >> > > seems that the controller will either send > > > > >> > > > StopReplicaRequest > > > > >> > > > > V3 > > > > >> > > > > > > > >> > > or StopReplicaRequest V4, but never mixed V3 > > and > > > V4 > > > > >> for > > > > >> > > > > > different > > > > >> > > > > > > > >> topics. > > > > >> > > > > > > > >> > > Basically, before the IBP bump, V3 will be > > used. > > > > >> After > > > > >> > the > > > > >> > > > IBP > > > > >> > > > > > > bump, > > > > >> > > > > > > > >> > > topicId will be created and V4 will be used. > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 23. Given that we depend on IBP, do we still > > need > > > > "0 > > > > >> > > > > > UNSPECIFIED" > > > > >> > > > > > > > >> > > in LeaderAndIsr? > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 24. LeaderAndIsrResponse v5 : It still has > the > > > > topic > > > > >> > > field. > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 25. LeaderAndIsrResponse v5, > > StopReplicaResponse > > > > v4: > > > > >> > Could > > > > >> > > > we > > > > >> > > > > > use > > > > >> > > > > > > > this > > > > >> > > > > > > > >> > > opportunity to organize the response in 2 > > levels, > > > > >> first > > > > >> > by > > > > >> > > > > > topic, > > > > >> > > > > > > > >> then by > > > > >> > > > > > > > >> > > partition, as most other requests/responses? > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 26. FetchRequest v13 : Should > > > forgotten_topics_data > > > > >> use > > > > >> > > > > topicId > > > > >> > > > > > > too? > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 27. "This file can either be plain text > > > (key/value > > > > >> > pairs) > > > > >> > > or > > > > >> > > > > > > JSON." > > > > >> > > > > > > > >> Have > > > > >> > > > > > > > >> > we > > > > >> > > > > > > > >> > > decided which one to use? Also, it would be > > > helpful > > > > >> to > > > > >> > > > provide > > > > >> > > > > > an > > > > >> > > > > > > > >> > example. > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > 28. Future improvement: Another future > > > improvement > > > > >> > > > opportunity > > > > >> > > > > > is > > > > >> > > > > > > to > > > > >> > > > > > > > >> use > > > > >> > > > > > > > >> > > topicId in > GroupMetadataManager.offsetCommitKey > > > in > > > > >> the > > > > >> > > > > > > offset_commit > > > > >> > > > > > > > >> > topic. > > > > >> > > > > > > > >> > > This may save some space. > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > Thanks, > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > Jun > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > On Wed, Sep 23, 2020 at 8:50 AM Justine > Olshan > > < > > > > >> > > > > > > > jols...@confluent.io> > > > > >> > > > > > > > >> > > wrote: > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > > Hi Tom, > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > Thanks for the comment. I think this is a > > > really > > > > >> good > > > > >> > > idea > > > > >> > > > > and > > > > >> > > > > > > it > > > > >> > > > > > > > >> has > > > > >> > > > > > > > >> > > been > > > > >> > > > > > > > >> > > > added to the KIP under the newly added > > tooling > > > > >> > section. > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > Thanks again, > > > > >> > > > > > > > >> > > > Justine > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > On Wed, Sep 23, 2020 at 3:17 AM Tom > Bentley < > > > > >> > > > > > > tbent...@redhat.com> > > > > >> > > > > > > > >> > wrote: > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > Hi Justine, > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > I know you started the vote thread, but > on > > > > >> > re-reading > > > > >> > > > the > > > > >> > > > > > KIP > > > > >> > > > > > > I > > > > >> > > > > > > > >> > noticed > > > > >> > > > > > > > >> > > > > that although the topic id is included in > > the > > > > >> > > > > > MetadataResponse > > > > >> > > > > > > > >> it's > > > > >> > > > > > > > >> > not > > > > >> > > > > > > > >> > > > > surfaced in the output from > > `kafka-topics.sh > > > > >> > > > --describe`. > > > > >> > > > > > > Maybe > > > > >> > > > > > > > >> that > > > > >> > > > > > > > >> > > was > > > > >> > > > > > > > >> > > > > intentional because ids are intentionally > > not > > > > >> really > > > > >> > > > > > something > > > > >> > > > > > > > the > > > > >> > > > > > > > >> > user > > > > >> > > > > > > > >> > > > > should care deeply about, but it would > also > > > > make > > > > >> > life > > > > >> > > > > harder > > > > >> > > > > > > for > > > > >> > > > > > > > >> > anyone > > > > >> > > > > > > > >> > > > > debugging Kafka and this would likely get > > > worse > > > > >> the > > > > >> > > more > > > > >> > > > > > topic > > > > >> > > > > > > > ids > > > > >> > > > > > > > >> > got > > > > >> > > > > > > > >> > > > > rolled out across the protocols, clients > > etc. > > > > It > > > > >> > seems > > > > >> > > > > > likely > > > > >> > > > > > > > that > > > > >> > > > > > > > >> > > > > `kafka-topics.sh` will eventually need > the > > > > >> ability > > > > >> > to > > > > >> > > > show > > > > >> > > > > > the > > > > >> > > > > > > > id > > > > >> > > > > > > > >> of > > > > >> > > > > > > > >> > a > > > > >> > > > > > > > >> > > > > topic and perhaps find a topic name given > > an > > > > id. > > > > >> Is > > > > >> > > > there > > > > >> > > > > > any > > > > >> > > > > > > > >> reason > > > > >> > > > > > > > >> > > not > > > > >> > > > > > > > >> > > > to > > > > >> > > > > > > > >> > > > > implement that in this KIP? > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > Many thanks, > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > Tom > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > On Mon, Sep 21, 2020 at 9:54 PM Justine > > > Olshan > > > > < > > > > >> > > > > > > > >> jols...@confluent.io > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > > > wrote: > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > Hi all, > > > > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > After thinking about it, I've decided > to > > > > remove > > > > >> > the > > > > >> > > > > topic > > > > >> > > > > > > name > > > > >> > > > > > > > >> from > > > > >> > > > > > > > >> > > the > > > > >> > > > > > > > >> > > > > > Fetch Request and Response after all. > > Since > > > > >> there > > > > >> > > are > > > > >> > > > so > > > > >> > > > > > > many > > > > >> > > > > > > > of > > > > >> > > > > > > > >> > > these > > > > >> > > > > > > > >> > > > > > requests per second, it is worth > removing > > > the > > > > >> > extra > > > > >> > > > > > > > information. > > > > >> > > > > > > > >> > I've > > > > >> > > > > > > > >> > > > > > updated the KIP to reflect this change. > > > > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > Please let me know if there is anything > > > else > > > > we > > > > >> > > should > > > > >> > > > > > > discuss > > > > >> > > > > > > > >> > before > > > > >> > > > > > > > >> > > > > > voting. > > > > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > Thank you, > > > > >> > > > > > > > >> > > > > > Justine > > > > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > On Fri, Sep 18, 2020 at 9:46 AM Justine > > > > Olshan > > > > >> < > > > > >> > > > > > > > >> > jols...@confluent.io > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > wrote: > > > > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > Hi Jun, > > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > > I see what you are saying. For now we > > can > > > > >> remove > > > > >> > > the > > > > >> > > > > > extra > > > > >> > > > > > > > >> > > > information. > > > > >> > > > > > > > >> > > > > > > I'll leave the option to add more > > fields > > > to > > > > >> the > > > > >> > > file > > > > >> > > > > in > > > > >> > > > > > > the > > > > >> > > > > > > > >> > future. > > > > >> > > > > > > > >> > > > The > > > > >> > > > > > > > >> > > > > > KIP > > > > >> > > > > > > > >> > > > > > > has been updated to reflect this > > change. > > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > > Thanks, > > > > >> > > > > > > > >> > > > > > > Justine > > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > > On Fri, Sep 18, 2020 at 8:46 AM Jun > > Rao < > > > > >> > > > > > j...@confluent.io > > > > >> > > > > > > > > > > > >> > > > > > > > >> > wrote: > > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> Hi, Justine, > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> Thanks for the reply. > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> 13. If the log directory is the > source > > > of > > > > >> > truth, > > > > >> > > it > > > > >> > > > > > means > > > > >> > > > > > > > >> that > > > > >> > > > > > > > >> > the > > > > >> > > > > > > > >> > > > > > >> redundant info in the metadata file > > will > > > > be > > > > >> > > > ignored. > > > > >> > > > > > Then > > > > >> > > > > > > > the > > > > >> > > > > > > > >> > > > question > > > > >> > > > > > > > >> > > > > > is > > > > >> > > > > > > > >> > > > > > >> why do we need to put the redundant > > info > > > > in > > > > >> the > > > > >> > > > > > metadata > > > > >> > > > > > > > file > > > > >> > > > > > > > >> > now? > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> Thanks, > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> Jun > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> On Thu, Sep 17, 2020 at 5:07 PM > > Justine > > > > >> Olshan > > > > >> > < > > > > >> > > > > > > > >> > > > jols...@confluent.io> > > > > >> > > > > > > > >> > > > > > >> wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> > Hi Jun, > > > > >> > > > > > > > >> > > > > > >> > Thanks for the quick response! > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > 12. I've decided to bump up the > > > versions > > > > >> on > > > > >> > the > > > > >> > > > > > > requests > > > > >> > > > > > > > >> and > > > > >> > > > > > > > >> > > > updated > > > > >> > > > > > > > >> > > > > > the > > > > >> > > > > > > > >> > > > > > >> > KIP. I think it's good we > thoroughly > > > > >> > discussed > > > > >> > > > the > > > > >> > > > > > > > options > > > > >> > > > > > > > >> > here, > > > > >> > > > > > > > >> > > > so > > > > >> > > > > > > > >> > > > > we > > > > >> > > > > > > > >> > > > > > >> know > > > > >> > > > > > > > >> > > > > > >> > we made a good choice. :) > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > 13. This is an interesting > > situation. > > > I > > > > >> think > > > > >> > > if > > > > >> > > > > this > > > > >> > > > > > > > does > > > > >> > > > > > > > >> > occur > > > > >> > > > > > > > >> > > > we > > > > >> > > > > > > > >> > > > > > >> should > > > > >> > > > > > > > >> > > > > > >> > give a warning. I agree that it's > > hard > > > > to > > > > >> > know > > > > >> > > > the > > > > >> > > > > > > source > > > > >> > > > > > > > >> of > > > > >> > > > > > > > >> > > truth > > > > >> > > > > > > > >> > > > > for > > > > >> > > > > > > > >> > > > > > >> sure > > > > >> > > > > > > > >> > > > > > >> > since the directory or the file > > could > > > be > > > > >> > > manually > > > > >> > > > > > > > >> modified. I > > > > >> > > > > > > > >> > > > guess > > > > >> > > > > > > > >> > > > > > the > > > > >> > > > > > > > >> > > > > > >> > directory could be used as the > > source > > > of > > > > >> > truth. > > > > >> > > > To > > > > >> > > > > be > > > > >> > > > > > > > >> honest, > > > > >> > > > > > > > >> > > I'm > > > > >> > > > > > > > >> > > > > not > > > > >> > > > > > > > >> > > > > > >> > really sure what happens in kafka > > when > > > > the > > > > >> > log > > > > >> > > > > > > directory > > > > >> > > > > > > > is > > > > >> > > > > > > > >> > > > renamed > > > > >> > > > > > > > >> > > > > > >> > manually in such a way. I'm also > > > > >> wondering if > > > > >> > > the > > > > >> > > > > > > > >> situation is > > > > >> > > > > > > > >> > > > > > >> recoverable > > > > >> > > > > > > > >> > > > > > >> > in this scenario. > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > Thanks, > > > > >> > > > > > > > >> > > > > > >> > Justine > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > On Thu, Sep 17, 2020 at 4:28 PM > Jun > > > Rao > > > > < > > > > >> > > > > > > > j...@confluent.io> > > > > >> > > > > > > > >> > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > > Hi, Justine, > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > Thanks for the reply. > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > 12. I don't have a strong > > preference > > > > >> > either. > > > > >> > > > > > However, > > > > >> > > > > > > > if > > > > >> > > > > > > > >> we > > > > >> > > > > > > > >> > > need > > > > >> > > > > > > > >> > > > > IBP > > > > >> > > > > > > > >> > > > > > >> > > anyway, maybe it's easier to > just > > > bump > > > > >> up > > > > >> > the > > > > >> > > > > > version > > > > >> > > > > > > > for > > > > >> > > > > > > > >> > all > > > > >> > > > > > > > >> > > > > inter > > > > >> > > > > > > > >> > > > > > >> > broker > > > > >> > > > > > > > >> > > > > > >> > > requests and add the topic id > > field > > > > as a > > > > >> > > > regular > > > > >> > > > > > > > field. A > > > > >> > > > > > > > >> > > > regular > > > > >> > > > > > > > >> > > > > > >> field > > > > >> > > > > > > > >> > > > > > >> > is > > > > >> > > > > > > > >> > > > > > >> > > a bit more concise in wire > > transfer > > > > >> than a > > > > >> > > > > flexible > > > > >> > > > > > > > >> field. > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > 13. The confusion that I was > > > referring > > > > >> to > > > > >> > is > > > > >> > > > > > between > > > > >> > > > > > > > the > > > > >> > > > > > > > >> > topic > > > > >> > > > > > > > >> > > > > name > > > > >> > > > > > > > >> > > > > > >> and > > > > >> > > > > > > > >> > > > > > >> > > partition number between the log > > dir > > > > and > > > > >> > the > > > > >> > > > > > metadata > > > > >> > > > > > > > >> file. > > > > >> > > > > > > > >> > > For > > > > >> > > > > > > > >> > > > > > >> example, > > > > >> > > > > > > > >> > > > > > >> > if > > > > >> > > > > > > > >> > > > > > >> > > the log dir is topicA-1 and the > > > > metadata > > > > >> > file > > > > >> > > > in > > > > >> > > > > it > > > > >> > > > > > > has > > > > >> > > > > > > > >> > topicB > > > > >> > > > > > > > >> > > > and > > > > >> > > > > > > > >> > > > > > >> > > partition 0 (say due to a bug or > > > > manual > > > > >> > > > > > > modification), > > > > >> > > > > > > > >> which > > > > >> > > > > > > > >> > > one > > > > >> > > > > > > > >> > > > > do > > > > >> > > > > > > > >> > > > > > we > > > > >> > > > > > > > >> > > > > > >> > use > > > > >> > > > > > > > >> > > > > > >> > > as the source of truth? > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > Jun > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > On Thu, Sep 17, 2020 at 3:43 PM > > > > Justine > > > > >> > > Olshan > > > > >> > > > < > > > > >> > > > > > > > >> > > > > > jols...@confluent.io> > > > > >> > > > > > > > >> > > > > > >> > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > > Hi Jun, > > > > >> > > > > > > > >> > > > > > >> > > > Thanks for the comments. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > >> > > > 12. I bumped the > > > LeaderAndIsrRequest > > > > >> > > because > > > > >> > > > I > > > > >> > > > > > > > removed > > > > >> > > > > > > > >> the > > > > >> > > > > > > > >> > > > topic > > > > >> > > > > > > > >> > > > > > >> name > > > > >> > > > > > > > >> > > > > > >> > > field > > > > >> > > > > > > > >> > > > > > >> > > > in the response. It may be > > > possible > > > > to > > > > >> > > avoid > > > > >> > > > > > > bumping > > > > >> > > > > > > > >> the > > > > >> > > > > > > > >> > > > version > > > > >> > > > > > > > >> > > > > > >> > without > > > > >> > > > > > > > >> > > > > > >> > > > that change, but I may be > > missing > > > > >> > > something. > > > > >> > > > > > > > >> > > > > > >> > > > I believe StopReplica is > > actually > > > on > > > > >> > > version > > > > >> > > > 3 > > > > >> > > > > > now, > > > > >> > > > > > > > but > > > > >> > > > > > > > >> > > > because > > > > >> > > > > > > > >> > > > > > >> > version 2 > > > > >> > > > > > > > >> > > > > > >> > > > is flexible, I kept that > listed > > as > > > > >> > version > > > > >> > > 2 > > > > >> > > > on > > > > >> > > > > > the > > > > >> > > > > > > > KIP > > > > >> > > > > > > > >> > > page. > > > > >> > > > > > > > >> > > > > > >> However, > > > > >> > > > > > > > >> > > > > > >> > > you > > > > >> > > > > > > > >> > > > > > >> > > > may be right in that we may > need > > > to > > > > >> bump > > > > >> > > the > > > > >> > > > > > > version > > > > >> > > > > > > > on > > > > >> > > > > > > > >> > > > > > StopReplica > > > > >> > > > > > > > >> > > > > > >> to > > > > >> > > > > > > > >> > > > > > >> > > deal > > > > >> > > > > > > > >> > > > > > >> > > > with deletion differently as > > > > mentioned > > > > >> > > > above. I > > > > >> > > > > > > don't > > > > >> > > > > > > > >> know > > > > >> > > > > > > > >> > > if > > > > >> > > > > > > > >> > > > I > > > > >> > > > > > > > >> > > > > > >> have a > > > > >> > > > > > > > >> > > > > > >> > > big > > > > >> > > > > > > > >> > > > > > >> > > > preference over used tagged > > fields > > > > or > > > > >> > not. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > >> > > > 13. I was thinking that in the > > > case > > > > >> where > > > > >> > > the > > > > >> > > > > > file > > > > >> > > > > > > > and > > > > >> > > > > > > > >> the > > > > >> > > > > > > > >> > > > > request > > > > >> > > > > > > > >> > > > > > >> > topic > > > > >> > > > > > > > >> > > > > > >> > > > ids don't match, it means that > > the > > > > >> > broker's > > > > >> > > > > > > topic/the > > > > >> > > > > > > > >> one > > > > >> > > > > > > > >> > in > > > > >> > > > > > > > >> > > > the > > > > >> > > > > > > > >> > > > > > >> file > > > > >> > > > > > > > >> > > > > > >> > has > > > > >> > > > > > > > >> > > > > > >> > > > been deleted. In that case, we > > > would > > > > >> need > > > > >> > > to > > > > >> > > > > > delete > > > > >> > > > > > > > the > > > > >> > > > > > > > >> > old > > > > >> > > > > > > > >> > > > > topic > > > > >> > > > > > > > >> > > > > > >> and > > > > >> > > > > > > > >> > > > > > >> > > start > > > > >> > > > > > > > >> > > > > > >> > > > receiving the new version. If > > the > > > > >> topic > > > > >> > > name > > > > >> > > > > were > > > > >> > > > > > > to > > > > >> > > > > > > > >> > change, > > > > >> > > > > > > > >> > > > but > > > > >> > > > > > > > >> > > > > > the > > > > >> > > > > > > > >> > > > > > >> > ids > > > > >> > > > > > > > >> > > > > > >> > > > still matched, the file would > > also > > > > >> need > > > > >> > to > > > > >> > > > > > update. > > > > >> > > > > > > > Am I > > > > >> > > > > > > > >> > > > missing > > > > >> > > > > > > > >> > > > > a > > > > >> > > > > > > > >> > > > > > >> case > > > > >> > > > > > > > >> > > > > > >> > > > where the file would be > correct > > > and > > > > >> not > > > > >> > the > > > > >> > > > > > > request? > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > >> > > > Thanks, > > > > >> > > > > > > > >> > > > > > >> > > > Justine > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > >> > > > On Thu, Sep 17, 2020 at 3:18 > PM > > > Jun > > > > >> Rao < > > > > >> > > > > > > > >> j...@confluent.io > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > Hi, Justine, > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > Thanks for the reply. A > couple > > > of > > > > >> more > > > > >> > > > > comments > > > > >> > > > > > > > >> below. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > 12. ListOffset and > > > OffsetForLeader > > > > >> > > > currently > > > > >> > > > > > > don't > > > > >> > > > > > > > >> > support > > > > >> > > > > > > > >> > > > > > >> flexible > > > > >> > > > > > > > >> > > > > > >> > > > fields. > > > > >> > > > > > > > >> > > > > > >> > > > > So, we have to bump up the > > > version > > > > >> > number > > > > >> > > > and > > > > >> > > > > > use > > > > >> > > > > > > > >> IBP at > > > > >> > > > > > > > >> > > > least > > > > >> > > > > > > > >> > > > > > for > > > > >> > > > > > > > >> > > > > > >> > > these > > > > >> > > > > > > > >> > > > > > >> > > > > two requests. Note that it > > seems > > > > >> 2.7.0 > > > > >> > > will > > > > >> > > > > > > require > > > > >> > > > > > > > >> IBP > > > > >> > > > > > > > >> > > > anyway > > > > >> > > > > > > > >> > > > > > >> > because > > > > >> > > > > > > > >> > > > > > >> > > of > > > > >> > > > > > > > >> > > > > > >> > > > > changes in KAFKA-10435. > Also, > > it > > > > >> seems > > > > >> > > that > > > > >> > > > > the > > > > >> > > > > > > > >> version > > > > >> > > > > > > > >> > > for > > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequest and > > > > StopReplica > > > > >> are > > > > >> > > > > bumped > > > > >> > > > > > > even > > > > >> > > > > > > > >> > though > > > > >> > > > > > > > >> > > > we > > > > >> > > > > > > > >> > > > > > only > > > > >> > > > > > > > >> > > > > > >> > > added > > > > >> > > > > > > > >> > > > > > >> > > > a > > > > >> > > > > > > > >> > > > > > >> > > > > tagged field. But since IBP > is > > > > >> needed > > > > >> > > > anyway, > > > > >> > > > > > we > > > > >> > > > > > > > may > > > > >> > > > > > > > >> > want > > > > >> > > > > > > > >> > > to > > > > >> > > > > > > > >> > > > > > >> revisit > > > > >> > > > > > > > >> > > > > > >> > > the > > > > >> > > > > > > > >> > > > > > >> > > > > overall tagged field choice. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > 13. The only downside is the > > > > >> potential > > > > >> > > > > > confusion > > > > >> > > > > > > on > > > > >> > > > > > > > >> > which > > > > >> > > > > > > > >> > > > one > > > > >> > > > > > > > >> > > > > is > > > > >> > > > > > > > >> > > > > > >> the > > > > >> > > > > > > > >> > > > > > >> > > > source > > > > >> > > > > > > > >> > > > > > >> > > > > of truth if they don't > match. > > > > >> Another > > > > >> > > > option > > > > >> > > > > is > > > > >> > > > > > > to > > > > >> > > > > > > > >> > include > > > > >> > > > > > > > >> > > > > those > > > > >> > > > > > > > >> > > > > > >> > fields > > > > >> > > > > > > > >> > > > > > >> > > > in > > > > >> > > > > > > > >> > > > > > >> > > > > the metadata file when we > > > actually > > > > >> > change > > > > >> > > > the > > > > >> > > > > > > > >> directory > > > > >> > > > > > > > >> > > > > > structure. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > Thanks, > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > Jun > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > On Thu, Sep 17, 2020 at 2:01 > > PM > > > > >> Justine > > > > >> > > > > Olshan > > > > >> > > > > > < > > > > >> > > > > > > > >> > > > > > >> jols...@confluent.io > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > Hello all, > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > I've thought some more > about > > > > >> removing > > > > >> > > the > > > > >> > > > > > topic > > > > >> > > > > > > > >> name > > > > >> > > > > > > > >> > > field > > > > >> > > > > > > > >> > > > > > from > > > > >> > > > > > > > >> > > > > > >> > some > > > > >> > > > > > > > >> > > > > > >> > > of > > > > >> > > > > > > > >> > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > requests. On closer > > inspection > > > > of > > > > >> the > > > > >> > > > > > > > >> > > requests/responses, > > > > >> > > > > > > > >> > > > it > > > > >> > > > > > > > >> > > > > > >> seems > > > > >> > > > > > > > >> > > > > > >> > > that > > > > >> > > > > > > > >> > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > internal changes would be > > much > > > > >> larger > > > > >> > > > than > > > > >> > > > > I > > > > >> > > > > > > > >> expected. > > > > >> > > > > > > > >> > > > Some > > > > >> > > > > > > > >> > > > > > >> > protocols > > > > >> > > > > > > > >> > > > > > >> > > > > > involve clients, so they > > would > > > > >> > require > > > > >> > > > > > changes > > > > >> > > > > > > > too. > > > > >> > > > > > > > >> > I'm > > > > >> > > > > > > > >> > > > > > thinking > > > > >> > > > > > > > >> > > > > > >> > that > > > > >> > > > > > > > >> > > > > > >> > > > for > > > > >> > > > > > > > >> > > > > > >> > > > > > now, removing the topic > name > > > > from > > > > >> > these > > > > >> > > > > > > requests > > > > >> > > > > > > > >> and > > > > >> > > > > > > > >> > > > > responses > > > > >> > > > > > > > >> > > > > > >> are > > > > >> > > > > > > > >> > > > > > >> > > out > > > > >> > > > > > > > >> > > > > > >> > > > of > > > > >> > > > > > > > >> > > > > > >> > > > > > scope. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > I have decided to just > keep > > > the > > > > >> > change > > > > >> > > > > > > > >> > > > LeaderAndIsrResponse > > > > >> > > > > > > > >> > > > > to > > > > >> > > > > > > > >> > > > > > >> > remove > > > > >> > > > > > > > >> > > > > > >> > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > topic name, and have > updated > > > the > > > > >> KIP > > > > >> > to > > > > >> > > > > > reflect > > > > >> > > > > > > > >> this > > > > >> > > > > > > > >> > > > > change. I > > > > >> > > > > > > > >> > > > > > >> have > > > > >> > > > > > > > >> > > > > > >> > > > also > > > > >> > > > > > > > >> > > > > > >> > > > > > mentioned the other > requests > > > and > > > > >> > > > responses > > > > >> > > > > in > > > > >> > > > > > > > >> future > > > > >> > > > > > > > >> > > work. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > I'm hoping to start the > > voting > > > > >> > process > > > > >> > > > > soon, > > > > >> > > > > > so > > > > >> > > > > > > > >> let me > > > > >> > > > > > > > >> > > > know > > > > >> > > > > > > > >> > > > > if > > > > >> > > > > > > > >> > > > > > >> > there > > > > >> > > > > > > > >> > > > > > >> > > is > > > > >> > > > > > > > >> > > > > > >> > > > > > anything else we should > > > discuss. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > Thank you, > > > > >> > > > > > > > >> > > > > > >> > > > > > Justine > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > On Tue, Sep 15, 2020 at > 3:57 > > > PM > > > > >> > Justine > > > > >> > > > > > Olshan > > > > >> > > > > > > < > > > > >> > > > > > > > >> > > > > > >> > jols...@confluent.io > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > Hello again, > > > > >> > > > > > > > >> > > > > > >> > > > > > > To follow up on some of > > the > > > > >> other > > > > >> > > > > comments: > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > 10/11) We can remove the > > > topic > > > > >> name > > > > >> > > > from > > > > >> > > > > > > these > > > > >> > > > > > > > >> > > > > > >> > requests/responses, > > > > >> > > > > > > > >> > > > > > >> > > > and > > > > >> > > > > > > > >> > > > > > >> > > > > > > that means we will just > > have > > > > to > > > > >> > make > > > > >> > > a > > > > >> > > > > few > > > > >> > > > > > > > >> internal > > > > >> > > > > > > > >> > > > > changes > > > > >> > > > > > > > >> > > > > > to > > > > >> > > > > > > > >> > > > > > >> > make > > > > >> > > > > > > > >> > > > > > >> > > > > > > partitions accessible by > > > topic > > > > >> id > > > > >> > and > > > > >> > > > > > > > partition. > > > > >> > > > > > > > >> I > > > > >> > > > > > > > >> > can > > > > >> > > > > > > > >> > > > > > update > > > > >> > > > > > > > >> > > > > > >> the > > > > >> > > > > > > > >> > > > > > >> > > KIP > > > > >> > > > > > > > >> > > > > > >> > > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > > remove them unless > anyone > > > > thinks > > > > >> > they > > > > >> > > > > > should > > > > >> > > > > > > > >> stay. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > 12) Addressed in the > > > previous > > > > >> > email. > > > > >> > > > I've > > > > >> > > > > > > > updated > > > > >> > > > > > > > >> > the > > > > >> > > > > > > > >> > > > KIP > > > > >> > > > > > > > >> > > > > to > > > > >> > > > > > > > >> > > > > > >> > > include > > > > >> > > > > > > > >> > > > > > >> > > > > > > tagged fields for the > > > requests > > > > >> and > > > > >> > > > > > responses. > > > > >> > > > > > > > >> (More > > > > >> > > > > > > > >> > on > > > > >> > > > > > > > >> > > > > that > > > > >> > > > > > > > >> > > > > > >> > below) > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > 13) I think part of the > > idea > > > > for > > > > >> > > > > including > > > > >> > > > > > > this > > > > >> > > > > > > > >> > > > > information > > > > >> > > > > > > > >> > > > > > >> is to > > > > >> > > > > > > > >> > > > > > >> > > > > prepare > > > > >> > > > > > > > >> > > > > > >> > > > > > > for future changes. > > Perhaps > > > > the > > > > >> > > > directory > > > > >> > > > > > > > >> structure > > > > >> > > > > > > > >> > > > might > > > > >> > > > > > > > >> > > > > > >> change > > > > >> > > > > > > > >> > > > > > >> > > from > > > > >> > > > > > > > >> > > > > > >> > > > > > > > topicName_partitionNumber > > to > > > > >> > > something > > > > >> > > > > like > > > > >> > > > > > > > >> > > > > > >> > > topicID_partitionNumber. > > > > >> > > > > > > > >> > > > > > >> > > > > Then > > > > >> > > > > > > > >> > > > > > >> > > > > > > it would be useful to > have > > > the > > > > >> > topic > > > > >> > > > name > > > > >> > > > > > in > > > > >> > > > > > > > the > > > > >> > > > > > > > >> > file > > > > >> > > > > > > > >> > > > > since > > > > >> > > > > > > > >> > > > > > it > > > > >> > > > > > > > >> > > > > > >> > > would > > > > >> > > > > > > > >> > > > > > >> > > > > not > > > > >> > > > > > > > >> > > > > > >> > > > > > be > > > > >> > > > > > > > >> > > > > > >> > > > > > > in the directory > > structure. > > > > >> > > Supporting > > > > >> > > > > > topic > > > > >> > > > > > > > >> renames > > > > >> > > > > > > > >> > > > might > > > > >> > > > > > > > >> > > > > > be > > > > >> > > > > > > > >> > > > > > >> > > easier > > > > >> > > > > > > > >> > > > > > >> > > > if > > > > >> > > > > > > > >> > > > > > >> > > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > > other fields are > included. > > > > Would > > > > >> > > there > > > > >> > > > be > > > > >> > > > > > any > > > > >> > > > > > > > >> > > downsides > > > > >> > > > > > > > >> > > > to > > > > >> > > > > > > > >> > > > > > >> > > including > > > > >> > > > > > > > >> > > > > > >> > > > > this > > > > >> > > > > > > > >> > > > > > >> > > > > > > information? > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > 14) Yes, we would need > to > > > > copy > > > > >> the > > > > >> > > > > > partition > > > > >> > > > > > > > >> > metadata > > > > >> > > > > > > > >> > > > > file > > > > >> > > > > > > > >> > > > > > in > > > > >> > > > > > > > >> > > > > > >> > this > > > > >> > > > > > > > >> > > > > > >> > > > > > > process. I've updated > the > > > KIP > > > > to > > > > >> > > > include > > > > >> > > > > > > this. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > 15) I believe Lucas > meant > > v1 > > > > >> and v2 > > > > >> > > > here. > > > > >> > > > > > He > > > > >> > > > > > > > was > > > > >> > > > > > > > >> > > > referring > > > > >> > > > > > > > >> > > > > > to > > > > >> > > > > > > > >> > > > > > >> how > > > > >> > > > > > > > >> > > > > > >> > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > > requests would fall > under > > > > >> different > > > > >> > > IBP > > > > >> > > > > and > > > > >> > > > > > > > meant > > > > >> > > > > > > > >> > that > > > > >> > > > > > > > >> > > > > older > > > > >> > > > > > > > >> > > > > > >> > > brokers > > > > >> > > > > > > > >> > > > > > >> > > > > > would > > > > >> > > > > > > > >> > > > > > >> > > > > > > have to use the older > > > version > > > > of > > > > >> > the > > > > >> > > > > > request > > > > >> > > > > > > > and > > > > >> > > > > > > > >> the > > > > >> > > > > > > > >> > > > > > existing > > > > >> > > > > > > > >> > > > > > >> > topic > > > > >> > > > > > > > >> > > > > > >> > > > > > > deletion process. At > > first, > > > it > > > > >> > seemed > > > > >> > > > > like > > > > >> > > > > > > > tagged > > > > >> > > > > > > > >> > > fields > > > > >> > > > > > > > >> > > > > > would > > > > >> > > > > > > > >> > > > > > >> > > > resolve > > > > >> > > > > > > > >> > > > > > >> > > > > > > the IBP issue. However, > we > > > may > > > > >> need > > > > >> > > IBP > > > > >> > > > > for > > > > >> > > > > > > > this > > > > >> > > > > > > > >> > > request > > > > >> > > > > > > > >> > > > > > after > > > > >> > > > > > > > >> > > > > > >> > all > > > > >> > > > > > > > >> > > > > > >> > > > > since > > > > >> > > > > > > > >> > > > > > >> > > > > > > the controller handles > the > > > > topic > > > > >> > > > deletion > > > > >> > > > > > > > >> > differently > > > > >> > > > > > > > >> > > > > > >> depending > > > > >> > > > > > > > >> > > > > > >> > on > > > > >> > > > > > > > >> > > > > > >> > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > IBP > > > > >> > > > > > > > >> > > > > > >> > > > > > > version. In an older > > > version, > > > > we > > > > >> > > can't > > > > >> > > > > just > > > > >> > > > > > > > send > > > > >> > > > > > > > >> a > > > > >> > > > > > > > >> > > > > > StopReplica > > > > >> > > > > > > > >> > > > > > >> > > delete > > > > >> > > > > > > > >> > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > > topic immediately like > > we'd > > > > >> want to > > > > >> > > for > > > > >> > > > > > this > > > > >> > > > > > > > KIP. > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > This makes me wonder if > we > > > > want > > > > >> > > tagged > > > > >> > > > > > fields > > > > >> > > > > > > > on > > > > >> > > > > > > > >> all > > > > >> > > > > > > > >> > > the > > > > >> > > > > > > > >> > > > > > >> requests > > > > >> > > > > > > > >> > > > > > >> > > > after > > > > >> > > > > > > > >> > > > > > >> > > > > > > all. Let me know your > > > > thoughts! > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > Justine > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > On Tue, Sep 15, 2020 at > > 1:03 > > > > PM > > > > >> > > Justine > > > > >> > > > > > > Olshan > > > > >> > > > > > > > < > > > > >> > > > > > > > >> > > > > > >> > > jols...@confluent.io > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >> Hi all, > > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun brought up a good > > point > > > > in > > > > >> his > > > > >> > > > last > > > > >> > > > > > > email > > > > >> > > > > > > > >> about > > > > >> > > > > > > > >> > > > > tagged > > > > >> > > > > > > > >> > > > > > >> > fields, > > > > >> > > > > > > > >> > > > > > >> > > > and > > > > >> > > > > > > > >> > > > > > >> > > > > > >> I've updated the KIP to > > > > reflect > > > > >> > that > > > > >> > > > the > > > > >> > > > > > > > >> changes to > > > > >> > > > > > > > >> > > > > > requests > > > > >> > > > > > > > >> > > > > > >> and > > > > >> > > > > > > > >> > > > > > >> > > > > > responses > > > > >> > > > > > > > >> > > > > > >> > > > > > >> will be in the form of > > > tagged > > > > >> > fields > > > > >> > > > to > > > > >> > > > > > > avoid > > > > >> > > > > > > > >> > > changing > > > > >> > > > > > > > >> > > > > IBP. > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> > > > > > >> Jun: I plan on sending > a > > > > >> followup > > > > >> > > > email > > > > >> > > > > to > > > > >> > > > > > > > >> address > > > > >> > > > > > > > >> > > some > > > > >> > > > > > > > >> > > > > of > > > > >> > > > > > > > >> > > > > > >> the > > > > >> > > > > > > > >> > > > > > >> > > other > > > > >> > > > > > > > >> > > > > > >> > > > > > >> points. > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> > > > > > >> Thanks, > > > > >> > > > > > > > >> > > > > > >> > > > > > >> Justine > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> > > > > > >> On Mon, Sep 14, 2020 at > > > 4:25 > > > > PM > > > > >> > Jun > > > > >> > > > Rao > > > > >> > > > > < > > > > >> > > > > > > > >> > > > > j...@confluent.io> > > > > >> > > > > > > > >> > > > > > >> > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Hi, Justine, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Thanks for the updated > > > KIP. > > > > A > > > > >> few > > > > >> > > > > > comments > > > > >> > > > > > > > >> below. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 10. LeaderAndIsr > > Response: > > > > Do > > > > >> we > > > > >> > > need > > > > >> > > > > the > > > > >> > > > > > > > topic > > > > >> > > > > > > > >> > > name? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 11. For the changed > > > > >> > > request/response, > > > > >> > > > > > other > > > > >> > > > > > > > >> than > > > > >> > > > > > > > >> > > > > > >> LeaderAndIsr, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UpdateMetadata, > > Metadata, > > > do > > > > >> we > > > > >> > > need > > > > >> > > > to > > > > >> > > > > > > > include > > > > >> > > > > > > > >> > the > > > > >> > > > > > > > >> > > > > topic > > > > >> > > > > > > > >> > > > > > >> name? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 12. It seems that > > upgrades > > > > >> don't > > > > >> > > > > require > > > > >> > > > > > > IBP. > > > > >> > > > > > > > >> Does > > > > >> > > > > > > > >> > > > that > > > > >> > > > > > > > >> > > > > > mean > > > > >> > > > > > > > >> > > > > > >> > the > > > > >> > > > > > > > >> > > > > > >> > > > new > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> fields > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> in all the > > > request/response > > > > >> are > > > > >> > > added > > > > >> > > > > as > > > > >> > > > > > > > tagged > > > > >> > > > > > > > >> > > fields > > > > >> > > > > > > > >> > > > > > >> without > > > > >> > > > > > > > >> > > > > > >> > > > > bumping > > > > >> > > > > > > > >> > > > > > >> > > > > > up > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the request version? > It > > > > would > > > > >> be > > > > >> > > > useful > > > > >> > > > > > to > > > > >> > > > > > > > make > > > > >> > > > > > > > >> > that > > > > >> > > > > > > > >> > > > > > clear. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 13. Partition Metadata > > > file: > > > > >> Do > > > > >> > we > > > > >> > > > need > > > > >> > > > > > to > > > > >> > > > > > > > >> include > > > > >> > > > > > > > >> > > the > > > > >> > > > > > > > >> > > > > > topic > > > > >> > > > > > > > >> > > > > > >> > name > > > > >> > > > > > > > >> > > > > > >> > > > and > > > > >> > > > > > > > >> > > > > > >> > > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> partition id since > they > > > are > > > > >> > implied > > > > >> > > > in > > > > >> > > > > > the > > > > >> > > > > > > > >> > directory > > > > >> > > > > > > > >> > > > > name? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 14. In the JBOD mode, > we > > > > >> support > > > > >> > > > > moving a > > > > >> > > > > > > > >> > > partition's > > > > >> > > > > > > > >> > > > > data > > > > >> > > > > > > > >> > > > > > >> from > > > > >> > > > > > > > >> > > > > > >> > > one > > > > >> > > > > > > > >> > > > > > >> > > > > > disk > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> another. Will the new > > > > >> partition > > > > >> > > > > metadata > > > > >> > > > > > > file > > > > >> > > > > > > > >> be > > > > >> > > > > > > > >> > > > copied > > > > >> > > > > > > > >> > > > > > >> during > > > > >> > > > > > > > >> > > > > > >> > > that > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> process? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> 15. The KIP says > "Remove > > > > >> deleted > > > > >> > > > topics > > > > >> > > > > > > from > > > > >> > > > > > > > >> > > replicas > > > > >> > > > > > > > >> > > > by > > > > >> > > > > > > > >> > > > > > >> > sending > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> StopReplicaRequest V2 > > for > > > > any > > > > >> > > topics > > > > >> > > > > > which > > > > >> > > > > > > do > > > > >> > > > > > > > >> not > > > > >> > > > > > > > >> > > > > contain > > > > >> > > > > > > > >> > > > > > a > > > > >> > > > > > > > >> > > > > > >> > topic > > > > >> > > > > > > > >> > > > > > >> > > > ID, > > > > >> > > > > > > > >> > > > > > >> > > > > > and > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> V3 for any topics > which > > do > > > > >> > contain > > > > >> > > a > > > > >> > > > > > topic > > > > >> > > > > > > > >> ID.". > > > > >> > > > > > > > >> > > > > However, > > > > >> > > > > > > > >> > > > > > it > > > > >> > > > > > > > >> > > > > > >> > > seems > > > > >> > > > > > > > >> > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> updated controller > will > > > > create > > > > >> > all > > > > >> > > > > > missing > > > > >> > > > > > > > >> topic > > > > >> > > > > > > > >> > IDs > > > > >> > > > > > > > >> > > > > first > > > > >> > > > > > > > >> > > > > > >> > before > > > > >> > > > > > > > >> > > > > > >> > > > > doing > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> other actions. So, is > > > > >> > > > > StopReplicaRequest > > > > >> > > > > > V2 > > > > >> > > > > > > > >> > needed? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Jun > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> On Fri, Sep 11, 2020 > at > > > > 10:31 > > > > >> AM > > > > >> > > John > > > > >> > > > > > > > Roesler < > > > > >> > > > > > > > >> > > > > > >> > > vvcep...@apache.org > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Thanks, Justine! > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Your response seems > > > > >> compelling > > > > >> > to > > > > >> > > > me. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > -John > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > On Fri, 2020-09-11 > at > > > > 10:17 > > > > >> > > -0700, > > > > >> > > > > > > Justine > > > > >> > > > > > > > >> > Olshan > > > > >> > > > > > > > >> > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hello all, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Thanks for > > continuing > > > > the > > > > >> > > > > > discussion! I > > > > >> > > > > > > > >> have a > > > > >> > > > > > > > >> > > few > > > > >> > > > > > > > >> > > > > > >> > responses > > > > >> > > > > > > > >> > > > > > >> > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > your > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > points. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Tom: You are > correct > > > in > > > > >> that > > > > >> > > this > > > > >> > > > > KIP > > > > >> > > > > > > has > > > > >> > > > > > > > >> not > > > > >> > > > > > > > >> > > > > > mentioned > > > > >> > > > > > > > >> > > > > > >> the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > DeleteTopicsRequest. I > > > > >> think > > > > >> > > that > > > > >> > > > > > this > > > > >> > > > > > > > >> would > > > > >> > > > > > > > >> > be > > > > >> > > > > > > > >> > > > out > > > > >> > > > > > > > >> > > > > of > > > > >> > > > > > > > >> > > > > > >> > scope > > > > >> > > > > > > > >> > > > > > >> > > > for > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> now, but > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > may be something > > worth > > > > >> adding > > > > >> > > in > > > > >> > > > > the > > > > >> > > > > > > > >> future. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > John: We did > > consider > > > > >> > sequence > > > > >> > > > ids, > > > > >> > > > > > but > > > > >> > > > > > > > >> there > > > > >> > > > > > > > >> > > are > > > > >> > > > > > > > >> > > > a > > > > >> > > > > > > > >> > > > > > few > > > > >> > > > > > > > >> > > > > > >> > > reasons > > > > >> > > > > > > > >> > > > > > >> > > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> favor > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. There are > > > several > > > > >> > cases > > > > >> > > > > where > > > > >> > > > > > > > topics > > > > >> > > > > > > > >> > from > > > > >> > > > > > > > >> > > > > > >> different > > > > >> > > > > > > > >> > > > > > >> > > > > clusters > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> may > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > interact now and > in > > > the > > > > >> > future. > > > > >> > > > For > > > > >> > > > > > > > >> example, > > > > >> > > > > > > > >> > > > Mirror > > > > >> > > > > > > > >> > > > > > >> Maker 2 > > > > >> > > > > > > > >> > > > > > >> > > may > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> benefit > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > from being able to > > > > detect > > > > >> > when > > > > >> > > a > > > > >> > > > > > > cluster > > > > >> > > > > > > > >> being > > > > >> > > > > > > > >> > > > > > mirrored > > > > >> > > > > > > > >> > > > > > >> is > > > > >> > > > > > > > >> > > > > > >> > > > > deleted > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > recreated and > > globally > > > > >> unique > > > > >> > > > > > > identifiers > > > > >> > > > > > > > >> > would > > > > >> > > > > > > > >> > > > make > > > > >> > > > > > > > >> > > > > > >> > > resolving > > > > >> > > > > > > > >> > > > > > >> > > > > > issues > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > easier than > sequence > > > IDs > > > > >> > which > > > > >> > > > may > > > > >> > > > > > > > collide > > > > >> > > > > > > > >> > > between > > > > >> > > > > > > > >> > > > > > >> > clusters. > > > > >> > > > > > > > >> > > > > > >> > > > > > KIP-405 > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > (tiered storage) > > will > > > > also > > > > >> > > > benefit > > > > >> > > > > > from > > > > >> > > > > > > > >> > globally > > > > >> > > > > > > > >> > > > > > unique > > > > >> > > > > > > > >> > > > > > >> IDs > > > > >> > > > > > > > >> > > > > > >> > > as > > > > >> > > > > > > > >> > > > > > >> > > > > > shared > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > buckets may be > used > > > > >> between > > > > >> > > > > clusters. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Globally unique > IDs > > > > would > > > > >> > also > > > > >> > > > make > > > > >> > > > > > > > >> > > functionality > > > > >> > > > > > > > >> > > > > like > > > > >> > > > > > > > >> > > > > > >> > moving > > > > >> > > > > > > > >> > > > > > >> > > > > > topics > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > between disparate > > > > clusters > > > > >> > > easier > > > > >> > > > > in > > > > >> > > > > > > the > > > > >> > > > > > > > >> > future, > > > > >> > > > > > > > >> > > > > > >> simplify > > > > >> > > > > > > > >> > > > > > >> > any > > > > >> > > > > > > > >> > > > > > >> > > > > > future > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > implementations of > > > > backups > > > > >> > and > > > > >> > > > > > > restores, > > > > >> > > > > > > > >> and > > > > >> > > > > > > > >> > > more. > > > > >> > > > > > > > >> > > > > In > > > > >> > > > > > > > >> > > > > > >> > > general, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> unique IDs > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > would ensure that > > the > > > > >> source > > > > >> > > > > cluster > > > > >> > > > > > > > >> topics do > > > > >> > > > > > > > >> > > not > > > > >> > > > > > > > >> > > > > > >> conflict > > > > >> > > > > > > > >> > > > > > >> > > > with > > > > >> > > > > > > > >> > > > > > >> > > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > destination > cluster > > > > >> topics. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > If we were to use > > > > sequence > > > > >> > ids, > > > > >> > > > we > > > > >> > > > > > > would > > > > >> > > > > > > > >> need > > > > >> > > > > > > > >> > > > > > >> sufficiently > > > > >> > > > > > > > >> > > > > > >> > > > large > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> cluster > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > ids to be stored > > with > > > > the > > > > >> > topic > > > > >> > > > > > > > >> identifiers or > > > > >> > > > > > > > >> > > we > > > > >> > > > > > > > >> > > > > run > > > > >> > > > > > > > >> > > > > > >> the > > > > >> > > > > > > > >> > > > > > >> > > risk > > > > >> > > > > > > > >> > > > > > >> > > > of > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > collisions. This > > will > > > > >> give up > > > > >> > > any > > > > >> > > > > > > > >> advantage in > > > > >> > > > > > > > >> > > > > > >> compactness > > > > >> > > > > > > > >> > > > > > >> > > that > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> sequence > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > numbers may bring. > > > Given > > > > >> > these > > > > >> > > > > > > > advantages I > > > > >> > > > > > > > >> > > think > > > > >> > > > > > > > >> > > > it > > > > >> > > > > > > > >> > > > > > >> makes > > > > >> > > > > > > > >> > > > > > >> > > > sense > > > > >> > > > > > > > >> > > > > > >> > > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> use > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > UUIDs. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Gokul: This is an > > > > >> interesting > > > > >> > > > idea, > > > > >> > > > > > but > > > > >> > > > > > > > >> this > > > > >> > > > > > > > >> > is > > > > >> > > > > > > > >> > > a > > > > >> > > > > > > > >> > > > > > >> breaking > > > > >> > > > > > > > >> > > > > > >> > > > > change. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Out of > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > scope for now, but > > > maybe > > > > >> > worth > > > > >> > > > > > > discussing > > > > >> > > > > > > > >> in > > > > >> > > > > > > > >> > the > > > > >> > > > > > > > >> > > > > > future. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Hope this explains > > > some > > > > of > > > > >> > the > > > > >> > > > > > > decisions, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > Justine > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > On Fri, Sep 11, > 2020 > > > at > > > > >> 8:27 > > > > >> > AM > > > > >> > > > > Gokul > > > > >> > > > > > > > >> Ramanan > > > > >> > > > > > > > >> > > > > > >> Subramanian < > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > gokul24...@gmail.com> > > > > >> wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Hi. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks for the > > KIP. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Have you thought > > > about > > > > >> > > whether > > > > >> > > > it > > > > >> > > > > > > makes > > > > >> > > > > > > > >> > sense > > > > >> > > > > > > > >> > > to > > > > >> > > > > > > > >> > > > > > >> support > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> authorizing a > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > principal for a > > > topic > > > > ID > > > > >> > > rather > > > > >> > > > > > than > > > > >> > > > > > > a > > > > >> > > > > > > > >> topic > > > > >> > > > > > > > >> > > > name > > > > >> > > > > > > > >> > > > > to > > > > >> > > > > > > > >> > > > > > >> > > achieve > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> tighter > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > security? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Or is the topic > ID > > > > >> > > > fundamentally > > > > >> > > > > an > > > > >> > > > > > > > >> internal > > > > >> > > > > > > > >> > > > > detail > > > > >> > > > > > > > >> > > > > > >> > similar > > > > >> > > > > > > > >> > > > > > >> > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> epochs > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > used > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in a bunch of > > other > > > > >> places > > > > >> > in > > > > >> > > > > > Kafka? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Thanks. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > On Fri, Sep 11, > > 2020 > > > > at > > > > >> > 4:06 > > > > >> > > PM > > > > >> > > > > > John > > > > >> > > > > > > > >> > Roesler < > > > > >> > > > > > > > >> > > > > > >> > > > > > vvcep...@apache.org> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Hello Justine, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks for the > > > KIP! > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > I happen to > have > > > > been > > > > >> > > > > confronted > > > > >> > > > > > > > >> recently > > > > >> > > > > > > > >> > > with > > > > >> > > > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> need > > > > >> > > > > > > > >> > > > > > >> > > to > > > > >> > > > > > > > >> > > > > > >> > > > > keep > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > track of > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > a > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > large number > of > > > > >> topics as > > > > >> > > > > > compactly > > > > >> > > > > > > > as > > > > >> > > > > > > > >> > > > > possible. I > > > > >> > > > > > > > >> > > > > > >> was > > > > >> > > > > > > > >> > > > > > >> > > > going > > > > >> > > > > > > > >> > > > > > >> > > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> come > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > up > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > with some way > to > > > > >> > dictionary > > > > >> > > > > > encode > > > > >> > > > > > > > the > > > > >> > > > > > > > >> > topic > > > > >> > > > > > > > >> > > > > names > > > > >> > > > > > > > >> > > > > > >> as > > > > >> > > > > > > > >> > > > > > >> > > > > integers, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> but > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > seems much > > better! > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Apologies if > > this > > > > has > > > > >> > been > > > > >> > > > > raised > > > > >> > > > > > > > >> before, > > > > >> > > > > > > > >> > > but > > > > >> > > > > > > > >> > > > > I’m > > > > >> > > > > > > > >> > > > > > >> > > wondering > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> about the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > choice of UUID > > vs > > > > >> > sequence > > > > >> > > > > number > > > > >> > > > > > > for > > > > >> > > > > > > > >> the > > > > >> > > > > > > > >> > > ids. > > > > >> > > > > > > > >> > > > > > >> > Typically, > > > > >> > > > > > > > >> > > > > > >> > > > > I’ve > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seen > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUIDs > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > in two > > situations: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 1. When > > processes > > > > >> need to > > > > >> > > > > > generate > > > > >> > > > > > > > >> > > > non-colliding > > > > >> > > > > > > > >> > > > > > >> > > > identifiers > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> without > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > coordination. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2. When the > > > > identifier > > > > >> > > needs > > > > >> > > > to > > > > >> > > > > > be > > > > >> > > > > > > > >> > > > “universally > > > > >> > > > > > > > >> > > > > > >> > unique”; > > > > >> > > > > > > > >> > > > > > >> > > > > I.e., > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > identifier > needs > > > to > > > > >> > > > distinguish > > > > >> > > > > > the > > > > >> > > > > > > > >> entity > > > > >> > > > > > > > >> > > > from > > > > >> > > > > > > > >> > > > > > all > > > > >> > > > > > > > >> > > > > > >> > other > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> entities > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > could ever > > exist. > > > > >> This is > > > > >> > > > > useful > > > > >> > > > > > in > > > > >> > > > > > > > >> cases > > > > >> > > > > > > > >> > > > where > > > > >> > > > > > > > >> > > > > > >> > entities > > > > >> > > > > > > > >> > > > > > >> > > > from > > > > >> > > > > > > > >> > > > > > >> > > > > > all > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > kinds > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > of > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > systems get > > mixed > > > > >> > together, > > > > >> > > > > such > > > > >> > > > > > as > > > > >> > > > > > > > >> when > > > > >> > > > > > > > >> > > > dumping > > > > >> > > > > > > > >> > > > > > >> logs > > > > >> > > > > > > > >> > > > > > >> > > from > > > > >> > > > > > > > >> > > > > > >> > > > > all > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > processes > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > in > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a company > into a > > > > >> common > > > > >> > > > system. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Maybe I’m > being > > > > >> > > > short-sighted, > > > > >> > > > > > but > > > > >> > > > > > > it > > > > >> > > > > > > > >> > > doesn’t > > > > >> > > > > > > > >> > > > > seem > > > > >> > > > > > > > >> > > > > > >> like > > > > >> > > > > > > > >> > > > > > >> > > > > either > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> really > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > applies here. > It > > > > seems > > > > >> > like > > > > >> > > > the > > > > >> > > > > > > > brokers > > > > >> > > > > > > > >> > > could > > > > >> > > > > > > > >> > > > > and > > > > >> > > > > > > > >> > > > > > >> would > > > > >> > > > > > > > >> > > > > > >> > > > > achieve > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > consensus > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when creating > a > > > > topic > > > > >> > > anyway, > > > > >> > > > > > which > > > > >> > > > > > > > is > > > > >> > > > > > > > >> all > > > > >> > > > > > > > >> > > > > that’s > > > > >> > > > > > > > >> > > > > > >> > > required > > > > >> > > > > > > > >> > > > > > >> > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > generate > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > non-colliding > > > > sequence > > > > >> > ids. > > > > >> > > > For > > > > >> > > > > > the > > > > >> > > > > > > > >> > second, > > > > >> > > > > > > > >> > > as > > > > >> > > > > > > > >> > > > > you > > > > >> > > > > > > > >> > > > > > >> > > mention, > > > > >> > > > > > > > >> > > > > > >> > > > > we > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> could > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > assign > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a UUID to the > > > > cluster > > > > >> as > > > > >> > a > > > > >> > > > > whole, > > > > >> > > > > > > > which > > > > >> > > > > > > > >> > > would > > > > >> > > > > > > > >> > > > > > render > > > > >> > > > > > > > >> > > > > > >> > any > > > > >> > > > > > > > >> > > > > > >> > > > > > resource > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > scoped > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the broker > > > > universally > > > > >> > > unique > > > > >> > > > > as > > > > >> > > > > > > > well. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > The reason I > > > mention > > > > >> this > > > > >> > > is > > > > >> > > > > > that, > > > > >> > > > > > > > >> > although > > > > >> > > > > > > > >> > > a > > > > >> > > > > > > > >> > > > > UUID > > > > >> > > > > > > > >> > > > > > >> is > > > > >> > > > > > > > >> > > > > > >> > way > > > > >> > > > > > > > >> > > > > > >> > > > > more > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > compact > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > than topic > > names, > > > > it’s > > > > >> > > still > > > > >> > > > 16 > > > > >> > > > > > > > bytes. > > > > >> > > > > > > > >> In > > > > >> > > > > > > > >> > > > > > contrast, > > > > >> > > > > > > > >> > > > > > >> a > > > > >> > > > > > > > >> > > > > > >> > > > 4-byte > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> integer > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > sequence id > > would > > > > give > > > > >> > us 4 > > > > >> > > > > > billion > > > > >> > > > > > > > >> unique > > > > >> > > > > > > > >> > > > > topics > > > > >> > > > > > > > >> > > > > > >> per > > > > >> > > > > > > > >> > > > > > >> > > > > cluster, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > like enough ;) > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Considering > the > > > > >> number of > > > > >> > > > > > different > > > > >> > > > > > > > >> times > > > > >> > > > > > > > >> > > > these > > > > >> > > > > > > > >> > > > > > >> topic > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> identifiers are > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > sent > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > over the wire > or > > > > >> stored > > > > >> > in > > > > >> > > > > > memory, > > > > >> > > > > > > it > > > > >> > > > > > > > >> > seems > > > > >> > > > > > > > >> > > > like > > > > >> > > > > > > > >> > > > > > it > > > > >> > > > > > > > >> > > > > > >> > might > > > > >> > > > > > > > >> > > > > > >> > > > be > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> worth > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > additional 4x > > > space > > > > >> > > savings. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > What do you > > think > > > > >> about > > > > >> > > this? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > Thanks, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > John > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > On Fri, Sep > 11, > > > > 2020, > > > > >> at > > > > >> > > > 03:20, > > > > >> > > > > > Tom > > > > >> > > > > > > > >> > Bentley > > > > >> > > > > > > > >> > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Hi Justine, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > This looks > > like > > > a > > > > >> very > > > > >> > > > > welcome > > > > >> > > > > > > > >> > > improvement. > > > > >> > > > > > > > >> > > > > > >> Thanks! > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Maybe I > missed > > > it, > > > > >> but > > > > >> > > the > > > > >> > > > > KIP > > > > >> > > > > > > > >> doesn't > > > > >> > > > > > > > >> > > seem > > > > >> > > > > > > > >> > > > to > > > > >> > > > > > > > >> > > > > > >> > mention > > > > >> > > > > > > > >> > > > > > >> > > > > > changing > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > DeleteTopicsRequest > > > > >> to > > > > >> > > > > identify > > > > >> > > > > > > the > > > > >> > > > > > > > >> > topic > > > > >> > > > > > > > >> > > > > using > > > > >> > > > > > > > >> > > > > > an > > > > >> > > > > > > > >> > > > > > >> > id. > > > > >> > > > > > > > >> > > > > > >> > > > > Maybe > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > that's out > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > scope, but > > > > >> > > > > DeleteTopicsRequest > > > > >> > > > > > is > > > > >> > > > > > > > not > > > > >> > > > > > > > >> > > listed > > > > >> > > > > > > > >> > > > > > among > > > > >> > > > > > > > >> > > > > > >> > the > > > > >> > > > > > > > >> > > > > > >> > > > > Future > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Work > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > APIs > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > either. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Kind > regards, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > Tom > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > On Thu, Sep > > 10, > > > > >> 2020 at > > > > >> > > > 3:59 > > > > >> > > > > PM > > > > >> > > > > > > > >> Satish > > > > >> > > > > > > > >> > > > > Duggana < > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >> satish.dugg...@gmail.com > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Thanks > > > > >> Lucas/Justine > > > > >> > > for > > > > >> > > > > the > > > > >> > > > > > > nice > > > > >> > > > > > > > >> KIP. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > It has > > several > > > > >> > benefits > > > > >> > > > > which > > > > >> > > > > > > > also > > > > >> > > > > > > > >> > > include > > > > >> > > > > > > > >> > > > > > >> > > simplifying > > > > >> > > > > > > > >> > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> topic > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion > > > process > > > > >> by > > > > >> > > > > > controller > > > > >> > > > > > > > and > > > > >> > > > > > > > >> > logs > > > > >> > > > > > > > >> > > > > > cleanup > > > > >> > > > > > > > >> > > > > > >> by > > > > >> > > > > > > > >> > > > > > >> > > > > brokers > > > > >> > > > > > > > >> > > > > > >> > > > > > in > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > corner > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Best, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Satish. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > On Wed, > Sep > > 9, > > > > >> 2020 > > > > >> > at > > > > >> > > > > 10:07 > > > > >> > > > > > PM > > > > >> > > > > > > > >> > Justine > > > > >> > > > > > > > >> > > > > > Olshan < > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > jols...@confluent.io > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Hello > all, > > > > it's > > > > >> > been > > > > >> > > > > > almost a > > > > >> > > > > > > > >> year! > > > > >> > > > > > > > >> > > I've > > > > >> > > > > > > > >> > > > > > made > > > > >> > > > > > > > >> > > > > > >> > some > > > > >> > > > > > > > >> > > > > > >> > > > > > changes > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > KIP > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > and hope > to > > > > >> continue > > > > >> > > the > > > > >> > > > > > > > >> discussion. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > One of > the > > > > main > > > > >> > > changes > > > > >> > > > > > I've > > > > >> > > > > > > > >> added > > > > >> > > > > > > > >> > is > > > > >> > > > > > > > >> > > > now > > > > >> > > > > > > > >> > > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > metadata > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> response > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > include > the > > > > topic > > > > >> ID > > > > >> > > (as > > > > >> > > > > > Colin > > > > >> > > > > > > > >> > > suggested). > > > > >> > > > > > > > >> > > > > > >> Clients > > > > >> > > > > > > > >> > > > > > >> > > can > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> obtain the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > topicID > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of a given > > > topic > > > > >> > > through > > > > >> > > > a > > > > >> > > > > > > > >> > > > TopicDescription. > > > > >> > > > > > > > >> > > > > > The > > > > >> > > > > > > > >> > > > > > >> > > > topicId > > > > >> > > > > > > > >> > > > > > >> > > > > > will > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > also be > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > included > > with > > > > the > > > > >> > > > > > > UpdateMetadata > > > > >> > > > > > > > >> > > request. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Let me > > know > > > > what > > > > >> > you > > > > >> > > > all > > > > >> > > > > > > think. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Thank > you, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > Justine > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > On > > > 2019/09/13 > > > > >> > > 16:38:26, > > > > >> > > > > > > "Colin > > > > >> > > > > > > > >> > > McCabe" < > > > > >> > > > > > > > >> > > > > > >> > > > > > cmcc...@apache.org > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Hi > > Lucas, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Thanks > > for > > > > >> > tackling > > > > >> > > > > this. > > > > >> > > > > > > > >> Topic > > > > >> > > > > > > > >> > IDs > > > > >> > > > > > > > >> > > > > are a > > > > >> > > > > > > > >> > > > > > >> > great > > > > >> > > > > > > > >> > > > > > >> > > > > idea, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> and > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > this > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > is > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > really > good > > > > >> writeup. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > For > > > > >> > > > > > > /brokers/topics/[topic], > > > > >> > > > > > > > >> the > > > > >> > > > > > > > >> > > > schema > > > > >> > > > > > > > >> > > > > > >> version > > > > >> > > > > > > > >> > > > > > >> > > > > should > > > > >> > > > > > > > >> > > > > > >> > > > > > be > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > bumped > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > version 3, > > > > rather > > > > >> > than > > > > >> > > 2. > > > > >> > > > > > > > KIP-455 > > > > >> > > > > > > > >> > > bumped > > > > >> > > > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> > version > > > > >> > > > > > > > >> > > > > > >> > > > of > > > > >> > > > > > > > >> > > > > > >> > > > > > this > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > znode > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > 2 > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > already :) > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Given > > that > > > > >> we're > > > > >> > > > going > > > > >> > > > > to > > > > >> > > > > > > be > > > > >> > > > > > > > >> > seeing > > > > >> > > > > > > > >> > > > > these > > > > >> > > > > > > > >> > > > > > >> > things > > > > >> > > > > > > > >> > > > > > >> > > as > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> strings > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > as > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > lot > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > (in > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > logs, in > > > > >> ZooKeeper, > > > > >> > on > > > > >> > > > the > > > > >> > > > > > > > >> > command-line, > > > > >> > > > > > > > >> > > > > > etc.), > > > > >> > > > > > > > >> > > > > > >> > does > > > > >> > > > > > > > >> > > > > > >> > > it > > > > >> > > > > > > > >> > > > > > >> > > > > > make > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > sense to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > use > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > base64 > when > > > > >> > converting > > > > >> > > > them > > > > >> > > > > > to > > > > >> > > > > > > > >> > strings? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Here > is > > an > > > > >> > example > > > > >> > > of > > > > >> > > > > the > > > > >> > > > > > > hex > > > > >> > > > > > > > >> > > > > > >> representation: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > >> > > > > > > > >> > 6fcb514b-b878-4c9d-95b7-8dc3a7ce6fd8 > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > And > here > > > is > > > > an > > > > >> > > > example > > > > >> > > > > in > > > > >> > > > > > > > >> base64. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > >> > > > b8tRS7h4TJ2Vt43Dp85v2A > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > The > > base64 > > > > >> > version > > > > >> > > > > saves > > > > >> > > > > > 15 > > > > >> > > > > > > > >> > letters > > > > >> > > > > > > > >> > > > (to > > > > >> > > > > > > > >> > > > > be > > > > >> > > > > > > > >> > > > > > >> > fair, > > > > >> > > > > > > > >> > > > > > >> > > 4 > > > > >> > > > > > > > >> > > > > > >> > > > of > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> those > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > were > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > dashes > that > > we > > > > >> could > > > > >> > > have > > > > >> > > > > > > elided > > > > >> > > > > > > > in > > > > >> > > > > > > > >> > the > > > > >> > > > > > > > >> > > > hex > > > > >> > > > > > > > >> > > > > > >> > > > > > representation.) > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > Another > > > > thing > > > > >> to > > > > >> > > > > consider > > > > >> > > > > > > is > > > > >> > > > > > > > >> that > > > > >> > > > > > > > >> > we > > > > >> > > > > > > > >> > > > > > should > > > > >> > > > > > > > >> > > > > > >> > > specify > > > > >> > > > > > > > >> > > > > > >> > > > > > that > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > all-zeroes > > > UUID > > > > is > > > > >> > not > > > > >> > > a > > > > >> > > > > > valid > > > > >> > > > > > > > >> topic > > > > >> > > > > > > > >> > > UUID. > > > > >> > > > > > > > >> > > > > > We > > > > >> > > > > > > > >> > > > > > >> > can't > > > > >> > > > > > > > >> > > > > > >> > > > use > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> null > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > for > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > this > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > because we > > > can't > > > > >> > pass a > > > > >> > > > > null > > > > >> > > > > > > UUID > > > > >> > > > > > > > >> over > > > > >> > > > > > > > >> > > the > > > > >> > > > > > > > >> > > > > RPC > > > > >> > > > > > > > >> > > > > > >> > > protocol > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> (there > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is no > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > special > > > pattern > > > > >> for > > > > >> > > null, > > > > >> > > > > nor > > > > >> > > > > > > do > > > > >> > > > > > > > we > > > > >> > > > > > > > >> > want > > > > >> > > > > > > > >> > > > to > > > > >> > > > > > > > >> > > > > > >> waste > > > > >> > > > > > > > >> > > > > > >> > > space > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> reserving > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > such > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > a > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > pattern.) > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Maybe > I > > > > missed > > > > >> > it, > > > > >> > > > but > > > > >> > > > > > did > > > > >> > > > > > > > you > > > > >> > > > > > > > >> > > > describe > > > > >> > > > > > > > >> > > > > > >> > > "migration > > > > >> > > > > > > > >> > > > > > >> > > > > > of... > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > existing > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > topic[s] > > > without > > > > >> > topic > > > > >> > > > IDs" > > > > >> > > > > > in > > > > >> > > > > > > > >> detail > > > > >> > > > > > > > >> > in > > > > >> > > > > > > > >> > > > any > > > > >> > > > > > > > >> > > > > > >> > section? > > > > >> > > > > > > > >> > > > > > >> > > > It > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> seems > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > like > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > when > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the new > > > > controller > > > > >> > > > becomes > > > > >> > > > > > > > active, > > > > >> > > > > > > > >> it > > > > >> > > > > > > > >> > > > should > > > > >> > > > > > > > >> > > > > > >> just > > > > >> > > > > > > > >> > > > > > >> > > > > generate > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> random > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > UUIDs for > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > these, and > > > write > > > > >> the > > > > >> > > > random > > > > >> > > > > > > UUIDs > > > > >> > > > > > > > >> back > > > > >> > > > > > > > >> > > to > > > > >> > > > > > > > >> > > > > > >> > ZooKeeper. > > > > >> > > > > > > > >> > > > > > >> > > > It > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> would be > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > good > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > spell that > > > out. > > > > >> We > > > > >> > > > should > > > > >> > > > > > make > > > > >> > > > > > > > it > > > > >> > > > > > > > >> > clear > > > > >> > > > > > > > >> > > > > that > > > > >> > > > > > > > >> > > > > > >> this > > > > >> > > > > > > > >> > > > > > >> > > > > happens > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > regardless > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > of > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > the > > > inter-broker > > > > >> > > protocol > > > > >> > > > > > > version > > > > >> > > > > > > > >> > (it's > > > > >> > > > > > > > >> > > a > > > > >> > > > > > > > >> > > > > > >> > compatible > > > > >> > > > > > > > >> > > > > > >> > > > > > change). > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > >> > > "LeaderAndIsrRequests > > > > >> > > > > > > > >> including an > > > > >> > > > > > > > >> > > > > > >> > > > is_every_partition > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> flag" > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > seems a > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > bit wordy. > > > Can > > > > we > > > > >> > just > > > > >> > > > > call > > > > >> > > > > > > > these > > > > >> > > > > > > > >> > "full > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> LeaderAndIsrRequests"? > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > Then > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field > > > could > > > > be > > > > >> > > named > > > > >> > > > > > > "full". > > > > >> > > > > > > > >> > Also, > > > > >> > > > > > > > >> > > it > > > > >> > > > > > > > >> > > > > > would > > > > >> > > > > > > > >> > > > > > >> > > > probably > > > > >> > > > > > > > >> > > > > > >> > > > > > be > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > better > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > for the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > RPC field > to > > > be > > > > an > > > > >> > enum > > > > >> > > > of > > > > >> > > > > { > > > > >> > > > > > > > >> > > UNSPECIFIED, > > > > >> > > > > > > > >> > > > > > >> > > INCREMENTAL, > > > > >> > > > > > > > >> > > > > > >> > > > > FULL > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> }, so > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > that > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > we > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > can > cleanly > > > > handle > > > > >> > old > > > > >> > > > > > versions > > > > >> > > > > > > > (by > > > > >> > > > > > > > >> > > > treating > > > > >> > > > > > > > >> > > > > > >> them > > > > >> > > > > > > > >> > > > > > >> > as > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UNSPECIFIED) > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > In the > > > > >> > > > > > LeaderAndIsrRequest > > > > >> > > > > > > > >> > section, > > > > >> > > > > > > > >> > > > you > > > > >> > > > > > > > >> > > > > > >> write > > > > >> > > > > > > > >> > > > > > >> > "A > > > > >> > > > > > > > >> > > > > > >> > > > > final > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > deletion > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > event > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will be > > > > secheduled > > > > >> > for > > > > >> > > X > > > > >> > > > ms > > > > >> > > > > > > after > > > > >> > > > > > > > >> the > > > > >> > > > > > > > >> > > > > > >> > > > LeaderAndIsrRequest > > > > >> > > > > > > > >> > > > > > >> > > > > > was > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > first > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > received..." > > > I > > > > >> guess > > > > >> > > > the X > > > > >> > > > > > > was a > > > > >> > > > > > > > >> > > > > placeholder > > > > >> > > > > > > > >> > > > > > >> that > > > > >> > > > > > > > >> > > > > > >> > > you > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> intended > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > replace > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > before > > > posting? > > > > :) > > > > >> > In > > > > >> > > > any > > > > >> > > > > > > case, > > > > >> > > > > > > > >> this > > > > >> > > > > > > > >> > > > seems > > > > >> > > > > > > > >> > > > > > like > > > > >> > > > > > > > >> > > > > > >> > the > > > > >> > > > > > > > >> > > > > > >> > > > kind > > > > >> > > > > > > > >> > > > > > >> > > > > > of > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > thing > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > we'd > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > want a > > > > >> configuration > > > > >> > > for. > > > > >> > > > > > > Let's > > > > >> > > > > > > > >> > > describe > > > > >> > > > > > > > >> > > > > that > > > > >> > > > > > > > >> > > > > > >> > > > > > configuration > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> key > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > somewhere > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in this > KIP, > > > > >> > including > > > > >> > > > what > > > > >> > > > > > its > > > > >> > > > > > > > >> > default > > > > >> > > > > > > > >> > > > > value > > > > >> > > > > > > > >> > > > > > >> is. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > We > > should > > > > >> > probably > > > > >> > > > also > > > > >> > > > > > > log a > > > > >> > > > > > > > >> > bunch > > > > >> > > > > > > > >> > > of > > > > >> > > > > > > > >> > > > > > >> messages > > > > >> > > > > > > > >> > > > > > >> > > at > > > > >> > > > > > > > >> > > > > > >> > > > > WARN > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> level > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > when > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > something > is > > > > >> > scheduled > > > > >> > > > for > > > > >> > > > > > > > >> deletion, > > > > >> > > > > > > > >> > as > > > > >> > > > > > > > >> > > > > well. > > > > >> > > > > > > > >> > > > > > >> > (Maybe > > > > >> > > > > > > > >> > > > > > >> > > > > this > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> was > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > assumed, but > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > it would > be > > > good > > > > >> to > > > > >> > > > mention > > > > >> > > > > > > it). > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > I feel > > > like > > > > >> there > > > > >> > > > are a > > > > >> > > > > > few > > > > >> > > > > > > > >> > sections > > > > >> > > > > > > > >> > > > > that > > > > >> > > > > > > > >> > > > > > >> > should > > > > >> > > > > > > > >> > > > > > >> > > be > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> moved to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "rejected > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > alternatives." > > > > >> For > > > > >> > > > > example, > > > > >> > > > > > in > > > > >> > > > > > > > the > > > > >> > > > > > > > >> > > > > > DeleteTopics > > > > >> > > > > > > > >> > > > > > >> > > > section, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> since > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > we're > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > not > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > going to > do > > > > >> option 1 > > > > >> > or > > > > >> > > > 2, > > > > >> > > > > > > these > > > > >> > > > > > > > >> > should > > > > >> > > > > > > > >> > > be > > > > >> > > > > > > > >> > > > > > moved > > > > >> > > > > > > > >> > > > > > >> > into > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> "rejected > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > alternatives," > > > > >> > rather > > > > >> > > > than > > > > >> > > > > > > > >> appearing > > > > >> > > > > > > > >> > > > > inline. > > > > >> > > > > > > > >> > > > > > >> > > Another > > > > >> > > > > > > > >> > > > > > >> > > > > case > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> is > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > "Should > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > we remove > > > topic > > > > >> name > > > > >> > > from > > > > >> > > > > the > > > > >> > > > > > > > >> protocol > > > > >> > > > > > > > >> > > > where > > > > >> > > > > > > > >> > > > > > >> > > possible" > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> section. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > This > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > is > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > clearly > > > > >> discussing a > > > > >> > > > design > > > > >> > > > > > > > >> > alternative > > > > >> > > > > > > > >> > > > that > > > > >> > > > > > > > >> > > > > > >> we're > > > > >> > > > > > > > >> > > > > > >> > > not > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> proposing > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > implement: > > > > >> removing > > > > >> > the > > > > >> > > > > topic > > > > >> > > > > > > > name > > > > >> > > > > > > > >> > from > > > > >> > > > > > > > >> > > > > those > > > > >> > > > > > > > >> > > > > > >> > > > protocols. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Is it > > > really > > > > >> > > > necessary > > > > >> > > > > to > > > > >> > > > > > > > have > > > > >> > > > > > > > >> a > > > > >> > > > > > > > >> > new > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > /admin/delete_topics_by_id > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > path > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > in > > ZooKeeper? > > > > It > > > > >> > seems > > > > >> > > > > like > > > > >> > > > > > we > > > > >> > > > > > > > >> don't > > > > >> > > > > > > > >> > > > really > > > > >> > > > > > > > >> > > > > > >> need > > > > >> > > > > > > > >> > > > > > >> > > this. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Whenever > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > there is > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > a new > > > > controller, > > > > >> > we'll > > > > >> > > > > send > > > > >> > > > > > > out > > > > >> > > > > > > > >> full > > > > >> > > > > > > > >> > > > > > >> > > > > LeaderAndIsrRequests > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> which > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > will > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > trigger > the > > > > stale > > > > >> > > topics > > > > >> > > > to > > > > >> > > > > > be > > > > >> > > > > > > > >> cleaned > > > > >> > > > > > > > >> > > up. > > > > >> > > > > > > > >> > > > > > The > > > > >> > > > > > > > >> > > > > > >> > > active > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > controller > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > will > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > also send > > the > > > > full > > > > >> > > > > > > > >> LeaderAndIsrRequest > > > > >> > > > > > > > >> > > to > > > > >> > > > > > > > >> > > > > > >> brokers > > > > >> > > > > > > > >> > > > > > >> > > that > > > > >> > > > > > > > >> > > > > > >> > > > > are > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> just > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > starting > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > up. So > we > > > > don't > > > > >> > > really > > > > >> > > > > > need > > > > >> > > > > > > > this > > > > >> > > > > > > > >> > kind > > > > >> > > > > > > > >> > > > of > > > > >> > > > > > > > >> > > > > > >> > two-phase > > > > >> > > > > > > > >> > > > > > >> > > > > > commit > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > (send > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > out > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > >> StopReplicasRequest, > > > > >> > > get > > > > >> > > > > ACKs > > > > >> > > > > > > > from > > > > >> > > > > > > > >> all > > > > >> > > > > > > > >> > > > > nodes, > > > > >> > > > > > > > >> > > > > > >> > commit > > > > >> > > > > > > > >> > > > > > >> > > by > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> removing > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > >> /admin/delete_topics > > > > >> > > > node) > > > > >> > > > > > any > > > > >> > > > > > > > >> more. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > You > > > mention > > > > >> that > > > > >> > > > > > > FetchRequest > > > > >> > > > > > > > >> will > > > > >> > > > > > > > >> > > now > > > > >> > > > > > > > >> > > > > > >> include > > > > >> > > > > > > > >> > > > > > >> > > UUID > > > > >> > > > > > > > >> > > > > > >> > > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> avoid > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > issues > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > where > > requests > > > > are > > > > >> > made > > > > >> > > > to > > > > >> > > > > > > stale > > > > >> > > > > > > > >> > > > partitions. > > > > >> > > > > > > > >> > > > > > >> > > However, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> adding a > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > UUID > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > MetadataRequest > > > > is > > > > >> > > listed > > > > >> > > > > as > > > > >> > > > > > > > future > > > > >> > > > > > > > >> > > work, > > > > >> > > > > > > > >> > > > > out > > > > >> > > > > > > > >> > > > > > of > > > > >> > > > > > > > >> > > > > > >> > > scope > > > > >> > > > > > > > >> > > > > > >> > > > > for > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > KIP. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > How > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > will the > > > client > > > > >> learn > > > > >> > > > what > > > > >> > > > > > the > > > > >> > > > > > > > >> topic > > > > >> > > > > > > > >> > > UUID > > > > >> > > > > > > > >> > > > > is, > > > > >> > > > > > > > >> > > > > > if > > > > >> > > > > > > > >> > > > > > >> > the > > > > >> > > > > > > > >> > > > > > >> > > > > > metadata > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > response > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > doesn't > > > include > > > > >> that > > > > >> > > > > > > information? > > > > >> > > > > > > > >> It > > > > >> > > > > > > > >> > > > seems > > > > >> > > > > > > > >> > > > > > like > > > > >> > > > > > > > >> > > > > > >> > > adding > > > > >> > > > > > > > >> > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> UUID > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > MetadataResponse > > > > >> > would > > > > >> > > be > > > > >> > > > > an > > > > >> > > > > > > > >> > improvement > > > > >> > > > > > > > >> > > > > here > > > > >> > > > > > > > >> > > > > > >> that > > > > >> > > > > > > > >> > > > > > >> > > > might > > > > >> > > > > > > > >> > > > > > >> > > > > > not > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> be > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > too > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > hard to > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > make. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > best, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > Colin > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > On > Mon, > > > Sep > > > > 9, > > > > >> > > 2019, > > > > >> > > > at > > > > >> > > > > > > > 17:48, > > > > >> > > > > > > > >> > > Ryanne > > > > >> > > > > > > > >> > > > > > Dolan > > > > >> > > > > > > > >> > > > > > >> > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Lucas, > > > > this > > > > >> > would > > > > >> > > > be > > > > >> > > > > > > great. > > > > >> > > > > > > > >> I've > > > > >> > > > > > > > >> > > run > > > > >> > > > > > > > >> > > > > > into > > > > >> > > > > > > > >> > > > > > >> > > issues > > > > >> > > > > > > > >> > > > > > >> > > > > with > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > topics > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > being > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > resurrected > > > > >> > > > > > accidentally, > > > > >> > > > > > > > >> since > > > > >> > > > > > > > >> > a > > > > >> > > > > > > > >> > > > > client > > > > >> > > > > > > > >> > > > > > >> > cannot > > > > >> > > > > > > > >> > > > > > >> > > > > > easily > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > distinguish > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > between > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > a > > > deleted > > > > >> topic > > > > >> > > > and a > > > > >> > > > > > new > > > > >> > > > > > > > >> topic > > > > >> > > > > > > > >> > > with > > > > >> > > > > > > > >> > > > > the > > > > >> > > > > > > > >> > > > > > >> same > > > > >> > > > > > > > >> > > > > > >> > > > name. > > > > >> > > > > > > > >> > > > > > >> > > > > > I'd > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > need > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > ID > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > accessible > > > > >> from > > > > >> > > the > > > > >> > > > > > > client > > > > >> > > > > > > > to > > > > >> > > > > > > > >> > > solve > > > > >> > > > > > > > >> > > > > that > > > > >> > > > > > > > >> > > > > > >> > issue, > > > > >> > > > > > > > >> > > > > > >> > > > but > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> this > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > is a > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > good > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > first > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > step. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Ryanne > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > On > > Wed, > > > > Sep > > > > >> 4, > > > > >> > > 2019 > > > > >> > > > > at > > > > >> > > > > > > 1:41 > > > > >> > > > > > > > >> PM > > > > >> > > > > > > > >> > > Lucas > > > > >> > > > > > > > >> > > > > > >> > > Bradstreet < > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > lu...@confluent.io> > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > wrote: > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > Hi > > > all, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > I > > > would > > > > >> like > > > > >> > to > > > > >> > > > > kick > > > > >> > > > > > > off > > > > >> > > > > > > > >> > > > discussion > > > > >> > > > > > > > >> > > > > of > > > > >> > > > > > > > >> > > > > > >> > > KIP-516, > > > > >> > > > > > > > >> > > > > > >> > > > > an > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > implementation > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > of topic > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > IDs > > > for > > > > >> > Kafka. > > > > >> > > > > Topic > > > > >> > > > > > > IDs > > > > >> > > > > > > > >> aim > > > > >> > > > > > > > >> > to > > > > >> > > > > > > > >> > > > > solve > > > > >> > > > > > > > >> > > > > > >> topic > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> uniqueness > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > problems in > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > Kafka, > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > where > > > > >> > referring > > > > >> > > > to > > > > >> > > > > a > > > > >> > > > > > > > topic > > > > >> > > > > > > > >> by > > > > >> > > > > > > > >> > > name > > > > >> > > > > > > > >> > > > > > >> alone is > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> insufficient. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > Such > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > cases > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > include > > > > >> when > > > > >> > a > > > > >> > > > > topic > > > > >> > > > > > > has > > > > >> > > > > > > > >> been > > > > >> > > > > > > > >> > > > > deleted > > > > >> > > > > > > > >> > > > > > >> and > > > > >> > > > > > > > >> > > > > > >> > > > > recreated > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> with > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > the > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > same > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > name. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > Unique > > > > >> > > > identifiers > > > > >> > > > > > will > > > > >> > > > > > > > >> help > > > > >> > > > > > > > >> > > > > simplify > > > > >> > > > > > > > >> > > > > > >> and > > > > >> > > > > > > > >> > > > > > >> > > > improve > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> Kafka's > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > topic > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > deletion > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > process, > > > > >> as > > > > >> > > well > > > > >> > > > as > > > > >> > > > > > > > prevent > > > > >> > > > > > > > >> > > cases > > > > >> > > > > > > > >> > > > > > where > > > > >> > > > > > > > >> > > > > > >> > > brokers > > > > >> > > > > > > > >> > > > > > >> > > > > may > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > incorrectly > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > interact > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > with > > > > stale > > > > >> > > > versions > > > > >> > > > > > of > > > > >> > > > > > > > >> topics. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-516%3A+Topic+Identifiers > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > Looking > > > > >> > forward > > > > >> > > > to > > > > >> > > > > > your > > > > >> > > > > > > > >> > > thoughts. > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > Lucas > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > > >> > > > > > > > >> > > > > > >> > > > > > >>> > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > >> > > > > > > > >> > > > > > >> > > > > >> > > > > > > > >> > > > > > > > > > > >> > > > > > > > >> > > > > > > > > > >> > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > > > >> > > > > > >> > > > > > > > >> > > > > >> > > > > > > > > > > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > > > > > > > > > > > >