Hi Colin, Once again, thanks a lot for the feedback! Regarding the first point about using a single version field, you are correct. Recently I understood the idea and realized that both the '__data_version__' and '__schema_version__' can be folded into a single field. This can be bumped anytime when there is a change to data or schema. I have updated the KIP now using a single field called '__version__' within the '/features' ZK node.
Please refer to this section: https://cwiki.apache.org/confluence/display/KAFKA/KIP-584%3A+Versioning+scheme+for+features#KIP-584:Versioningschemeforfeatures-Persistenceoffinalizedfeatureversions Cheers, Kowshik On Thu, Mar 26, 2020 at 7:24 PM Kowshik Prakasam <kpraka...@confluent.io> wrote: > Hi Colin, > > Thanks for the feedback! I've changed the KIP to address your suggestions. > Please find below my explanation. Here is a link to KIP 584: > https://cwiki.apache.org/confluence/display/KAFKA/KIP-584%3A+Versioning+scheme+for+features > . > > 1. '__data_version__' is the version of the finalized feature metadata > (i.e. actual ZK node contents), while the '__schema_version__' is the > version of the schema of the data persisted in ZK. These serve different > purposes. '__data_version__' is is useful mainly to clients during reads, > to differentiate between the 2 versions of eventually consistent 'finalized > features' metadata (i.e. larger metadata version is more recent). > '__schema_version__' provides an additional degree of flexibility, where if > we decide to change the schema for '/features' node in ZK (in the future), > then we can manage broker roll outs suitably (i.e. > serialization/deserialization of the ZK data can be handled safely). > > 2. Regarding admin client needing min and max information - you are right! > I've changed the KIP such that the Admin API also allows the user to read > 'supported features' from a specific broker. Please look at the section > "Admin API changes". > > 3. Regarding the use of `long` vs `Long` - it was not deliberate. I've > improved the KIP to just use `long` at all places. > > 4. Regarding kafka.admin.FeatureCommand tool - you are right! I've updated > the KIP sketching the functionality provided by this tool, with some > examples. Please look at the section "Tooling support examples". > > Thank you! > > > Cheers, > Kowshik > > On Wed, Mar 25, 2020 at 11:31 PM Colin McCabe <cmcc...@apache.org> wrote: > >> Thanks, Kowshik, this looks good. >> >> In the "Schema" section, do we really need both __schema_version__ and >> __data_version__? Can we just have a single version field here? >> >> Shouldn't the Admin(Client) function have some way to get the min and max >> information that we're exposing as well? I guess we could have min, max, >> and current. Unrelated: is the use of Long rather than long deliberate >> here? >> >> It would be good to describe how the command line tool >> kafka.admin.FeatureCommand will work. For example the flags that it will >> take and the output that it will generate to STDOUT. >> >> cheers, >> Colin >> >> >> On Tue, Mar 24, 2020, at 17:08, Kowshik Prakasam wrote: >> > Hi all, >> > >> > I've opened KIP-584 <https://issues.apache.org/jira/browse/KIP-584> >> > which >> > is intended to provide a versioning scheme for features. I'd like to use >> > this thread to discuss the same. I'd appreciate any feedback on this. >> > Here >> > is a link to KIP-584: >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-584%3A+Versioning+scheme+for+features >> > . >> > >> > Thank you! >> > >> > >> > Cheers, >> > Kowshik >> > >> >
