Thank you David for the detailed KIP. 1. Generate snapshot on upgrade > Metadata snapshot is generated and sent to the other nodes Why does the Active Controller need to generate a new snapshot and force a snapshot fetch from the replicas (inactive controller and brokers) on an upgrade? Isn't writing the FeatureLevelRecord good enough to communicate the upgrade to the replicas?
2. Generate snapshot on downgrade > Metadata snapshot is generated and sent to the other inactive controllers and > to brokers (this snapshot may be lossy!) Why do we need to send this downgraded snapshot to the brokers? The replicas have seen the FeatureLevelRecord and noticed the downgrade. Can we have the replicas each independently generate a downgraded snapshot at the offset for the downgraded FeatureLevelRecord? I assume that the active controller will guarantee that all records after the FatureLevelRecord use the downgraded version. If so, it would be good to mention that explicitly. 3. Max metadata version >For the first release that supports metadata.version, we can simply initialize >metadata.version with the current (and only) version. For future releases, we >will need a mechanism to bootstrap a particular version. This could be done >using the meta.properties file or some similar mechanism. The reason we need >the allow for a specific initial version is to support the use case of >starting a Kafka cluster at version X with an older metadata.version. I assume that the Active Controller will learn the metadata version of the broker through the BrokerRegistrationRequest. How will the Active Controller learn about the max metadata version of the inactive controller nodes? We currently don't send a registration request from the inactive controller to the active controller. Why do you need to bootstrap a particular version? Isn't the intent that the broker will learn the active metadata version by reading the metadata before unfencing? 4. Reject Registration - This is related to the bullet point above. What will be the behavior of the active controller if the broker sends a metadata version that is not compatible with the cluster wide metadata version? 5. Discover upgrade > This snapshot will be a convenient way to let broker and controller > components rebuild their entire in-memory state following an upgrade. Can we rely on the presence of the FeatureLevelRecord for the metadata version for this functionality? If so, it avoids having to reload the snapshot. 6. Metadata version specification > V4(version=4, isBackwardsCompatible=false, description="New metadata record > type Bar"), Very cool. Do you have plans to generate Apache Kafka HTML documentation for this information? Would be helpful to display this information to the user using the kafka-features.sh and feature RPC? -- -Jose
