kowshik opened a new pull request #8680: URL: https://github.com/apache/kafka/pull/8680
**TL;DR:** In this PR, I have implemented various classes and integration for the read path of the feature versioning system ([KIP-584](https://cwiki.apache.org/confluence/display/KAFKA/KIP-584%3A+Versioning+scheme+for+features)). The finalized features information is going to be stored in ZK under the node `/feature`. The read path implemented in this PR comprises broadly of the following: - A facility is provided in the broker to listen to and propagate finalized feature changes from ZK. - When new finalized features are read, feature incompatibilities are detected by comparing against supported features in the broker. - `ApiVersionsResponse` is served containing supported and finalized feature information (using the newly added tagged fields). **New IBP config value:** The feature versioning system is implemented such that it is activated only if the IBP >= `KAFKA_2_6_IV1` (newly defined value in this PR). That is because, we would like to keep the feature version system disabled (almost as if it is non-existent) until the user upgrades the cluster to or beyond IBP `KAFKA_2_6_IV1`. **New common libraries (Java):** A set of common libraries abstracting features have been introduced. Features are a map with key being `String` (feature name) and value being a `VersionRangeType` (the range of supported or finalized versions for the feature). The libraries are defined as 3 new classes in: `org.apache.kafka.common.feature.{Features, VersionRange, VersionLevelRange}`. The reason why it is kept in this common package is that future client and tooling development could reuse some/all of these abstractions. **New broker libraries (Scala):** The above common libraries are used within the Broker code to implement the read path for feature information, as explained below. 1. Implemented a cache of cluster-wide finalized features (see class: `FinalizedFeatureCache`). This cache stores the latest finalized features and epoch read from the `/feature` node in ZK. Currently the main reader of this cache is the read path that serves an `ApiVersionsRequest` returning the features information in the response. In the future, as we start using the versioning system more, the cache could be read by future read paths intending to learn the finalized feature information. 2. Implemented a feature ZK node change listener, that, listens to changes in the `/feature` node in ZK, and, invalidates the cache defined in `FinalizedFeatureCache` (see above point). See class: `FinalizedFeatureChangeListener`. An instance of this class is maintained in `KafkaServer`, and initialized only if the IBP is set to `KAFKA_2_6_IV1` or higher. 3. Implemented a facility to define supported features within the Broker (these are specific to a broker binary). See companion object: `SupportedFeatures`. Currently the list of supported features is empty, because, we have not identified features (yet). In the future this class can be populated as we begin to define supported features. Note that the public interface of this class provides a way to compare supported vs finalized features to detect incompatibilities. **New Zookeeper node: `/feature`:** The finalized feature information is going to be stored in ZK under the node `/feature`. 1. Implemented a class `FeatureZNode` providing attributes and encode/decode APIs that abstract the structure of the new feature ZK node. 2. Implemented new APIs in `KafkaZkClient.scala` to read/update/delete the `/feature` ZK node. **Updated ApiVersionsResponse**: Defined new tagged fields in existing `ApiVersionsResponse` schema. In the request serving path (in `KafkaApis.scala`), these fields are populated in the response with the latest supported features (from `SupportedFeatures`) and latest finalized features (from `FinalizedFeatureCache`). **Tests:** Added test suites for all of the new classes, and changes made in this PR. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
