lucasbru opened a new pull request, #22393: URL: https://github.com/apache/kafka/pull/22393
This is a proof-of-concept implementation of KIP-1331 (Streams Group Topology Description Plugin), opened as a draft to support the ongoing DISCUSS thread on the dev list. The KIP is at https://cwiki.apache.org/confluence/display/KAFKA/KIP-1331%3A+Streams+Group+Topology+Description+Plugin. The change introduces a broker-side SPI (`StreamsGroupTopologyDescriptionPlugin`) and an admin/client surface for storing, retrieving and tombstoning Kafka Streams topology descriptions. Clients push the description via a new `StreamsGroupTopologyDescriptionUpdate` RPC when solicited by a heartbeat flag, the broker dispatches set / get / delete through the plugin, and admin clients can request the description back via `StreamsGroupDescribe`. A per-group `StoredTopologyEpoch` and `LastFailedTopologyEpoch` are persisted as tagged fields on `StreamsGroupMetadataValue` so heartbeat solicitation, describe gating, and natural-expiration cleanup are all driven by broker-side state without per-heartbeat plugin RPCs. The plugin SPI is intentionally narrow — `setTopology` / `getTopology` / `deleteTopology`, each async via `CompletableFuture`. Permanent vs transient failure is signalled through two SPI exception classes that drive a broker-internal back-off ratchet. The branch tracks the KIP wording. Review the KIP itself rather than this code; comments here should focus on implementation feasibility questions. This POC will serve as a blueprint for the implementation — it will not be merged as-is. Once the KIP is accepted, the changes will be broken up and independently proposed and reviewed as separate PRs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
