Hi Andrew,
Thanks for the review.
AS1: The kafka-client-metrics.sh —describe —name for a non-existent
client-metric shows default configuration.
The behavior is inconsistent with kafka-configs.sh in this KIP. I add this case
to proposed changes as well. Thanks.
AS2: Thanks for the great suggestion. We have an API DescribeConfig to get all
kinds of configuration.
It’s better to have a API to list all types of configuration. I change
ListClientMetricsResources to ListConfigResources
and remove change for ListGroups. For backward compatibility, we also need to
make sure v0 ListConfigResources
can work as ListClientMetricsResources.
AS3: Without change for ListGroupsResponse, we don’t need to handle state and
type fields in this KIP.
AS4: Remove change for ListGroupsResponse.
Thanks,
PoAn
> On Mar 14, 2025, at 6:37 PM, Andrew Schofield
> <andrew_schofield_j...@outlook.com> wrote:
>
> Hi PoAn,
> Thanks for the KIP. It would certainly be good to fix this slightly odd
> behaviour in Kafka configs.
>
> AS1: I'd like to confirm that the KIP also alters the behaviour of
> kafka-client-metrics.sh --describe
> for the case where the resource does not exist. You do show this in an
> example, but I wanted to
> confirm that this reflects a behaviour change that this KIP will enact.
>
> AS2: I wonder whether there's an alternative way to do this in the protocol.
> In the way that
> you have specified it, if ListGroups specified
> IncludeNonExistentGroupWithDynamicConfig,
> the returned list of groups comes from two sources - the groups in the group
> coordinator
> and the group config resources in the metadata.
>
> I wonder whether actually you could rename the ListClientMetricsResources RPC
> to
> ListConfigResources, bump it to version 1 with a schema like this:
>
> {
> "apiKey": 74,
> "type": "request",
> "listeners": ["broker"],
> "name": "ListConfigResourcesRequest",
> "validVersions": "0-1",
> "flexibleVersions": "0+",
> "fields": [
> { "name": "ResourceType", "type": "int8", "versions": "1+", "mapKey":
> true,
> "about": "The resource type." }
> ]
> }
>
> For v0, the resource type is client metrics.
>
> Then, the response would be:
>
> {
> "apiKey": 74,
> "type": "response",
> "name": "ListConfigResourcesResponse",
> "validVersions": "0-1",
> "flexibleVersions": "0+",
> "fields": [
> { "name": "ThrottleTimeMs", "type": "int32", "versions": "0+",
> "about": "The duration in milliseconds for which the request was
> throttled due to a quota violation, or zero if the request did not violate
> any quota." },
> { "name": "ErrorCode", "type": "int16", "versions": "0+",
> "about": "The error code, or 0 if there was no error." },
> { "name": "ConfigResources", "type": "[]ConfigResource", "versions":
> "0+",
> "about": "Each resource in the response.", "fields": [
> { "name": "Name", "type": "string", "versions": "0+",
> "about": "The resource name." }
> ]}
> ]
> }
>
> Then, this could list the group config resources (which is not the same as
> listing the existing groups):
>
>> bin/kafka-configs.sh --describe --entity-type groups
>
> And perhaps, this could list the config for the existing groups because it
> uses ListGroups to get the list of groups:
>
>> bin/kafka-configs.sh --describe --entity-type groups --effective
>
> The advantage to having ListConfigResources is that it would apply to future
> resources too without needing
> additional RPCs. It also lets you distinguish between what's configuration
> and what's live.
>
> Just a random idea for your consideration.
>
> AS3: A group which does not exist but is only represented by a configuration
> resource has
> neither a type nor a state. What values will be used for these fields in the
> ListGroupsResponse.
>
> AS4: The comment in ListGroupsResponse for v6 is incorrect. v6 adds the
> NonExistentGroupWithDynamicConfig field.
>
>
> Thanks,
> Andrew
>
> ________________________________________
> From: PoAn Yang <yangp...@gmail.com>
> Sent: 12 March 2025 16:00
> To: dev@kafka.apache.org <dev@kafka.apache.org>
> Subject: [DISCUSS] KIP-1142: Allow to list non-existent group which has
> dynamic config
>
> Hi all,
>
> I would like to start a discussion thread on KIP-1142.
>
> Currently, kafka-configs.sh requires explicit group names to describe
> configurations, which limits flexibility in managing dynamically configured
> groups. This KIP proposes extending the ListGroups API to include groups that
> do not exist in the coordinator but have dynamic configurations.
>
> Please take a look and feel free to share any thoughts.
>
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-1142%3A+Allow+to+list+non-existent+group+which+has+dynamic+config
>
> Thanks,
> PoAn
