[
https://issues.apache.org/jira/browse/KAFKA-20848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthias J. Sax updated KAFKA-20848:
------------------------------------
Description:
With the new KIP-848 based consumer/share/streams groups, many configs moved
from the client into the broker.
Some of these config are sent back to the client (as the client really uses
them), and we recently started to log them
([https://github.com/apache/kafka/pull/22749] and
[https://github.com/apache/kafka/pull/22730]) for improved observability.
However, there is a few more configs which have broker defaults plus group
level overwrites, and they are not visible on the client (only describe-config
or describe-group would reveal them). At the same time, the broker might
overwrite group-level config (clamping to min/max for example), what is not
visible either on the client.
For Kafka Streams, we have one hybrid config (client + broker combination)
([https://github.com/apache/kafka/pull/22213]) and introduced and error code in
the StreamsHeartbeatResponse to surface a misconfiguration client side.
The ides of this ticket is, to do a similar thing for _all_ group level
configs. We would add _all_ effective group config to the HB response for
logging, with relogging if changed. Additionally, if a value is clamped, and
the set group-config is not the effective config, we return some error code,
allowing the client to also log a WARNING about it.
This would be a protocol change, so thus we need a KIP.
was:
With the new KIP-848 based consumer/share/streams groups, many configs moved
from the client into the broker.
Some of these config are sent back to the client (as the client really uses
them), and we recently started to log them
([https://github.com/apache/kafka/pull/22749] and
[https://github.com/apache/kafka/pull/22730]) for improved observability.
However, there is a few more configs which have broker defaults plus group
level overwrites, and they are not visible on the client (only describe-config
or describe-group would reveal them). At the same time, the broker might
overwrite group-level config (clamping to min/max for example), what it not
visible either on the client.
For Kafka Streams, we have one hybrid config (client + broker combination)
([https://github.com/apache/kafka/pull/22213]) and introduced and error code in
the StreamsHeartbeatResponse to surface a misconfiguration client side.
The ides of this ticket is, to do a similar thing for _all_ group level
configs. We would add _all_ effective group config to the HB response for
logging, with relogging if changed. Additionally, if a value is clamped, and
the set group-config is not the effective config, we return some error code,
allowing the client to also log a WARNING about it.
This would be a protocol change, so thus we need a KIP.
> Consider to surface group-level misconfigurations client side
> -------------------------------------------------------------
>
> Key: KAFKA-20848
> URL: https://issues.apache.org/jira/browse/KAFKA-20848
> Project: Kafka
> Issue Type: Improvement
> Components: clients, consumer, group-coordinator, streams
> Reporter: Matthias J. Sax
> Priority: Minor
> Labels: needs-kip
>
> With the new KIP-848 based consumer/share/streams groups, many configs moved
> from the client into the broker.
> Some of these config are sent back to the client (as the client really uses
> them), and we recently started to log them
> ([https://github.com/apache/kafka/pull/22749] and
> [https://github.com/apache/kafka/pull/22730]) for improved observability.
> However, there is a few more configs which have broker defaults plus group
> level overwrites, and they are not visible on the client (only
> describe-config or describe-group would reveal them). At the same time, the
> broker might overwrite group-level config (clamping to min/max for example),
> what is not visible either on the client.
> For Kafka Streams, we have one hybrid config (client + broker combination)
> ([https://github.com/apache/kafka/pull/22213]) and introduced and error code
> in the StreamsHeartbeatResponse to surface a misconfiguration client side.
> The ides of this ticket is, to do a similar thing for _all_ group level
> configs. We would add _all_ effective group config to the HB response for
> logging, with relogging if changed. Additionally, if a value is clamped, and
> the set group-config is not the effective config, we return some error code,
> allowing the client to also log a WARNING about it.
> This would be a protocol change, so thus we need a KIP.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
