[
https://issues.apache.org/jira/browse/KAFKA-13744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17508473#comment-17508473
]
Jorge Esteban Quilcate Otoya commented on KAFKA-13744:
------------------------------------------------------
Thanks, [~junrao]!
I think I got it now after looking deeper into the code and reading the KIPs.
So, depending on the type of quotas defined, if all of them are of the same
type, then all the clients are tagged with that type:
```
case QuotaTypes.NoQuotas | QuotaTypes.ClientIdQuotaEnabled =>
("", clientId)
case QuotaTypes.UserQuotaEnabled =>
(sanitizedUser, "")
case QuotaTypes.UserClientIdQuotaEnabled =>
(sanitizedUser, clientId)
```
But, as soon as there is a mix of types, that leads to evaluate each client
against the quotas registered:
```
case _ =>
val userEntity = Some(UserEntity(sanitizedUser))
val clientIdEntity = Some(ClientIdEntity(clientId))
var metricTags = (sanitizedUser, clientId)
// 1) /config/users/<user>/clients/<client-id>
if (!overriddenQuotas.containsKey(KafkaQuotaEntity(userEntity,
clientIdEntity))) {
// 2) /config/users/<user>/clients/<default>
metricTags = (sanitizedUser, clientId)
if (!overriddenQuotas.containsKey(KafkaQuotaEntity(userEntity,
Some(DefaultClientIdEntity)))) {
// 3) /config/users/<user>
metricTags = (sanitizedUser, "")
if (!overriddenQuotas.containsKey(KafkaQuotaEntity(userEntity,
None))) {
// 4) /config/users/<default>/clients/<client-id>
metricTags = (sanitizedUser, clientId)
if
(!overriddenQuotas.containsKey(KafkaQuotaEntity(Some(DefaultUserEntity),
clientIdEntity))) {
// 5) /config/users/<default>/clients/<default>
metricTags = (sanitizedUser, clientId)
if
(!overriddenQuotas.containsKey(DefaultUserClientIdQuotaEntity)) {
// 6) /config/users/<default>
metricTags = (sanitizedUser, "")
if (!overriddenQuotas.containsKey(DefaultUserQuotaEntity)) {
// 7) /config/clients/<client-id>
// 8) /config/clients/<default>
metricTags = ("", clientId)
}
}
}
}
}
}
```
If the client doesn’t match, then it falls back to the client id. If there’s
any quota for that client, or default, that matches, it applies the tagging
depending on the match.
This now explains why I'm seeing different behavior depending on what quota I
define first.
> Quota metric tags are inconsistent
> ----------------------------------
>
> Key: KAFKA-13744
> URL: https://issues.apache.org/jira/browse/KAFKA-13744
> Project: Kafka
> Issue Type: Bug
> Components: core, metrics
> Reporter: Jorge Esteban Quilcate Otoya
> Priority: Major
> Labels: quotas
> Attachments: image-2022-03-15-16-57-12-583.png
>
>
> When enabling metrics for quotas the metrics apply to _all_ clients (see
> https://issues.apache.org/jira/browse/KAFKA-13742).
> Though, the tags are calculated depending on the quotas registered and
> applied to all clients:
> [https://github.com/apache/kafka/blob/0b9a8bac36f16b5397e9ec3a0441758e4b60a384/core/src/main/scala/kafka/server/ClientQuotaManager.scala#L649-L694]
> This causes different metric tags result depending on which quota is
> registered first.
> For instance, if a quota is registered with userId and clientId, then metrics
> are tagged with both, though if then a quota is registered with only tagged
> with clientId, then all metrics are only tagged by clientId — even though
> user principal is available.
> !image-2022-03-15-16-57-12-583.png|width=1034,height=415!
> I managed to reproduce this behavior here:
> * From 10:30 to 10:45, there was a quota with both client-id and user-id
> * It was removed by 10:45, so no metrics were exposed.
> * After, a quota with client id was created, and metrics were collected only
> with client id, even though the user was available.
> I'd expect metrics to always contain both, if available — and simplify the
> logic here
> [https://github.com/apache/kafka/blob/0b9a8bac36f16b5397e9ec3a0441758e4b60a384/core/src/main/scala/kafka/server/ClientQuotaManager.scala#L649-L694].
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
