splett2 commented on a change in pull request #9386:
URL: https://github.com/apache/kafka/pull/9386#discussion_r521856417
##########
File path: core/src/main/scala/kafka/network/SocketServer.scala
##########
@@ -1324,7 +1401,59 @@ class ConnectionQuotas(config: KafkaConfig, time: Time,
metrics: Metrics) extend
private[network] def updateBrokerMaxConnectionRate(maxConnectionRate: Int):
Unit = {
// if there is a connection waiting on the rate throttle delay, we will
let it wait the original delay even if
// the rate limit increases, because it is just one connection per
listener and the code is simpler that way
- updateConnectionRateQuota(maxConnectionRate)
+ updateConnectionRateQuota(maxConnectionRate, BrokerQuotaEntity)
+ }
+
+ /**
+ * Update the connection rate quota for a given IP and updates quota configs
for updated IPs.
+ * If an IP is given, metric config will be updated only for the given IP,
otherwise
+ * all metric configs will be checked and updated if required.
+ *
+ * @param ip ip to update or default if None
+ * @param maxConnectionRate new connection rate, or resets entity to default
if None
+ */
+ def updateIpConnectionRateQuota(ip: Option[String], maxConnectionRate:
Option[Int]): Unit = {
+ def isIpConnectionRateMetric(metricName: MetricName) = {
+ metricName.name == ConnectionRateMetricName &&
+ metricName.group == MetricsGroup &&
+ metricName.tags.containsKey(IpMetricTag)
+ }
+
+ def shouldUpdateQuota(metric: KafkaMetric, quotaLimit: Int) = {
+ quotaLimit != metric.config.quota.bound
+ }
+ counts.synchronized {
Review comment:
I'm thinking on this a bit more, and I think that only locking on
updates to `defaultConnectionRatePerIp` should be sufficient for correctness.
ZK dynamic config changes are processed within one thread, so we will only
have one thread executing in `updateIpConnectionRateQuota`. If we want to be
really careful about this, we can have `IpConfigHandler` synchronize on
`ConnectionQuotas`.
The case we want to avoid by synchronizing on `counts` is that another
thread `defaultConnectionRatePerIp` as connection rate limit `A` while calling
`inc()`, then we update connection rate and quota metrics to `B`, then the
other thread creates a sensor/metric with quota limit `A` => inconsistency.
If we synchronize on `counts` for only updates to
`connectionRateForIp/defaultConnectionRate`, we know that any thread that has
read a connection rate quota as `A` will finish creating quota metrics with
limit `A` before we acquire the `counts` lock.
After acquiring the `counts` lock and updating
`connectionRateForIp/defaultConnectionRate` to `B`, any subsequent threads
calling `inc()` will read the quota as `B` and create a metric as `B`. We will
then be able to update any quota metrics from `A` to `B`, knowing that all new
ones will be created as `B`.
The only issue remaining is that we can get concurrent reads of
`connectionRatePerIp`, but we can just replace `mutable.Map` with
`ConcurrentHashMap`.
Let me know if I'm missing something here with respect to thread safety.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
