chia7712 commented on code in PR #15889:
URL: https://github.com/apache/kafka/pull/15889#discussion_r1610336329
##########
clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java:
##########
@@ -40,7 +40,7 @@ public abstract class SampledStat implements MeasurableStat {
public SampledStat(double initialValue) {
this.initialValue = initialValue;
- this.samples = new ArrayList<>(2);
+ this.samples = new ArrayList<>(3);
Review Comment:
It would be nice to add comments for this magic number :)
##########
clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java:
##########
@@ -106,44 +109,51 @@ public String toString() {
public abstract double combine(List<Sample> samples, MetricConfig config,
long now);
- /* Timeout any windows that have expired in the absence of any events */
+ // purge any samples that lack observed events within the monitored window
protected void purgeObsoleteSamples(MetricConfig config, long now) {
long expireAge = config.samples() * config.timeWindowMs();
for (Sample sample : samples) {
- if (now - sample.lastWindowMs >= expireAge)
+ // samples overlapping the monitored window are kept,
+ // even if they started before it
+ if (now - sample.lastEventMs >= expireAge) {
sample.reset(now);
+ }
}
}
protected static class Sample {
public double initialValue;
public long eventCount;
- public long lastWindowMs;
+ public long startTimeMs;
+ public long lastEventMs;
public double value;
public Sample(double initialValue, long now) {
this.initialValue = initialValue;
this.eventCount = 0;
- this.lastWindowMs = now;
+ this.startTimeMs = now;
+ this.lastEventMs = now;
this.value = initialValue;
}
public void reset(long now) {
this.eventCount = 0;
- this.lastWindowMs = now;
+ this.startTimeMs = now;
+ this.lastEventMs = now;
this.value = initialValue;
}
public boolean isComplete(long timeMs, MetricConfig config) {
- return timeMs - lastWindowMs >= config.timeWindowMs() ||
eventCount >= config.eventWindow();
+ return timeMs - startTimeMs >= config.timeWindowMs() || eventCount
>= config.eventWindow();
Review Comment:
This is unrelated to this PR, but it seems `eventCount` could be another
issue if we set a non-maximum value. For example, all samples are within a
single `timeWindowMs` due to a bunch of records. in this case,
`totalElapsedTimeMs` will be equal to `(config.samples() - 1) * timeWindowMs`,
and then we will get smaller measure due to the larger denominator
Maybe we can remove `eventWindow` as it is unused in production. This can be
another PR if it is valid
##########
clients/src/main/java/org/apache/kafka/common/metrics/stats/SampledStat.java:
##########
@@ -50,10 +50,13 @@ public void record(MetricConfig config, double value, long
timeMs) {
sample = advance(config, timeMs);
update(sample, config, value, timeMs);
sample.eventCount += 1;
+ sample.lastEventMs = timeMs;
}
private Sample advance(MetricConfig config, long timeMs) {
- this.current = (this.current + 1) % config.samples();
+ // need to keep one extra sample (see purgeObsoleteSamples() logic)
Review Comment:
How about saying "we have one extra sample to remember the last recording
time in order to keep the overlapping sample (see purgeObsoleteSamples() logic)"
##########
clients/src/test/java/org/apache/kafka/common/metrics/stats/SampledStatTest.java:
##########
@@ -0,0 +1,110 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.kafka.common.metrics.stats;
+
+import org.apache.kafka.common.metrics.MetricConfig;
+import org.apache.kafka.common.utils.MockTime;
+import org.apache.kafka.common.utils.Time;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.DisplayName;
+import org.junit.jupiter.api.Test;
+
+import java.util.List;
+
+import static java.util.concurrent.TimeUnit.SECONDS;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+
+class SampledStatTest {
+
+ private SampledStat stat;
+ private Time time;
+
+ @BeforeEach
+ public void setup() {
+ stat = new SampleCount(0);
+ time = new MockTime();
+ }
+
+ @Test
+ @DisplayName("Sample should be purged if doesn't overlap the window")
+ public void testSampleIsPurgedIfDoesntOverlap() {
+ MetricConfig config = new MetricConfig().timeWindow(1,
SECONDS).samples(2);
+
+ // Monitored window: 2s. Complete a sample and wait 2.5s after.
+ completeSample(config);
+ time.sleep(2500);
+
+ double numSamples = stat.measure(config, time.milliseconds());
+ assertEquals(0, numSamples);
+ }
+
+ @Test
+ @DisplayName("Sample should be kept if overlaps the window")
+ public void testSampleIsKeptIfOverlaps() {
+ MetricConfig config = new MetricConfig().timeWindow(1,
SECONDS).samples(2);
+
+ // Monitored window: 2s. Complete a sample and wait 1.5s after.
+ completeSample(config);
+ time.sleep(1500);
+
+ double numSamples = stat.measure(config, time.milliseconds());
+ assertEquals(1, numSamples);
+ }
+
+ @Test
+ @DisplayName("Sample should be kept if overlaps the window and is n+1")
+ public void testSampleIsKeptIfOverlapsAndExtra() {
+ MetricConfig config = new MetricConfig().timeWindow(1,
SECONDS).samples(2);
+
+ // Monitored window: 2s. Create 2 samples with gaps in between and
+ // take a measurement at 2.2s from the start.
+ completeSample(config);
+ time.sleep(100);
+ completeSample(config);
+ time.sleep(100);
+ stat.record(config, 1, time.milliseconds());
+
+ double numSamples = stat.measure(config, time.milliseconds());
+ assertEquals(3, numSamples);
+ }
+
+ // Creates a sample with events at the start and at the end. Positions
clock at the end.
+ private void completeSample(MetricConfig config) {
+ stat.record(config, 1, time.milliseconds());
+ time.sleep(config.timeWindowMs() - 1);
+ stat.record(config, 1, time.milliseconds());
+ time.sleep(1);
+ }
+
+ // measure() of this impl returns the number of samples
+ static class SampleCount extends SampledStat {
+
+ SampleCount(double initialValue) {
Review Comment:
We don't use `initialValue` in testing, so it can be simplified to
```java
SampleCount() {
super(0);
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
