mxm commented on code in PR #945:
URL:
https://github.com/apache/flink-kubernetes-operator/pull/945#discussion_r1961334586
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java:
##########
@@ -149,19 +149,31 @@ public CollectedMetricHistory updateMetrics(
// Add scaling metrics to history if they were computed successfully
metricHistory.put(now, scalingMetrics);
- if (isStabilizing) {
- LOG.info("Stabilizing until {}", readable(stableTime));
- stateStore.storeCollectedMetrics(ctx, metricHistory);
- return new CollectedMetricHistory(topology,
Collections.emptySortedMap(), jobRunningTs);
- }
-
var collectedMetrics = new CollectedMetricHistory(topology,
metricHistory, jobRunningTs);
if (now.isBefore(windowFullTime)) {
- LOG.info("Metric window not full until {}",
readable(windowFullTime));
+ if (isStabilizing) {
+ LOG.info(
+ "Stabilizing... until {}. {} samples collected",
Review Comment:
NIT: Can we keep the existing output?
```suggestion
"Stabilizing until {}",
```
I don't think we need the number of samples here. This is just confusing
because we are not supposed to collect any metrics for evaluation.
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java:
##########
@@ -149,19 +149,31 @@ public CollectedMetricHistory updateMetrics(
// Add scaling metrics to history if they were computed successfully
metricHistory.put(now, scalingMetrics);
- if (isStabilizing) {
- LOG.info("Stabilizing until {}", readable(stableTime));
- stateStore.storeCollectedMetrics(ctx, metricHistory);
- return new CollectedMetricHistory(topology,
Collections.emptySortedMap(), jobRunningTs);
- }
-
var collectedMetrics = new CollectedMetricHistory(topology,
metricHistory, jobRunningTs);
if (now.isBefore(windowFullTime)) {
- LOG.info("Metric window not full until {}",
readable(windowFullTime));
+ if (isStabilizing) {
+ LOG.info(
+ "Stabilizing... until {}. {} samples collected",
+ readable(stableTime),
+ metricHistory.size());
+ } else {
+ LOG.info(
+ "Metric window is not full until {}. {} samples
collected",
+ readable(windowFullTime),
+ metricHistory.size());
+ }
} else {
collectedMetrics.setFullyCollected(true);
// Trim metrics outside the metric window from metrics history
- metricHistory.headMap(now.minus(metricWindowSize)).clear();
+ var trimBefore = now.minus(metricWindowSize);
+ var head = metricHistory.headMap(trimBefore);
+ var dropped = head.size();
+ head.clear();
+ LOG.debug(
+ "Metric window is now full. Dropped {} samples before {},
keeping {}.",
+ dropped,
+ readable(trimBefore),
+ metricHistory.size());
Review Comment:
This change is unrelated to the issue which we're trying to fix here. A
couple of changes to avoid confusion:
NIT
```suggestion
var trimBefore = now.minus(metricWindowSize);
int numDropped = removeMetricsBefore(trimBefore);
LOG.debug(
"Metric window is now full. Dropped {} samples before
{}, keeping {}.",
numDropped,
readable(trimBefore),
metricHistory.size());
```
```java
private int removeMetricsBefore(Instant cutOffTimestamp) {
var toBeDropped = metricHistory.headMap(cutOffTimestamp);
var numDropped = toBeDropped.size();
toBeDropped.clear();
return numDropped;
}
```
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/utils/DateTimeUtils.java:
##########
@@ -26,7 +26,7 @@
public class DateTimeUtils {
private static final DateTimeFormatter DEFAULT_FORMATTER =
- DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
+ DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss.SSS");
Review Comment:
Why this change? I think seconds accuracy is sufficient and more easily
readable.
##########
flink-autoscaler/src/main/java/org/apache/flink/autoscaler/ScalingMetricCollector.java:
##########
@@ -149,19 +149,31 @@ public CollectedMetricHistory updateMetrics(
// Add scaling metrics to history if they were computed successfully
metricHistory.put(now, scalingMetrics);
- if (isStabilizing) {
- LOG.info("Stabilizing until {}", readable(stableTime));
- stateStore.storeCollectedMetrics(ctx, metricHistory);
- return new CollectedMetricHistory(topology,
Collections.emptySortedMap(), jobRunningTs);
- }
-
var collectedMetrics = new CollectedMetricHistory(topology,
metricHistory, jobRunningTs);
if (now.isBefore(windowFullTime)) {
- LOG.info("Metric window not full until {}",
readable(windowFullTime));
+ if (isStabilizing) {
+ LOG.info(
+ "Stabilizing... until {}. {} samples collected",
+ readable(stableTime),
+ metricHistory.size());
+ } else {
+ LOG.info(
+ "Metric window is not full until {}. {} samples
collected",
Review Comment:
```suggestion
"Metric window is not full until {}. {} samples
collected so far",
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
