swatiksi273-ksolves opened a new pull request, #1139:
URL: https://github.com/apache/flink-kubernetes-operator/pull/1139
## What is the purpose of the change
When the JobManager loses connection to TaskManagers (e.g. due to transient
network issues on
Kubernetes), the Flink REST API returns `complete: false` for vertex IO
metrics (`read-records`,
`write-records`) along with zero values. The operator was ignoring the
`complete` flag and treating
these zeros as valid throughput data, causing `TARGET_DATA_RATE` to drop to
zero and triggering
incorrect scale down even when the job was busy.
This was confirmed by the reporter (FLINK-39925) — killing the JM and
letting a new one spin up
resolved the issue, with metrics returning `complete: true` and correct
values after restart.
## Brief change log
- Added `metricsComplete` field to `IOMetrics` to track whether vertex IO
metrics are complete
- `IOMetrics.from()` now checks `isRecordsReadComplete()` and
`isRecordsWrittenComplete()` flags
from `IOMetricsInfo` to determine if metrics are reliable
- `ScalingMetrics.computeDataRateMetrics()` now skips scaling decisions
and sets
`NUM_RECORDS_IN` and `NUM_RECORDS_OUT` to `Double.NaN` when metrics are
incomplete
- Added convenience constructor `IOMetrics(long, long, double)` that
defaults `metricsComplete`
to `true` for backward compatibility
## Verifying this change
This change is already covered by existing tests, such as
`ScalingMetricEvaluatorTest` (14 tests),
`ScalingMetricCollectorTest` (8 tests) and `ScalingMetricsTest` — all pass
with the new changes.
The fix was also verified against the reporter's payload where
`read-records-complete: false` and
`write-records: 0` were returned by the REST API while the job was clearly
busy (33% busy time).
## Does this pull request potentially affect one of the following parts:
- Dependencies (does it add or upgrade a dependency): no
- The public API, i.e., is any changes to the `CustomResourceDescriptors`:
no
- Core observer or reconciler logic that is regularly executed: yes
## Documentation
- Does this pull request introduce a new feature? no
- If yes, how is the feature documented? not applicable
## Jira
https://issues.apache.org/jira/browse/FLINK-39925
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
