lizhiqiang created FLINK-37340:
----------------------------------
Summary: Flink Flink supports pushing metrics immediately at the
end of the task.
Key: FLINK-37340
URL: https://issues.apache.org/jira/browse/FLINK-37340
Project: Flink
Issue Type: Improvement
Components: Runtime / Metrics
Affects Versions: 2.0-preview
Reporter: lizhiqiang
Fix For: 2.1.0
In Flink's stream computing, metrics should be pushed once at the end of the
task. But now the metrics are relying on timed scheduling to configure them,
which may lead to inaccurate metrics at the end of the task.
scheduler interval config :
{code:java}
metrics.reporter.xxx.interval: 60 SECONDS {code}
If A is set to be particularly large, then the metric may be more imprecise
Flink's batch computing can't utilize the flink-metric plugin to push metrics
to third-party systems. It may be necessary to make some changes to Flink's
metrics process.
Some of my thoughts are whether to aggregate metrics to JobManager through
heartbeat detection.
{code:java}
//代码占位符
org.apache.flink.runtime.metrics.MetricRegistryImpl#MetricRegistryImpl(org.apache.flink.runtime.metrics.MetricRegistryConfiguration,
java.util.Collection<org.apache.flink.runtime.metrics.ReporterSetup>,
java.util.concurrent.ScheduledExecutorService)
executor.scheduleWithFixedDelay(
new ReporterTask((Scheduled) reporterInstance),
period.toMillis(),
period.toMillis(),
TimeUnit.MILLISECONDS); {code}
{code:java}
//代码占位符
@Override
public void report() {
try {
//Push every 60 seconds
pushGateway.push(registry, jobName, groupingKey);
} catch (Exception e) {
log.warn(
"Failed to push metrics to PushGateway with jobName {},
groupingKey {}.",
jobName,
groupingKey,
e);
}
} {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
