Github user zentol commented on a diff in the pull request:
https://github.com/apache/flink/pull/5857#discussion_r191376001
--- Diff:
flink-metrics/flink-metrics-prometheus/src/main/java/org/apache/flink/metrics/prometheus/PrometheusPushGatewayReporter.java
---
@@ -0,0 +1,79 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.flink.metrics.prometheus;
+
+import org.apache.flink.annotation.PublicEvolving;
+import org.apache.flink.metrics.Metric;
+import org.apache.flink.metrics.MetricConfig;
+import org.apache.flink.metrics.reporter.MetricReporter;
+import org.apache.flink.metrics.reporter.Scheduled;
+import org.apache.flink.util.AbstractID;
+
+import io.prometheus.client.CollectorRegistry;
+import io.prometheus.client.exporter.PushGateway;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * /**
+ * {@link MetricReporter} that exports {@link Metric Metrics} via
Prometheus Pushgateway.
+ */
+@PublicEvolving
+public class PrometheusPushGatewayReporter extends
AbstractPrometheusReporter implements Scheduled {
+ private static final Logger LOG =
LoggerFactory.getLogger(PrometheusPushGatewayReporter.class);
+
+ public static final String ARG_HOST = "host";
+ public static final String ARG_PORT = "port";
+
+ public static final char JOB_NAME_SEPARATOR = '-';
+ public static final String JOB_NAME_PREFIX = "flink" +
JOB_NAME_SEPARATOR;
+
+ private PushGateway pushGateway;
+ private final String jobName;
+
+ public PrometheusPushGatewayReporter() {
+ String random = new AbstractID().toString();
+ jobName = JOB_NAME_PREFIX + random;
--- End diff --
To implement this we will have to change the design of the new reporter
significantly. This will entail moving more logic from the abstract class to
the concrete `PrometheusReporter` class as it will no longer be shared. Ideally
the abstract class will only contain shared utility methods, and does not
dictate the actual report/notification logic.
The pushgateway reporter will create a separate collector for each grouping
key (i.e. variable names/values, retrieved via
`MetricGroup#getAllVariables()`). The collectors are not registered with the
registry as this is only necessary in the pull model. The grouping by labels
should ensure that we don't send to many requests; we may want to introduce a
delay parameter to be safe.
A new metric will be assigned to a collector based on the hash of the
variables map; we may not use the variables map or MetricGroup direclty as a
mapping key as the map may change over time.
Similar to the existing vanilla reporter we will have to rely on
ref-counting to cleanup existing collectors.
This will allow users to freely switch between both reporters without
having to worry about any additional detail but the port configuration.
---
