[
https://issues.apache.org/jira/browse/BEAM-6165?focusedWorklogId=171972&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-171972
]
ASF GitHub Bot logged work on BEAM-6165:
----------------------------------------
Author: ASF GitHub Bot
Created on: 04/Dec/18 17:50
Start Date: 04/Dec/18 17:50
Worklog Time Spent: 10m
Work Description: ryan-williams commented on a change in pull request
#7183: [BEAM-6165] send metrics to Flink in portable Flink runner
URL: https://github.com/apache/beam/pull/7183#discussion_r238770025
##########
File path:
runners/flink/src/main/java/org/apache/beam/runners/flink/metrics/FileReporter.java
##########
@@ -0,0 +1,74 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.beam.runners.flink.metrics;
+
+import java.io.FileNotFoundException;
+import java.io.FileOutputStream;
+import java.io.PrintStream;
+import org.apache.flink.metrics.Metric;
+import org.apache.flink.metrics.MetricConfig;
+import org.apache.flink.metrics.MetricGroup;
+import org.apache.flink.metrics.reporter.AbstractReporter;
+
+/**
+ * Flink {@link org.apache.flink.metrics.reporter.MetricReporter metrics
reporter} for writing
+ * metrics to a file (specified via the "metrics.reporter.test.file" config
key).
+ */
+public class FileReporter extends AbstractReporter {
Review comment:
> it's configured via a test property
ah, [I left the reporter alias as `test` in
`flink_runner_test`](https://github.com/apache/beam/pull/7183/files#diff-195e4a1918aec54f6245e49081cadb5aR99)
when I renamed this class to remove the `Test` prefix. I should change that
alias to `file`, either way.
This is a reporter that writes the final state of all metrics to a file of
the user's choosing. It can do this in test or production environments. It
doesn't perform any assertions, or otherwise do things that only make sense in
a testing context. It is fairly similar to
[Slf4jReporter](https://github.com/apache/flink/blob/release-1.5.5/flink-metrics/flink-metrics-slf4j/src/main/java/org/apache/flink/metrics/slf4j/Slf4jReporter.java#L41),
but its output is easier to parse (for machines or humans), and would normally
be configured like:
```
metrics.reporters: file,
metrics.reporter.file.class:
org.apache.beam.runners.flink.metrics.FileReporter,
metrics.reporter.file.file: /some/path/on/job-mgr/node
```
To me, it feels weirder to name it `Test*` and leave it under `main/`, when
it is not really *that* test-specific an abstraction.
However I understand if the possible non-test uses feel contrived, and
perhaps having `Test*` names under `main/` isn't too smelly.
I just want to make sure the vestigial `metrics.reporter.test.*` namespacing
isn't driving the decision too heavily.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 171972)
Time Spent: 4h 10m (was: 4h)
> Send metrics to Flink in portable Flink runner
> ----------------------------------------------
>
> Key: BEAM-6165
> URL: https://issues.apache.org/jira/browse/BEAM-6165
> Project: Beam
> Issue Type: Improvement
> Components: runner-flink
> Affects Versions: 2.8.0
> Reporter: Ryan Williams
> Assignee: Ryan Williams
> Priority: Major
> Labels: metrics, portability, portability-flink
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> Metrics are sent from the fn harness to runner in the Python SDK (and likely
> Java soon), but the portable Flink runner doesn't pass them on to Flink,
> which it should, so that users can see them in e.g. the Flink UI or via any
> Flink metrics reporters.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
