I think we could add this functionality to the (operator) test harnesses. I.e. add a mock MetricGroup thingy in there that you can query to check the state of metrics.
> On 13. Oct 2017, at 13:50, Chesnay Schepler <ches...@apache.org> wrote: > > I meant that you could unit-test the behavior of the function in isolation. > You could create a dummy metric group that > verifies that the correct counters are being registered (based on names i > guess), as well as provide access to them. > Mock some input and observe whether the counter value is being modified. > > Whether this is a viable option depends a bit on the complexity of the > function of course, that is how much how mocking > you would have to do. > > On 13.10.2017 11:18, Piotr Nowojski wrote: >> For testing Link applications in general you can read >> https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/testing.html >> >> <https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/stream/testing.html> >> >> However as we said before, testing metrics would require using custom or a >> imx reporter. >> >> Yes, please report this bug in Jira. >> >> Thanks, Piotrek >> >>> On 13 Oct 2017, at 04:31, Colin Williams <colin.williams.seat...@gmail.com >>> <mailto:colin.williams.seat...@gmail.com>> wrote: >>> >>> Team wants an integration test, I'm not sure what unit test you had in >>> mind. Actually feel that I've been trying to avoid the reporter method but >>> that would be more end to end. >>> >>> The documentation for metrics and Scala are missing with the exception of >>> Gauge: >>> https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html >>> >>> <https://ci.apache.org/projects/flink/flink-docs-release-1.3/monitoring/metrics.html> >>> . Should I file a issue against that? >>> >>> Then it leaves you guessing a little bit how to implement Counters. One >>> approach tried was using objects >>> >>> object PointFilter extends RichMapFunction[... >>> @transient lazy val someCounter = >>> getRuntimeContext.getMetricGroup.counter(...) >>> >>> This allowed access to the counter before and after execution . However >>> between the unit tests the Counter kept its value also and that's a no for >>> the test. Think that might be an issue with ScalaTest. >>> >>> I've tried to get at the counter from some other directions like trying to >>> find a way to inject a reporter to get it's state. But don't see a way to >>> do it. So probably the best thing to do is fire up something to collect the >>> metrics from the reporter. >>> >>> On Thu, Oct 12, 2017 at 5:29 AM, Chesnay Schepler <ches...@apache.org >>> <mailto:ches...@apache.org>> wrote: >>> Well damn, i should've read the second part of the initial mail. >>> >>> I'm wondering though, could you not unit-test this behavior? >>> >>> >>> On 12.10.2017 14:25, Chesnay Schepler wrote: >>> You could also write a custom reporter that opens a socket or similar for >>> communication purposes. >>> >>> You can then either query it for the metrics, or even just trigger the >>> verification in the reporter, >>> and fail with an error if the reporter returns an error. >>> >>> On 12.10.2017 14:02, Piotr Nowojski wrote: >>> Hi, >>> >>> Doing as you proposed using JMXReporter (or custom reporter) should work. I >>> think there is no easier way to do this at the moment. >>> >>> Piotrek >>> >>> On 12 Oct 2017, at 04:58, Colin Williams <colin.williams.seat...@gmail.com >>> <mailto:colin.williams.seat...@gmail.com>> wrote: >>> >>> I have a RichMapFunction and I'd like to ensure Meter fields are properly >>> incremented. I've been trying to think of the best way to do this. >>> Currently I think that I'd need to either implement my own reporter (or use >>> JMX) and write to a socket, create a listener and wait for the reporter to >>> send the message. >>> >>> Is this a good approach for writing the test, or should I be considering >>> something else? >>> >>> >>> >>> >>> >> >
