Thanks for the help. I ended up creating a custom metric reporter and accessing it's fields in an integration test. However I do think that the test that Martin checked in is another good way to test. I opened https://issues.apache.org/jira/browse/FLINK-7907 regarding the missing Scala examples in the metrics documentation. I can take a stab at that on the weekend if somebody doesn't beat me to it.
On Fri, Oct 13, 2017 at 6:09 AM, Martin Eden <martineden...@gmail.com> wrote: > Hi, > Not merged in yet but this is an example pr that is mocking metrics and > checking they are properly updated: > https://github.com/apache/flink/pull/4725 > > > On Fri, Oct 13, 2017 at 1:49 PM, Aljoscha Krettek <aljos...@apache.org> > wrote: > >> I think we could add this functionality to the (operator) test harnesses. >> I.e. add a mock MetricGroup thingy in there that you can query to check the >> state of metrics. >> >> >> On 13. Oct 2017, at 13:50, Chesnay Schepler <ches...@apache.org> wrote: >> >> I meant that you could unit-test the behavior of the function in >> isolation. You could create a dummy metric group that >> verifies that the correct counters are being registered (based on names i >> guess), as well as provide access to them. >> Mock some input and observe whether the counter value is being modified. >> >> Whether this is a viable option depends a bit on the complexity of the >> function of course, that is how much how mocking >> you would have to do. >> >> On 13.10.2017 11:18, Piotr Nowojski wrote: >> >> For testing Link applications in general you can read >> https://ci.apache.org/projects/flink/flink-docs-release >> -1.4/dev/stream/testing.html >> >> However as we said before, testing metrics would require using custom or >> a imx reporter. >> >> Yes, please report this bug in Jira. >> >> Thanks, Piotrek >> >> On 13 Oct 2017, at 04:31, Colin Williams <colin.williams.seattle@gmail. >> com> wrote: >> >> Team wants an integration test, I'm not sure what unit test you had in >> mind. Actually feel that I've been trying to avoid the reporter method but >> that would be more end to end. >> >> The documentation for metrics and Scala are missing with the exception of >> Gauge: https://ci.apache.org/projects/flink/flink-docs-release-1.3/ >> monitoring/metrics.html . Should I file a issue against that? >> >> Then it leaves you guessing a little bit how to implement Counters. One >> approach tried was using objects >> >> object PointFilter extends RichMapFunction[... >> >> @transient lazy val someCounter = >> getRuntimeContext.getMetricGroup.counter(...) >> >> >> This allowed access to the counter before and after execution . However >> between the unit tests the Counter kept its value also and that's a no for >> the test. Think that might be an issue with ScalaTest. >> >> I've tried to get at the counter from some other directions like trying >> to find a way to inject a reporter to get it's state. But don't see a way >> to do it. So probably the best thing to do is fire up something to collect >> the metrics from the reporter. >> >> On Thu, Oct 12, 2017 at 5:29 AM, Chesnay Schepler <ches...@apache.org> >> wrote: >> >>> Well damn, i should've read the second part of the initial mail. >>> >>> I'm wondering though, could you not unit-test this behavior? >>> >>> >>> On 12.10.2017 14:25, Chesnay Schepler wrote: >>> >>>> You could also write a custom reporter that opens a socket or similar >>>> for communication purposes. >>>> >>>> You can then either query it for the metrics, or even just trigger the >>>> verification in the reporter, >>>> and fail with an error if the reporter returns an error. >>>> >>>> On 12.10.2017 14:02, Piotr Nowojski wrote: >>>> >>>>> Hi, >>>>> >>>>> Doing as you proposed using JMXReporter (or custom reporter) should >>>>> work. I think there is no easier way to do this at the moment. >>>>> >>>>> Piotrek >>>>> >>>>> On 12 Oct 2017, at 04:58, Colin Williams < >>>>>> colin.williams.seat...@gmail.com> wrote: >>>>>> >>>>>> I have a RichMapFunction and I'd like to ensure Meter fields are >>>>>> properly incremented. I've been trying to think of the best way to do >>>>>> this. >>>>>> Currently I think that I'd need to either implement my own reporter (or >>>>>> use >>>>>> JMX) and write to a socket, create a listener and wait for the reporter >>>>>> to >>>>>> send the message. >>>>>> >>>>>> Is this a good approach for writing the test, or should I be >>>>>> considering something else? >>>>>> >>>>> >>>>> >>>> >>>> >>> >> >> >> >> >
