Hi Pablo, thanks for your help! We certainly could change our testing code and involve execution of a pipeline during tests.
But currently we are leveraging DoFnTester, i.e. scoping our tests to the DoFn only, which means, there is neither a pipeline nor a pipeline result involved, which i could call upon. It might be a bad idea trying to test counters on this basis, but as it was supported previously i thought we might have overlooked an API for accessing these metrics somehow within DoFnTesting. Not sure, wether it makes sense for the DoFnTester to somehow provide Metrics-Support to enable this kind of tests. I certainly do not like the idea to much starting to do some mocking of the metrics api within my test implementation. Regards, michel On Wed, May 10, 2017 at 1:10 AM, Pablo Estrada <[email protected]> wrote: > Hi Michael, > For the Metrics API, the way to programatically query the value of a > metric is by using the MetricResults.queryMetrics method. You get the > MetricResults object from the PipelineResult object, and query it like so: > > PipelineResult res = p.run(); > MetricQueryResults metricResult = res.metrics().queryMetrics(....); > > The queryMetrics method takes in a MetricsFilter instance. > > Not all runners support this operation. For the dataflow runner, PR > 2896[1] should add it. > > Let me know if you need more help with this. > Best > -P. > > [1] - https://github.com/apache/beam/pull/2896 > > On Tue, May 9, 2017 at 3:48 PM Michael Luckey <[email protected]> wrote: > >> Hi, >> >> currently we are evaluating a migration from 0.6.0 to current. We >> encountered the following problem, which we currently not sure, how to best >> solve. >> >> Say we have a DoFn, which is using Aggregators, e.g. >> >> ctr = createAggregator("someCounter", Sum.ofLongs()); >> >> We were testing them with DoFn-Tester like >> >> final DoFnTester<Record, Record> doFnTester = >> DoFnTester.of(fixture); >> doFnTester.processElement(input); >> >> assertThat(doFnTester.getAggregatorValue(fixture.ctr).longValue(), >> is(1L)); >> >> As aggregators are removed now from the codebase, we are considering >> using Metrics instead. But we did not find an equivalent to the >> getAggregatorValue method on DoFnTester. >> >> Any suggestion how we could keep our counters tested within a unit test >> based on DoFnTester? Or do we have to find a completely different solution? >> Are we doing something completely wrong trying to test correct workings of >> our counters with this approach? >> >> Regards, >> >> michel >> >> >>
