Hi Pablo, thx for the pointers. I'll have a look into it and let you know.
Regards, michel On Wed, May 10, 2017 at 3:18 AM, Pablo Estrada <[email protected]> wrote: > Hi Michael, > I'm sorry. I see I did not read your first email properly. > > There are a couple places in the core SDK or runner code and tests that > used to use aggregators, and now use metrics. There are two reasonable > options for this: > 1. In [1], the test sets up the metrics global environment by setting the > current container (e.g. MetricsEnvironment.setCurrentContainer(new > MetricsContainer("anystep"));), and the LateDataFilter uses metrics > normally[2], by creating a counter that relies on the environment set up in > the test. > > 2. If you'd rather not rely on setting up a global environment, you can > use CounterCell, and pass it in to your test. In [3] it's not a test, but a > CounterCell is still created to keep internal statistics, and later its > value is checked [4]. As a note, you may note in [3] that CounterCells are > a bit quirky to create, as we did not intend for external users to be able > to create them. > > Let me know if these suggestions are helpful. > Best > -P. > > [1] - https://github.com/apache/beam/blob/master/runners/core- > java/src/test/java/org/apache/beam/runners/core/ > LateDataDroppingDoFnRunnerTest.java#L61 > [2] - https://github.com/apache/beam/blob/master/runners/core- > java/src/main/java/org/apache/beam/runners/core/ > LateDataDroppingDoFnRunner.java#L96 > [3] - https://github.com/apache/beam/blob/master/runners/ > spark/src/main/java/org/apache/beam/runners/spark/stateful/ > SparkGroupAlsoByWindowViaWindowSet.java#L210 > [4] - https://github.com/apache/beam/blob/master/runners/ > spark/src/main/java/org/apache/beam/runners/spark/stateful/ > SparkGroupAlsoByWindowViaWindowSet.java#L326 > > On Tue, May 9, 2017 at 4:33 PM Michael Luckey <[email protected]> wrote: > >> Hi Pablo, >> >> thanks for your help! We certainly could change our testing code and >> involve execution of a pipeline during tests. >> >> But currently we are leveraging DoFnTester, i.e. scoping our tests to the >> DoFn only, which means, there is neither a pipeline nor a pipeline result >> involved, which i could call upon. >> >> It might be a bad idea trying to test counters on this basis, but as it >> was supported previously i thought we might have overlooked an API for >> accessing these metrics somehow within DoFnTesting. Not sure, wether it >> makes sense for the DoFnTester to somehow provide Metrics-Support to enable >> this kind of tests. I certainly do not like the idea to much starting to do >> some mocking of the metrics api within my test implementation. >> >> Regards, >> >> michel >> >> >> On Wed, May 10, 2017 at 1:10 AM, Pablo Estrada <[email protected]> >> wrote: >> >>> Hi Michael, >>> For the Metrics API, the way to programatically query the value of a >>> metric is by using the MetricResults.queryMetrics method. You get the >>> MetricResults object from the PipelineResult object, and query it like so: >>> >>> PipelineResult res = p.run(); >>> MetricQueryResults metricResult = res.metrics().queryMetrics(....); >>> >>> The queryMetrics method takes in a MetricsFilter instance. >>> >>> Not all runners support this operation. For the dataflow runner, PR >>> 2896[1] should add it. >>> >>> Let me know if you need more help with this. >>> Best >>> -P. >>> >>> [1] - https://github.com/apache/beam/pull/2896 >>> >>> On Tue, May 9, 2017 at 3:48 PM Michael Luckey <[email protected]> >>> wrote: >>> >>>> Hi, >>>> >>>> currently we are evaluating a migration from 0.6.0 to current. We >>>> encountered the following problem, which we currently not sure, how to best >>>> solve. >>>> >>>> Say we have a DoFn, which is using Aggregators, e.g. >>>> >>>> ctr = createAggregator("someCounter", Sum.ofLongs()); >>>> >>>> We were testing them with DoFn-Tester like >>>> >>>> final DoFnTester<Record, Record> doFnTester = >>>> DoFnTester.of(fixture); >>>> doFnTester.processElement(input); >>>> >>>> assertThat(doFnTester.getAggregatorValue(fixture.ctr).longValue(), >>>> is(1L)); >>>> >>>> As aggregators are removed now from the codebase, we are considering >>>> using Metrics instead. But we did not find an equivalent to the >>>> getAggregatorValue method on DoFnTester. >>>> >>>> Any suggestion how we could keep our counters tested within a unit test >>>> based on DoFnTester? Or do we have to find a completely different solution? >>>> Are we doing something completely wrong trying to test correct workings of >>>> our counters with this approach? >>>> >>>> Regards, >>>> >>>> michel >>>> >>>> >>>> >>
