Hi Pablo,
we ended up to follow the path with setup of the global environment.
Therefore we implemented a simple junit TestRule, which is used something
along the following:
@Rule TestMetrics metrics = new TestMetrics();
....
@Test
public void invalids() {
final DoFnTester<InputT, OutputT> doFnTester =
DoFnTester.of(fixture);
doFnTester.processElement(input);
assertThat(metrics,counterValue(fixture.ctr), is(1L));
}
Thats pretty close to what we did before and seems to be working very well.
So again, thanks a lot for your help, very much appreciated,
michel
On Wed, May 10, 2017 at 11:01 AM, Michael Luckey <[email protected]>
wrote:
> Hi Pablo,
>
> thx for the pointers. I'll have a look into it and let you know.
>
> Regards,
>
> michel
>
> On Wed, May 10, 2017 at 3:18 AM, Pablo Estrada <[email protected]> wrote:
>
>> Hi Michael,
>> I'm sorry. I see I did not read your first email properly.
>>
>> There are a couple places in the core SDK or runner code and tests that
>> used to use aggregators, and now use metrics. There are two reasonable
>> options for this:
>> 1. In [1], the test sets up the metrics global environment by setting the
>> current container (e.g. MetricsEnvironment.setCurrentContainer(new
>> MetricsContainer("anystep"));), and the LateDataFilter uses metrics
>> normally[2], by creating a counter that relies on the environment set up in
>> the test.
>>
>> 2. If you'd rather not rely on setting up a global environment, you can
>> use CounterCell, and pass it in to your test. In [3] it's not a test, but a
>> CounterCell is still created to keep internal statistics, and later its
>> value is checked [4]. As a note, you may note in [3] that CounterCells are
>> a bit quirky to create, as we did not intend for external users to be able
>> to create them.
>>
>> Let me know if these suggestions are helpful.
>> Best
>> -P.
>>
>> [1] - https://github.com/apache/beam/blob/master/runners/core-java
>> /src/test/java/org/apache/beam/runners/core/LateDataDropping
>> DoFnRunnerTest.java#L61
>> [2] - https://github.com/apache/beam/blob/master/runners/core-java
>> /src/main/java/org/apache/beam/runners/core/LateDataDropping
>> DoFnRunner.java#L96
>> [3] - https://github.com/apache/beam/blob/master/runners/spark/
>> src/main/java/org/apache/beam/runners/spark/stateful/SparkGr
>> oupAlsoByWindowViaWindowSet.java#L210
>> [4] - https://github.com/apache/beam/blob/master/runners/spark/
>> src/main/java/org/apache/beam/runners/spark/stateful/SparkGr
>> oupAlsoByWindowViaWindowSet.java#L326
>>
>> On Tue, May 9, 2017 at 4:33 PM Michael Luckey <[email protected]>
>> wrote:
>>
>>> Hi Pablo,
>>>
>>> thanks for your help! We certainly could change our testing code and
>>> involve execution of a pipeline during tests.
>>>
>>> But currently we are leveraging DoFnTester, i.e. scoping our tests to
>>> the DoFn only, which means, there is neither a pipeline nor a pipeline
>>> result involved, which i could call upon.
>>>
>>> It might be a bad idea trying to test counters on this basis, but as it
>>> was supported previously i thought we might have overlooked an API for
>>> accessing these metrics somehow within DoFnTesting. Not sure, wether it
>>> makes sense for the DoFnTester to somehow provide Metrics-Support to enable
>>> this kind of tests. I certainly do not like the idea to much starting to do
>>> some mocking of the metrics api within my test implementation.
>>>
>>> Regards,
>>>
>>> michel
>>>
>>>
>>> On Wed, May 10, 2017 at 1:10 AM, Pablo Estrada <[email protected]>
>>> wrote:
>>>
>>>> Hi Michael,
>>>> For the Metrics API, the way to programatically query the value of a
>>>> metric is by using the MetricResults.queryMetrics method. You get the
>>>> MetricResults object from the PipelineResult object, and query it like so:
>>>>
>>>> PipelineResult res = p.run();
>>>> MetricQueryResults metricResult = res.metrics().queryMetrics(....);
>>>>
>>>> The queryMetrics method takes in a MetricsFilter instance.
>>>>
>>>> Not all runners support this operation. For the dataflow runner, PR
>>>> 2896[1] should add it.
>>>>
>>>> Let me know if you need more help with this.
>>>> Best
>>>> -P.
>>>>
>>>> [1] - https://github.com/apache/beam/pull/2896
>>>>
>>>> On Tue, May 9, 2017 at 3:48 PM Michael Luckey <[email protected]>
>>>> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> currently we are evaluating a migration from 0.6.0 to current. We
>>>>> encountered the following problem, which we currently not sure, how to
>>>>> best
>>>>> solve.
>>>>>
>>>>> Say we have a DoFn, which is using Aggregators, e.g.
>>>>>
>>>>> ctr = createAggregator("someCounter", Sum.ofLongs());
>>>>>
>>>>> We were testing them with DoFn-Tester like
>>>>>
>>>>> final DoFnTester<Record, Record> doFnTester =
>>>>> DoFnTester.of(fixture);
>>>>> doFnTester.processElement(input);
>>>>>
>>>>>
>>>>> assertThat(doFnTester.getAggregatorValue(fixture.ctr).longValue(),
>>>>> is(1L));
>>>>>
>>>>> As aggregators are removed now from the codebase, we are considering
>>>>> using Metrics instead. But we did not find an equivalent to the
>>>>> getAggregatorValue method on DoFnTester.
>>>>>
>>>>> Any suggestion how we could keep our counters tested within a unit
>>>>> test based on DoFnTester? Or do we have to find a completely different
>>>>> solution? Are we doing something completely wrong trying to test correct
>>>>> workings of our counters with this approach?
>>>>>
>>>>> Regards,
>>>>>
>>>>> michel
>>>>>
>>>>>
>>>>>
>>>
>
