Re: [Flink Unit Tests] Unit test for Flink streaming codes

2020-08-03 Thread Niels Basjes
Hi, If you look at the example from my own project you'll see that this is not a problem (if you test it like this). In some rare testing cases you may run into this problem and for those: have a look at what I did a few weeks ago for testing the PubSub connector:

Re: [Flink Unit Tests] Unit test for Flink streaming codes

2020-08-03 Thread Vijayendra Yadav
Thank you Arvid, David and Niels for your valuable inputs. One last Question: How do I terminate the flink streaming execution environment after the integration test is completed? Regards Vijay On Sun, Aug 2, 2020 at 12:27 PM David Anderson wrote: > Vijay, > > There's a section of the docs

Re: [Flink Unit Tests] Unit test for Flink streaming codes

2020-08-02 Thread David Anderson
Vijay, There's a section of the docs that describes some strategies for writing tests of various types, and it includes some Scala examples [1]. There are also some nice examples from Konstantin Knauf in [2], though they are mostly in Java. [1]

Re: [Flink Unit Tests] Unit test for Flink streaming codes

2020-08-02 Thread Arvid Heise
Hi Vijay, Any unit test of Flink operators is actually an IT case as it involves a large portion of the stack. A real unit test, would be over a factored out logic class. Similar to Niels, I'd recommend to use simple sources (env.fromElements) and sinks to inject the data and retrieve the data

Re: [Flink Unit Tests] Unit test for Flink streaming codes

2020-08-01 Thread Niels Basjes
No, I only have Java. On Fri, 31 Jul 2020, 21:57 Vijayendra Yadav, wrote: > Thank You Niels. Would you have something for the scala object class. Say > for example if I want to implement a unit test ( not integration test) for > below code or similar : > > >

Re: [Flink Unit Tests] Unit test for Flink streaming codes

2020-07-31 Thread Vijayendra Yadav
Thank You Niels. Would you have something for the scala object class. Say for example if I want to implement a unit test ( not integration test) for below code or similar :

Re: [Flink Unit Tests] Unit test for Flink streaming codes

2020-07-31 Thread Niels Basjes
Does this test in one of my own projects do what you are looking for? https://github.com/nielsbasjes/yauaa/blob/1e1ceb85c507134614186e3e60952112a2daabff/udfs/flink/src/test/java/nl/basjes/parse/useragent/flink/TestUserAgentAnalysisMapperClass.java#L107 On Fri, 31 Jul 2020, 20:20 Vijayendra

[Flink Unit Tests] Unit test for Flink streaming codes

2020-07-31 Thread Vijayendra Yadav
Hi Team, Looking for some help and reference code / material to implement unit tests of possible scenarios in Flink *streaming *Code that should assert specific cases. Regards, Vijay