Hi list, Here's a concrete example of an issue that I've found when trying to unit test a flink app (scroll down to see the console output): https://gist.github.com/filipefigcorreia/fdf106eb3d40e035f82a
I am creating a custom datasink to collect the results, but the execution seems to finish before having the chance of actually collecting any results (race condition?). Any ideas of what I may be doing wrong? I have found this thread on the list that seems to describe a similar problem, although the solution of using "env.setParallelism(1)" didn't work for me: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Published-test-artifacts-for-flink-streaming-tp3379p3560.html Thanks in advance, Filipe On Sat, Jan 2, 2016 at 2:26 PM, Filipe Correia <[email protected]> wrote: > Hi Stephan, > > Yes, both generating the datastreams and verifying expectations. Is > the recommended way to create custom data sources and data sinks? I've > meanwhile started down this road, but still hoping for a better way. > > Filipe > > On Thu, Dec 31, 2015 at 3:20 PM, Stephan Ewen <[email protected]> wrote: >> Hi! >> >> Are you referring to testing streaming programs? >> What is the main obstacle for you? Generating test data streams? >> >> Thanks, >> Stephan >> >> >> On Thu, Dec 31, 2015 at 12:43 PM, Filipe Correia <[email protected]> >> wrote: >>> >>> Hi all, >>> >>> I'm finding it hard to unit test my Flink application. Are there any >>> guidelines / best practices for unit testing a Flink application, >>> especially for programming for the streaming API with Scala? >>> >>> Having a few good examples would also help a lot. >>> >>> I'm aware of flink-spector[1], and it looks great. Although the API is >>> not ready to be used from Scala yet, I've created my tests with Java >>> and managed to get it working for some very simple cases . But it does >>> still lack documentation / examples, so I'm having trouble using it >>> for most of the unit tests that I would like to create. >>> >>> Thanks, >>> >>> Filipe >>> >>> [1] https://github.com/ottogroup/flink-spector >> >>
