Hi Thomas, generally speaking, if you want to test a whole job, just run the pipeline in your test case with a collection-based source and a result collecting sink. If your single operator tests passes while the integration test fails, maybe you should first check the timestamp / watermark assigners or the partitioning mechanisms used.
Best, Xingcan > On 28 Feb 2018, at 5:46 AM, Thomas Weise <t...@apache.org> wrote: > > Hi Xingcan, > > thanks, this is a good way of testing an individual operator. I had written > my own mock code to intercept source context and collect the results, this > is a much better approach for operator testing. > > I wonder how I can verify with an embedded Flink cluster though. Even > though my single operator test passes, the results are not emitted as > expected within a topology (not observed in the attached sink). What's the > test approach there? > > Thanks, > Thomas > > > On Wed, Feb 21, 2018 at 12:43 AM, Xingcan Cui <xingc...@gmail.com> wrote: > >> Hi Thomas, >> >> some test cases in JoinHarnessTest <https://github.com/apache/ >> flink/blob/release-1.4/flink-libraries/flink-table/src/ >> test/scala/org/apache/flink/table/runtime/harness/JoinHarnessTest.scala> >> show how to verify the emitted watermarks. >> >> Hope this helps. >> >> Best, >> Xingcan >> >>> On 21 Feb 2018, at 2:09 PM, Thomas Weise <t...@apache.org> wrote: >>> >>> Hi, >>> >>> I have a streaming integration test with two operators. A source that >> emits >>> records and watermarks, and a sink that collects the records. The >> topology >>> runs in embedded mode and the results are collected in memory. Now, in >>> addition to the records, I also want to verify that watermarks have been >>> emitted. What's the recommended way of doing that? >>> >>> Thanks >> >>
