I had sorted out how to run the topology in embedded mode. What wasn't
clear to me is how I can verify the watermark, but as per following thread
that can be done by inserting a process function:
On Wed, Feb 28, 2018 at 4:35 AM, Xingcan Cui <xingc...@gmail.com> wrote:
> Hi Thomas,
> generally speaking, if you want to test a whole job, just run the pipeline
> in your test case with a collection-based source and a result collecting
> sink. If your single operator tests passes while the integration test
> fails, maybe you should first check the timestamp / watermark assigners or
> the partitioning mechanisms used.
> > On 28 Feb 2018, at 5:46 AM, Thomas Weise <t...@apache.org> wrote:
> > Hi Xingcan,
> > thanks, this is a good way of testing an individual operator. I had
> > my own mock code to intercept source context and collect the results,
> > is a much better approach for operator testing.
> > I wonder how I can verify with an embedded Flink cluster though. Even
> > though my single operator test passes, the results are not emitted as
> > expected within a topology (not observed in the attached sink). What's
> > test approach there?
> > Thanks,
> > Thomas
> > On Wed, Feb 21, 2018 at 12:43 AM, Xingcan Cui <xingc...@gmail.com>
> >> Hi Thomas,
> >> some test cases in JoinHarnessTest <https://github.com/apache/
> >> flink/blob/release-1.4/flink-libraries/flink-table/src/
> >> test/scala/org/apache/flink/table/runtime/harness/
> >> show how to verify the emitted watermarks.
> >> Hope this helps.
> >> Best,
> >> Xingcan
> >>> On 21 Feb 2018, at 2:09 PM, Thomas Weise <t...@apache.org> wrote:
> >>> Hi,
> >>> I have a streaming integration test with two operators. A source that
> >> emits
> >>> records and watermarks, and a sink that collects the records. The
> >> topology
> >>> runs in embedded mode and the results are collected in memory. Now, in
> >>> addition to the records, I also want to verify that watermarks have
> >>> emitted. What's the recommended way of doing that?
> >>> Thanks