Hi Thomas, Normally the streaming job will terminate when the sources are exhausted and all records have been processed.
I assume you have some unbounded source(s), thus this doesn’t work for your case. We’d run into a similar situation with a streaming job that has iterations. Our solution was your option #1 below, where we created a modified version of LocalStreamEnvironment <https://raw.githubusercontent.com/ScaleUnlimited/flink-crawler/master/src/main/java/org/apache/flink/streaming/api/environment/LocalStreamEnvironmentWithAsyncExecution.java> that supports async execution. — Ken > On Feb 6, 2018, at 4:21 PM, Thomas Weise <t...@apache.org> wrote: > > Hi, > > I'm looking for an example of an integration test that runs a streaming job > and terminates when the expected result becomes available. I could think of > 2 approaches: > > 1. Modified version of LocalStreamEnvironment that executes the job > asynchronously and polls for the result or > > 2. Source that emits a final watermark that causes the topology to > terminate after the watermark has traversed the topology. Is that possible > with Flink? > > But probably this is a rather common testing need that's already solved?! > > Thanks, > Thomas -------------------------- Ken Krugler http://www.scaleunlimited.com custom big data solutions & training Hadoop, Cascading, Cassandra & Solr
