[ https://issues.apache.org/jira/browse/BEAM-1562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890861#comment-15890861 ]
Eugene Kirpichov commented on BEAM-1562: ---------------------------------------- See my comment on BEAM-849: as an alternative, we can use "watermarks progressed to infinity" as a termination signal. > Use a "signal" to stop streaming tests as they finish. > ------------------------------------------------------ > > Key: BEAM-1562 > URL: https://issues.apache.org/jira/browse/BEAM-1562 > Project: Beam > Issue Type: Improvement > Components: runner-spark > Reporter: Amit Sela > Assignee: Amit Sela > > Streaming tests use a timeout that has to take a large enough buffer to avoid > slow runtimes stopping before test completes. > We can introduce a "poison pill" based on an {{EndOfStream}} element that > would be counted in a Metrics/Aggregator to know all data was processed. > Another option would be to follow the slowest WM and stop once it hits > end-of-time. > This requires the Spark runner to stop blocking the execution of a streaming > pipeline until it's complete - which is something relevant to > {{PipelineResult}} (re)design that is being discussed in BEAM-849 -- This message was sent by Atlassian JIRA (v6.3.15#6346)