Sam Whittle created BEAM-3464:
---------------------------------
Summary: NexmarkLauncher should not cancel jobs once the input
events has been reached
Key: BEAM-3464
URL: https://issues.apache.org/jira/browse/BEAM-3464
Project: Beam
Issue Type: Bug
Components: testing
Reporter: Sam Whittle
Assignee: Sam Whittle
Priority: Trivial
NexmarkLauncher monitors the nexmark job and cancels it in several cases: if it
gets stuck, has fatal errors or the input processed counter equals the desired
amount.
Cancelling based upon the processed counter is problematic for the following
reasons:
- Dataflow (and other runners?) streaming counters are non-determinisitic and
thus the cancellation is deterministic
- The event counter is from the beginning stage of the pipeline and the
pipeline may have internally buffered data
This cancellation is also unnessary as once all the events are processed, the
nexmark source watermark goes to infinity and the pipeline will terminate due
to all watermarks going to infinity after it completes draining.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
