[
https://issues.apache.org/jira/browse/BEAM-1594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15893219#comment-15893219
]
ASF GitHub Bot commented on BEAM-1594:
--------------------------------------
GitHub user jkff opened a pull request:
https://github.com/apache/beam/pull/2146
[BEAM-1594] Supports DRAINING and DRAINED states
Dataflow's DRAINING is treated as RUNNING, and DRAINED as DONE.
R: @tgroh
CC: @dhalperi
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jkff/incubator-beam df-drained
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2146.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2146
----
commit be64925e3c0095d49760a84583677fdf54e47ef6
Author: Eugene Kirpichov <[email protected]>
Date: 2017-03-02T23:12:14Z
[BEAM-1594] Supports DRAINING and DRAINED states
Dataflow's DRAINING is treated as RUNNING, and DRAINED as DONE.
----
> JOB_STATE_DRAINED should be terminal for waitUntilFinish()
> ----------------------------------------------------------
>
> Key: BEAM-1594
> URL: https://issues.apache.org/jira/browse/BEAM-1594
> Project: Beam
> Issue Type: Bug
> Components: runner-dataflow
> Reporter: Eugene Kirpichov
> Assignee: Eugene Kirpichov
>
> Dataflow runner supports pipeline draining for streaming pipelines, which
> stops consuming input and waits for all watermarks to progress to infinity
> and then declares the pipeline complete. When watermarks progress to infinity
> after a drain request, the Dataflow runner will mark the job DRAINED (In the
> future, when watermarks progress to infinity without a drain request, it will
> mark the job DONE instead). JOB_STATE_DRAINED is logically a terminal state -
> no more data will be processed and the pipeline's resources are torn down -
> and waitUntilFinish() should terminate if the job enters this state.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
