[
https://issues.apache.org/jira/browse/BEAM-1050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15701400#comment-15701400
]
ASF GitHub Bot commented on BEAM-1050:
--------------------------------------
GitHub user staslev opened a pull request:
https://github.com/apache/incubator-beam/pull/1438
[BEAM-1050] Fixing pipeline's result state in streaming mode
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
R: @amitsela
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/staslev/incubator-beam fixing-BEAM-1050
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-beam/pull/1438.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1438
----
commit 412cab38c6ff4ced7a313b9c51e9e383a7df8392
Author: Stas Levin <stasle...@gmail.com>
Date: 2016-11-28T09:11:10Z
Handled some exception flows in waitUntilFinish() so that the pipeline's
state is correctly maintained, there's more work to be done in that department.
Also, added waitUntilFinish() to a bunch of pipeline.run() instances which
in now needed as part of the move to a more strict waitUntilFinish() semantics.
----
> PipelineResult.State is not set to FAILED in when a streaming job fails
> -----------------------------------------------------------------------
>
> Key: BEAM-1050
> URL: https://issues.apache.org/jira/browse/BEAM-1050
> Project: Beam
> Issue Type: Bug
> Components: runner-spark
> Reporter: Stas Levin
> Assignee: Stas Levin
> Priority: Minor
>
> In case of failure, {{SteamingContext#awaitTerminationOrTimeout}} and
> {{SteamingContext#awaitTermination}} will both throw an exception, and so
> {{state = State.DONE}} will not be executed in the code block below.
> In addition, it would probably make sense to set {{state = State.FAILED}} in
> cases where an exception takes place.
> {code:java}
> if (isStreamingPipeline()) {
> // stop streaming context
> if (timeout > 0) {
> jssc.awaitTerminationOrTimeout(timeout);
> } else {
> jssc.awaitTermination();
> }
> // stop streaming context gracefully, so checkpointing (and other
> computations) get to
> // finish before shutdown.
> jssc.stop(false, gracefully);
> }
> state = State.DONE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
