[jira] [Commented] (BEAM-759) PipelineResult needs waitToFinish() and cancel()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15861881#comment-15861881
 ] 

ASF GitHub Bot commented on BEAM-759:
-

Github user asfgit closed the pull request at:

https://github.com/apache/beam/pull/1956


> PipelineResult needs waitToFinish() and cancel()
> 
>
> Key: BEAM-759
> URL: https://issues.apache.org/jira/browse/BEAM-759
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py
>Reporter: Ahmet Altay
>Assignee: Ahmet Altay
>Priority: Minor
>  Labels: sdk-consistency
> Fix For: Not applicable
>
>
> Java SDK, added waitToFinish() and cancel() to the PipelineResults, and as a 
> result were able to remove BlockingDataflowRunner.
> (See: https://issues.apache.org/jira/browse/BEAM-443)
> The same changes needs to happen in python sdk.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-759) PipelineResult needs waitToFinish() and cancel()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15858802#comment-15858802
 ] 

ASF GitHub Bot commented on BEAM-759:
-

GitHub user sb2nov opened a pull request:

https://github.com/apache/beam/pull/1956

[BEAM-759] Remove blocking dataflow runner

R: @aaltay PTAL 

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.txt).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sb2nov/beam BEAM-759-remove-blocking-runners

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1956.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1956


commit b61634b209b538c52a172c987f19a2e7d8cc0afd
Author: Sourabh Bajaj 
Date:   2017-02-09T00:55:01Z

Remove blocking dataflow runner




> PipelineResult needs waitToFinish() and cancel()
> 
>
> Key: BEAM-759
> URL: https://issues.apache.org/jira/browse/BEAM-759
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py
>Reporter: Ahmet Altay
>Assignee: Ahmet Altay
>Priority: Minor
>  Labels: sdk-consistency
> Fix For: Not applicable
>
>
> Java SDK, added waitToFinish() and cancel() to the PipelineResults, and as a 
> result were able to remove BlockingDataflowRunner.
> (See: https://issues.apache.org/jira/browse/BEAM-443)
> The same changes needs to happen in python sdk.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-759) PipelineResult needs waitToFinish() and cancel()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816494#comment-15816494
 ] 

ASF GitHub Bot commented on BEAM-759:
-

GitHub user aaltay reopened a pull request:

https://github.com/apache/beam/pull/1762

[BEAM-759] Implement wait_until_finish method for existing runners.

Implement wait_until_finish method for existing runners. 

Also defines the not implemented cancel() method and updates existing
usages to use wait_until_finish() instead of blocking runners.

Main changes are in the runners/ folder
runner.py - has the APIs
dataflow_runner.py, direct_runner.py modified to implement the API (moving 
the existing blocking code around.)

Rest of the changes are mechanical to mainly convert
p.run() to p.run().wait_until_finish() in tests and examples. Changed tests 
because they run validation after the run and need to block until completion. 
We may revert the changes in examples. I converted the because in the 
instructions we directed users to blocking runners before and this change keeps 
the behavior same.

I have started a local post commit run (not completed yet) and it was 
successful with the first few tests so far and the changes are same for all 
tests.

Remaining work after this PR:
- Removing BlockingDataflowRunner. It is not used after this change with 
the SDK code/examples/tests.
- Support for duration argument in wait_until_finish is missing.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/aaltay/incubator-beam expand

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/1762.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #1762


commit c6c78a3fe15910acf2d1cc0bf244af06794dea1f
Author: Ahmet Altay 
Date:   2017-01-10T02:23:20Z

Implement wait_until_finish method for existing runners.

Also defines the not implemented cancel() method and updates existing
usages to use wait_until_finish() instead of blocking runners.

commit a27773088d1ecad371725fdd5d27c5a1dd58a5cb
Author: Ahmet Altay 
Date:   2017-01-10T22:15:36Z

Fix wrong use of wait_until_complete in examples.




> PipelineResult needs waitToFinish() and cancel()
> 
>
> Key: BEAM-759
> URL: https://issues.apache.org/jira/browse/BEAM-759
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py
>Reporter: Ahmet Altay
>Priority: Minor
>  Labels: sdk-consistency
>
> Java SDK, added waitToFinish() and cancel() to the PipelineResults, and as a 
> result were able to remove BlockingDataflowRunner.
> (See: https://issues.apache.org/jira/browse/BEAM-443)
> The same changes needs to happen in python sdk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (BEAM-759) PipelineResult needs waitToFinish() and cancel()

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15816493#comment-15816493
 ] 

ASF GitHub Bot commented on BEAM-759:
-

Github user aaltay closed the pull request at:

https://github.com/apache/beam/pull/1762


> PipelineResult needs waitToFinish() and cancel()
> 
>
> Key: BEAM-759
> URL: https://issues.apache.org/jira/browse/BEAM-759
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py
>Reporter: Ahmet Altay
>Priority: Minor
>  Labels: sdk-consistency
>
> Java SDK, added waitToFinish() and cancel() to the PipelineResults, and as a 
> result were able to remove BlockingDataflowRunner.
> (See: https://issues.apache.org/jira/browse/BEAM-443)
> The same changes needs to happen in python sdk.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)