[
https://issues.apache.org/jira/browse/BEAM-646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15860539#comment-15860539
]
ASF GitHub Bot commented on BEAM-646:
-------------------------------------
GitHub user tgroh opened a pull request:
https://github.com/apache/beam/pull/1971
[BEAM-646] Construct Pipeline Nodes after Pipeline#run
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.txt).
---
Before-run is not compatible with Pipeline Surgery
Specifically, before-execution fails because the snapshot that is taken of
the Pipeline is before the PipelineRunner replaces all the nodes, and as a
result the final graph (after run() and the test complete successfully) has
different nodes, even though it is (in terms of pipeline behavior) the same
graph. Neither of the TestPipeline Enforcements require a pre-run traversal, so
this should not have observable effects outside of surgery.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/tgroh/beam test_pipeline_sequencing
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/1971.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1971
----
commit af403d7d1b2cd4a404d882f8a3300b7558f4c3f1
Author: Thomas Groh <[email protected]>
Date: 2017-02-09T19:39:08Z
Construct Pipeline Nodes after Pipeline#run
Before-run is not compatible with Pipeline Surgery
----
> Get runners out of the apply()
> ------------------------------
>
> Key: BEAM-646
> URL: https://issues.apache.org/jira/browse/BEAM-646
> Project: Beam
> Issue Type: Improvement
> Components: beam-model-runner-api, sdk-java-core
> Reporter: Kenneth Knowles
> Assignee: Thomas Groh
>
> Right now, the runner intercepts calls to apply() and replaces transforms as
> we go. This means that there is no "original" user graph. For portability and
> misc architectural benefits, we would like to build the original graph first,
> and have the runner override later.
> Some runners already work in this manner, but we could integrate it more
> smoothly, with more validation, via some handy APIs on e.g. the Pipeline
> object.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
