[
https://issues.apache.org/jira/browse/BEAM-802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969637#comment-15969637
]
ASF GitHub Bot commented on BEAM-802:
-------------------------------------
GitHub user aaltay opened a pull request:
https://github.com/apache/beam/pull/2545
[BEAM-802] Python templates
Re-reverts the python templates. First two commits are clean reverts of the
original commits.
The last commit:
- Removes the concept of options id. (Except for a few places marked with
TODO. Dataflow workers depend on those paths for now, I will remove them in a
follow up PR.)
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/aaltay/beam templ
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2545.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2545
----
commit 7a94697a57279b55f7c46bb74f7c4f9812900e62
Author: Ahmet Altay <[email protected]>
Date: 2017-04-14T21:55:25Z
Revert "Revert "Revert "Revert "Add ValueProvider class for FileBasedSource
I/O Transforms""""
This reverts commit 0f5c36398aa321d25b3a045cedc29203d75956f7.
commit b7aee2d15cdc98c38a8de8ed4f5a23c42af4542d
Author: Ahmet Altay <[email protected]>
Date: 2017-04-14T21:55:50Z
Revert "Revert "Throw specialized exception in value providers""
This reverts commit 4854291fe38df7b624446311956d0ae0677f6b8c.
commit 2b064906e07a89e0d4b031b8f14ba8fa7009249d
Author: Ahmet Altay <[email protected]>
Date: 2017-04-14T22:33:39Z
Remove options_id concept from templated runs.
----
> Support Dynamic PipelineOptions for python
> ------------------------------------------
>
> Key: BEAM-802
> URL: https://issues.apache.org/jira/browse/BEAM-802
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py
> Reporter: María GH
> Assignee: María GH
> Priority: Minor
> Original Estimate: 1,680h
> Remaining Estimate: 1,680h
>
> Goal: Enable users to run pipelines from templates filled via CL (pipeline
> options)
> Background: Currently, the Runner creates the JSON pipeline description which
> can be sent to the worker as is, since everything is already defined there
> (with links to gs:// for input and binaries). With the parametrized approach,
> those descriptions are empty and filled by the user or defaulted, so the
> pipeline needs to be stored somewhere first until the values become available.
> Tasks:
> 1- Create template-style pipeline description (TemplateRunner)
> The graph description is now a template (some parts are not filled) that
> needs to be saved.
> 2- Define values to inject to the template (ValueProviders API)
> The placeholders can be filled with default values (static) or with dynamic
> key/value pairs provided at runtime (dynamic)
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
