[
https://issues.apache.org/jira/browse/BEAM-115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979640#comment-15979640
]
ASF GitHub Bot commented on BEAM-115:
-------------------------------------
GitHub user robertwb opened a pull request:
https://github.com/apache/beam/pull/2644
[BEAM-115] Fn API support for Python
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
Travis-CI on your fork and ensure the whole test matrix passes).
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf).
---
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/robertwb/incubator-beam fn-api
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2644.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2644
----
commit 4f5c5241949473d81b18cb034640189ea98c49df
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T16:59:52Z
Add instructions to regenerate Python proto wrappers.
commit f4f94db5bb3b64c015fd95de64bafd34770aaa21
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T17:00:20Z
Generate python proto wrappers for runner and fn API.
commit 6f0e487749b777c5606a69865b7e47134d878a58
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T18:04:03Z
Ignore generated files for linter.
commit aa1425c86f5def43ff9032b8666352b747d20440
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T18:09:03Z
Ignore generated files in rat plugin.
commit 91428b06f8d4ced0bd3965801920cd94ee8298c6
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T19:15:02Z
Add apache licence to generated files.
commit 27f718e89d1829b5b217f263d4b4deb8832c947d
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T20:59:25Z
Add fn api runner.
commit f987622d524fa4245089f82fbd150c0cf9ae380d
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T21:20:29Z
Restore __init__.py
commit 45ccb571aed5e51ef893428c2a5350ba01864aac
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T20:44:33Z
Add runner core files.
commit c6310ad8236578c05432244ced40fe572c2d71a7
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T21:23:30Z
move files around
commit 0f9de53c7007e200e5f04c9a30349313b2bdff39
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T21:27:14Z
more moving
commit 2f5c518bff028878b7d18a30141beea734cb36a0
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T21:43:28Z
Rename runners.
commit 85b2172c1d8d6a5797015c15cd5a301de4a252c6
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T21:53:26Z
cythonization works
commit 9f1177db48fa0230fab463cb7a5d784b96c9e084
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T22:03:52Z
test module import renames
commit a0e8d792bbfac4550e4399b29c9004f67b82cbd0
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T22:06:30Z
remove google3s
commit dfc0b2f3ca04594a2167eaadc7882ca257cdca5f
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T22:07:37Z
move sdk_harness to sdk_worker
commit 8b08216c54e65fd0eb5a6c9588405cb4301267b2
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-20T22:09:33Z
Remove unused end_time from statesampler.
commit 011dec575a4066eb7e50c0bff2bad78a9cadd449
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T16:59:52Z
Add instructions to regenerate Python proto wrappers.
commit 4a74cdd528c6e9800c66152f9498592f51b59248
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T17:00:20Z
Generate python proto wrappers for runner and fn API.
commit 5dcd25d144f784a5b9bd6bd668c2ceb8bc2680f3
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T18:04:03Z
Ignore generated files for linter.
commit 5697ecdca1c14ce926f0be63399df77609c12aaf
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T18:09:03Z
Ignore generated files in rat plugin.
commit 57a16aa810f5e4ef14c5f2ced901c3bb6fbc7b1a
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T19:15:02Z
Add apache licence to generated files.
commit a8456be3830869e5b51e24970655819bc832bd79
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T19:36:27Z
implement portpicker, fix more imports
commit c4633f174d436304da142606145a4827c9da7f4e
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T20:04:22Z
portpicker fixes
commit f349f14e49e03b675ad7f7ab06e2c871c9d5eae4
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T20:04:35Z
more import fixes
commit cc1dc88e83a5ed7ba0c04fbd10943ec98d920c73
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T21:22:07Z
Adapt to PR #2505 changes to protos.
commit cb09e464c3da6358b203fad681bba41fb3d4a837
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T21:22:37Z
Merge branch 'gen-protos' into fn-api
commit a11349156833c9fb1df0263c9289f9bebe1c51d3
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T21:38:12Z
disble simple state creation
commit b64991b2f08a5f21da440f86f90870e323ae0407
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T21:43:28Z
more package renames
commit c857789993cc8333c7d27c3527701e794fd3615d
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T21:50:00Z
remove google3, fix imports
commit acc28a2b75cb338ed6f1f7e9edebec646c1e29db
Author: Robert Bradshaw <[email protected]>
Date: 2017-04-21T21:52:25Z
fix more imports, google3 in portability dir
----
> Beam Runner API
> ---------------
>
> Key: BEAM-115
> URL: https://issues.apache.org/jira/browse/BEAM-115
> Project: Beam
> Issue Type: Improvement
> Components: beam-model-runner-api
> Reporter: Kenneth Knowles
> Assignee: Kenneth Knowles
>
> The PipelineRunner API from the SDK is not ideal for the Beam technical
> vision.
> It has technical limitations:
> - The user's DAG (even including library expansions) is never explicitly
> represented, so it cannot be analyzed except incrementally, and cannot
> necessarily be reconstructed (for example, to display it!).
> - The flattened DAG of just primitive transforms isn't well-suited for
> display or transform override.
> - The TransformHierarchy isn't well-suited for optimizations.
> - The user must realistically pre-commit to a runner, and its configuration
> (batch vs streaming) prior to graph construction, since the runner will be
> modifying the graph as it is built.
> - It is fairly language- and SDK-specific.
> It has usability issues (these are not from intuition, but derived from
> actual cases of failure to use according to the design)
> - The interleaving of apply() methods in PTransform/Pipeline/PipelineRunner
> is confusing.
> - The TransformHierarchy, accessible only via visitor traversals, is
> cumbersome.
> - The staging of construction-time vs run-time is not always obvious.
> These are just examples. This ticket tracks designing, coming to consensus,
> and building an API that more simply and directly supports the technical
> vision.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
