[
https://issues.apache.org/jira/browse/BEAM-2333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16043628#comment-16043628
]
ASF GitHub Bot commented on BEAM-2333:
--------------------------------------
GitHub user kennknowles opened a pull request:
https://github.com/apache/beam/pull/3334
[BEAM-2333] Go to proto and back before running a pipeline in Java
DirectRunner
Be sure to do all of the following to help us incorporate your contribution
quickly and easily:
- [ ] Make sure the PR title is formatted like:
`[BEAM-<Jira issue #>] Description of pull request`
- [ ] Make sure tests pass via `mvn clean verify`.
- [ ] Replace `<Jira issue #>` in the title with the actual Jira issue
number, if there is one.
- [ ] If this contribution is large, please file an Apache
[Individual Contributor License
Agreement](https://www.apache.org/licenses/icla.pdf).
---
Not ready for review. Opening this to get ongoing Jenkins feedback.
- this includes #3330 adjusting DirectRunner overrides
- this requires #3329 to get additional inputs correctly set up I think
Notably, we don't need to port all evaluators because replacements occur
after deserialization, so things like `GroupByKey` are already mapped to
`DirectGroupByKey` so its evaluator remains unchanged.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kennknowles/beam Pipeline-rehydrate
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/3334.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #3334
----
commit 6b515e560aed423ea3e934046fa2098a21fe7ddb
Author: Kenneth Knowles <[email protected]>
Date: 2017-05-26T21:26:55Z
Port ViewOverrideFactory to SDK-agnostic APIs
commit be1a4e84ea08d1f950e5035624f7147a61376506
Author: Kenneth Knowles <[email protected]>
Date: 2017-05-26T21:27:23Z
Port DirectGroupByKey to SDK-agnostic APIs
commit 876cfa55502f9980066032ba9d6c458d41437bb2
Author: Kenneth Knowles <[email protected]>
Date: 2017-05-26T23:07:45Z
Port DirectRunner WriteFiles override to SDK-agnostic APIs
commit 183db6b1df0a2935a5323d6106d9e64096898de2
Author: Kenneth Knowles <[email protected]>
Date: 2017-06-08T20:39:32Z
Port DirectRunner TestStream override to SDK-agnostic APIs
commit c9efad0247b96ad89c4a32723742974bb3224c11
Author: Kenneth Knowles <[email protected]>
Date: 2017-06-08T20:44:52Z
Port DirectRunner ParDo overrides to SDK-agnostic APIs
commit 9b1c5ef125a9ab4929e4167d32a372a54cce7eb1
Author: Kenneth Knowles <[email protected]>
Date: 2017-06-08T20:46:18Z
Add more utilities to ParDoTranslation
commit 88c8df8228e40b4563bf15619f6a3a8f8d90f2f2
Author: Kenneth Knowles <[email protected]>
Date: 2017-06-08T21:27:02Z
Enable SplittableParDo on rehydrated ParDo transform
commit 7a206389a30449d97d3b40b30ef7d3d7e812971b
Author: Kenneth Knowles <[email protected]>
Date: 2017-06-08T23:21:57Z
fixup! Add more utilities to ParDoTranslation
commit e4191f9733d6185c43263516f831854919e53832
Author: Kenneth Knowles <[email protected]>
Date: 2017-05-26T18:22:50Z
Add Pipeline rehydration from proto
commit c106f1600d5dd5596f3e64fd35c699045ead4cad
Author: Kenneth Knowles <[email protected]>
Date: 2017-05-26T18:23:05Z
Dehydrate then rehydrate Pipeline before DirectRunner.run()
commit c76aafca18008e34dd16d292e02358326d888938
Author: Kenneth Knowles <[email protected]>
Date: 2017-06-07T20:58:11Z
Use dehydration-insensitive APIs in WindowEvaluatorFactory
commit 328790f280bf633801cd36aa55e3b3b2e1220666
Author: Kenneth Knowles <[email protected]>
Date: 2017-06-07T21:35:09Z
Use dehydration-insensitive APIs in ParDoEvaluatorFactory
----
> Rehydrate Pipeline from Runner API proto
> ----------------------------------------
>
> Key: BEAM-2333
> URL: https://issues.apache.org/jira/browse/BEAM-2333
> Project: Beam
> Issue Type: Bug
> Components: runner-core
> Reporter: Kenneth Knowles
> Assignee: Kenneth Knowles
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
