[
https://issues.apache.org/jira/browse/BEAM-3513?focusedWorklogId=90608&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-90608
]
ASF GitHub Bot logged work on BEAM-3513:
----------------------------------------
Author: ASF GitHub Bot
Created on: 12/Apr/18 22:17
Start Date: 12/Apr/18 22:17
Worklog Time Spent: 10m
Work Description: youngoli opened a new pull request #5119: [BEAM-3513]
Removing PrimitiveCombineGroupedValues override w/ FnAPI.
URL: https://github.com/apache/beam/pull/5119
When using the FnAPI, make it so that Combine.GroupedValues are not
replaced by the Dataflow specific PTransforms. The reason for this is
to allow the Combines to be translated to a Dataflow pipeline as the
constituent parts of the composite transform (in other words, send
Dataflow a GBK and a ParDo, instead of a CombineGroupedValues black
box).
This does not fully accomplish the final goal of portable combines since
this method doesn't do anything to help Combiner lifting, but it works
for the case of the basic Combine that has been implemented. It
successfully treats the Combine as a GBK and ParDo when not lifted.
Additional changes will need to be made to detect when we want to lift
Combines and in those situations handle that properly, which may mean
putting the Combines in the black box.
------------------------
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [x] Make sure there is a [JIRA
issue](https://issues.apache.org/jira/projects/BEAM/issues/) filed for the
change (usually before you start working on it). Trivial changes like typos do
not require a JIRA issue. Your pull request should address just this issue,
without pulling in other changes.
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue.
- [x] Write a pull request description that is detailed enough to
understand:
- [x] What the pull request does
- [x] Why it does it
- [x] How it does it
- [x] Why this approach
- [x] Each commit in the pull request should have a meaningful subject line
and body.
- [x] Run `mvn clean verify` to make sure basic checks pass. A more
thorough check will be performed on your pull request automatically.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 90608)
Time Spent: 10m
Remaining Estimate: 0h
> Use portable CombinePayload in Java DataflowRunner
> --------------------------------------------------
>
> Key: BEAM-3513
> URL: https://issues.apache.org/jira/browse/BEAM-3513
> Project: Beam
> Issue Type: Sub-task
> Components: runner-dataflow
> Reporter: Kenneth Knowles
> Assignee: Daniel Oliveira
> Priority: Major
> Labels: portability
> Time Spent: 10m
> Remaining Estimate: 0h
>
> The Java-specific blobs transmitted to Dataflow need more context, in the
> form of portability framework protos.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
