[
https://issues.apache.org/jira/browse/BEAM-4813?focusedWorklogId=128794&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-128794
]
ASF GitHub Bot logged work on BEAM-4813:
----------------------------------------
Author: ASF GitHub Bot
Created on: 30/Jul/18 16:47
Start Date: 30/Jul/18 16:47
Worklog Time Spent: 10m
Work Description: herohde commented on a change in pull request #5994:
[BEAM-4813] Refactor Go Dataflow runner and translation
URL: https://github.com/apache/beam/pull/5994#discussion_r206243940
##########
File path: sdks/go/test/regression/pardo.go
##########
@@ -22,23 +22,73 @@ import (
"github.com/apache/beam/sdks/go/pkg/beam/testing/ptest"
)
-func directCountFn(_ int, values func(*int) bool) (int, error) {
- sum := 0
- var i int
- for values(&i) {
- sum += i
- }
- return sum, nil
+func directFn(elm int) int {
+ return elm + 1
}
-func emitCountFn(_ int, values func(*int) bool, emit func(int)) error {
+// DirectParDo tests direct form output DoFns.
+func DirectParDo() *beam.Pipeline {
+ p, s := beam.NewPipelineWithRoot()
+
+ direct := beam.ParDo(s, directFn, beam.Create(s, 1, 2, 3))
+ passert.Sum(s, direct, "direct", 3, 9)
+
+ return p
+}
+
+func emitFn(elm int, emit func(int)) {
+ emit(elm + 1)
+}
+
+// EmitParDo tests emit form output DoFns.
+func EmitParDo() *beam.Pipeline {
+ p, s := beam.NewPipelineWithRoot()
+
+ emit := beam.ParDo(s, emitFn, beam.Create(s, 1, 2, 3))
+ passert.Sum(s, emit, "emit", 3, 9)
+
+ return p
+}
+
+func emit2Fn(elm int, emit, emit2 func(int)) {
+ emit(elm + 1)
+ emit2(elm + 2)
+}
+
+// MultiEmitParDo tests dubble emit form output DoFns.
Review comment:
Done
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 128794)
Time Spent: 3h 40m (was: 3.5h)
> Make Go Dataflow translation use protos directly
> ------------------------------------------------
>
> Key: BEAM-4813
> URL: https://issues.apache.org/jira/browse/BEAM-4813
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Henning Rohde
> Assignee: Henning Rohde
> Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> The Go SDK maintains 2 pipeline translations and keeps various tweaks in
> sync. It would be better to remove the Dataflow one and extract a more
> flexible (such as running as a separate proxy) translation from proto to
> v1beta3.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
