[
https://issues.apache.org/jira/browse/BEAM-10169?focusedWorklogId=445419&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-445419
]
ASF GitHub Bot logged work on BEAM-10169:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 13/Jun/20 19:59
Start Date: 13/Jun/20 19:59
Worklog Time Spent: 10m
Work Description: lostluck commented on a change in pull request #11976:
URL: https://github.com/apache/beam/pull/11976#discussion_r439765118
##########
File path: sdks/go/pkg/beam/pardo.go
##########
@@ -414,7 +414,45 @@ func ParDo6(s Scope, dofn interface{}, col PCollection,
opts ...Option) (PCollec
func ParDo7(s Scope, dofn interface{}, col PCollection, opts ...Option)
(PCollection, PCollection, PCollection, PCollection, PCollection, PCollection,
PCollection) {
ret := MustN(TryParDo(s, dofn, col, opts...))
if len(ret) != 7 {
- panic(fmt.Sprintf("expected 7 output. Found: %v", ret))
+ panic(ParDoErrorFormatter(dofn, ParDo7))
}
return ret[0], ret[1], ret[2], ret[3], ret[4], ret[5], ret[6]
}
+
+//ParDoErrorFormatter is a helper function to provide a more concise error
+// message to the users when a DoFn and its ParDo pairing is incorrect.
+func ParDoErrorFormatter(doFn interface{}, parDo interface{}) string {
+ doFnName := reflectx.FunctionName(doFn)
Review comment:
I see where the confusion is coming in. What I meant was to call
`graph.NewFn(doFn)` which will return the `graph.Fn` value which you can then
call `Name()` on.
https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/graph/fn.go#L78
At the point you're calling it, you already know it's going to be valid
(since it was called by the TryParDo earlier). Go isn't magic, and won't
change the type of an interface{} value unless assigned explicitly (which won't
happen in this scope). interface{} values are passed by value, so even if you
change the type them they'll remain the same in the outer scope: See an example
here https://play.golang.org/p/A5QDrGyRwUu
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 445419)
Time Spent: 3h 20m (was: 3h 10m)
> ParDo* functions should declare the correct output N in their error message
> ---------------------------------------------------------------------------
>
> Key: BEAM-10169
> URL: https://issues.apache.org/jira/browse/BEAM-10169
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Aaron Tillekeratne
> Priority: P3
> Labels: noob, starter
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> User report noted the confusion in the error if you use a DoFn with 0 outputs
> with beam.ParDo instead of beam.ParDo0.
> In that case, a panic stack trace is followed by the cryptic: "expected 1
> output. Found: []"
> We can do better.
> While we can't change the return signature dynamically (that's for ParDoN
> only), we can instead clearly indicate:
> * the DoFn in question.
> * the number of outputs the DoFn has
> * and recommend using ParDo0, ParDo, ParDo2,...ParDo7, or ParDoN, as
> appropriate.
> https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/pardo.go#L361
> would need to change as well as any of the specific cases that follow.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
