[
https://issues.apache.org/jira/browse/BEAM-11095?focusedWorklogId=724243&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-724243
]
ASF GitHub Bot logged work on BEAM-11095:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Feb/22 02:54
Start Date: 10/Feb/22 02:54
Worklog Time Spent: 10m
Work Description: lostluck merged pull request #16794:
URL: https://github.com/apache/beam/pull/16794
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 724243)
Time Spent: 3h (was: 2h 50m)
> Improve error messages from Invalid element types in emitter signatures
> -----------------------------------------------------------------------
>
> Key: BEAM-11095
> URL: https://issues.apache.org/jira/browse/BEAM-11095
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Assignee: Danny McCormick
> Priority: P3
> Labels: starter
> Time Spent: 3h
> Remaining Estimate: 0h
>
> If a DoFn's `ProcessElement`uses an emitter `func(MyType)` where `MyType` is
> not a concrete type, (such as having function type fields, which are not
> valid) the pipeline can fail with incomprehensible errors that don't explain
> the problem correctly. eg.
> {{panic: inserting ParDo in scope *mypackage.studyGenerationElement
> constructing DoFn
> invalid DoFn
> caused by:
> method ProcessElement invalid
> caused by:
> bad parameter type for reflect.methodValueCall:
> func(*mypackage.studyGenerationElement)
> goroutine 740 [running]:
> panic(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/runtime/panic.go:1066+0x47e fp=0xc0061a18c8 sp=0xc0061a17d8
> pc=0x55e49ff4b67e
> testing.tRunner.func1.1(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/testing/testing.go:1076 +0x30d fp=0xc0061a1980 sp=0xc0061a18c8
> pc=0x55e4a097254d
> testing.tRunner.func1(0xc005046900)
> go/gc/src/testing/testing.go:1079 +0x41a fp=0xc0061a1a28 sp=0xc0061a1980
> pc=0x55e4a097297a
> runtime.call32(0x0, 0x55e4c3f0e388, 0xc0050dbf38, 0x800000008)
> go/gc/src/runtime/asm_amd64.s:540 +0x40 fp=0xc0061a1a58 sp=0xc0061a1a28
> pc=0x55e49ff86fc0
> runtime.reflectcallSave(0xc0061a1b88, 0x55e4c3f0e388, 0xc0050dbf38,
> 0x55e400000008)
> go/gc/src/runtime/panic.go:881 +0x5a fp=0xc0061a1a88 sp=0xc0061a1a58
> pc=0x55e49ff4b13a
> runtime.runOpenDeferFrame(0xc0035bc180, 0xc0050dbef0, 0xc0061a1c08)
> go/gc/src/runtime/panic.go:855 +0x2cd fp=0xc0061a1b18 sp=0xc0061a1a88
> pc=0x55e49ff4afed
> panic(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/runtime/panic.go:969 +0x175 fp=0xc0061a1c08 sp=0xc0061a1b18
> pc=0x55e49ff4b375
> ...pkg/beam/beam.MustN(...)...pkg/beam/util.go:104
> ...pkg/beam/beam.ParDo(0xc00507d9e0, 0xc0052a7b30, 0x55e4c30faae0,
> 0xc0053244e0, 0xc0053245a0, 0x0, 0x0, 0x0, 0x0)
> ...pkg/beam/pardo.go:358 +0x135 fp=0xc0061a1c80 sp=0xc0061a1c08
> pc=0x55e4a0f3cc95
> }}
> In particular this message is generated by
> [funcx.New|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/funcx/fn.go#L323]
> and occurs because we don't maintain an error state when trying to analyze
> the DoFn.
> We would catch most errors like this if we provide a justification for it
> better.
> This will likely mean reimplementing the IsConcrete function and similar in
> terms of errors with the explanations, and having a justification handler to
> return that error from funcx.New if the type is invalid. This is subtle as
> while most errors are due to a type not being concrete, but the type might be
> nested in a emitter or iterator function.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
