[
https://issues.apache.org/jira/browse/BEAM-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487192#comment-17487192
]
Beam JIRA Bot commented on BEAM-11095:
--------------------------------------
This issue was marked "stale-assigned" and has not received a public comment in
7 days. It is now automatically unassigned. If you are still working on it, you
can assign it to yourself again. Please also give an update about the status of
the work.
> Improve error messages from Invalid element types in emitter signatures
> -----------------------------------------------------------------------
>
> Key: BEAM-11095
> URL: https://issues.apache.org/jira/browse/BEAM-11095
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Priority: P3
> Labels: starter
>
> If a DoFn's `ProcessElement`uses an emitter `func(MyType)` where `MyType` is
> not a concrete type, (such as having function type fields, which are not
> valid) the pipeline can fail with incomprehensible errors that don't explain
> the problem correctly. eg.
> {{panic: inserting ParDo in scope *mypackage.studyGenerationElement
> constructing DoFn
> invalid DoFn
> caused by:
> method ProcessElement invalid
> caused by:
> bad parameter type for reflect.methodValueCall:
> func(*mypackage.studyGenerationElement)
> goroutine 740 [running]:
> panic(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/runtime/panic.go:1066+0x47e fp=0xc0061a18c8 sp=0xc0061a17d8
> pc=0x55e49ff4b67e
> testing.tRunner.func1.1(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/testing/testing.go:1076 +0x30d fp=0xc0061a1980 sp=0xc0061a18c8
> pc=0x55e4a097254d
> testing.tRunner.func1(0xc005046900)
> go/gc/src/testing/testing.go:1079 +0x41a fp=0xc0061a1a28 sp=0xc0061a1980
> pc=0x55e4a097297a
> runtime.call32(0x0, 0x55e4c3f0e388, 0xc0050dbf38, 0x800000008)
> go/gc/src/runtime/asm_amd64.s:540 +0x40 fp=0xc0061a1a58 sp=0xc0061a1a28
> pc=0x55e49ff86fc0
> runtime.reflectcallSave(0xc0061a1b88, 0x55e4c3f0e388, 0xc0050dbf38,
> 0x55e400000008)
> go/gc/src/runtime/panic.go:881 +0x5a fp=0xc0061a1a88 sp=0xc0061a1a58
> pc=0x55e49ff4b13a
> runtime.runOpenDeferFrame(0xc0035bc180, 0xc0050dbef0, 0xc0061a1c08)
> go/gc/src/runtime/panic.go:855 +0x2cd fp=0xc0061a1b18 sp=0xc0061a1a88
> pc=0x55e49ff4afed
> panic(0x55e4c34f3e80, 0xc005952680)
> go/gc/src/runtime/panic.go:969 +0x175 fp=0xc0061a1c08 sp=0xc0061a1b18
> pc=0x55e49ff4b375
> ...pkg/beam/beam.MustN(...)...pkg/beam/util.go:104
> ...pkg/beam/beam.ParDo(0xc00507d9e0, 0xc0052a7b30, 0x55e4c30faae0,
> 0xc0053244e0, 0xc0053245a0, 0x0, 0x0, 0x0, 0x0)
> ...pkg/beam/pardo.go:358 +0x135 fp=0xc0061a1c80 sp=0xc0061a1c08
> pc=0x55e4a0f3cc95
> }}
> In particular this message is generated by
> [funcx.New|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/funcx/fn.go#L323]
> and occurs because we don't maintain an error state when trying to analyze
> the DoFn.
> We would catch most errors like this if we provide a justification for it
> better.
> This will likely mean reimplementing the IsConcrete function and similar in
> terms of errors with the explanations, and having a justification handler to
> return that error from funcx.New if the type is invalid. This is subtle as
> while most errors are due to a type not being concrete, but the type might be
> nested in a emitter or iterator function.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
