[
https://issues.apache.org/jira/browse/BEAM-4531?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17475687#comment-17475687
]
Robert Burke commented on BEAM-4531:
------------------------------------
I still have a prototype of this somewhere that a pandemic interrupted.
> [Go SDK] Allow Dynamic Structural DoFns
> ---------------------------------------
>
> Key: BEAM-4531
> URL: https://issues.apache.org/jira/browse/BEAM-4531
> Project: Beam
> Issue Type: Improvement
> Components: sdk-go
> Reporter: Robert Burke
> Priority: P3
>
> Presently, the Go SDK permits 3 kinds of functions for use in ParDos.
> # Native functions
> # Structural DoFns with correctly named methods, that reflect the bundle
> lifecycle, or combine lifecycle (Setup StartBundle, FinishBundle,
> ProcessElement, Teardown etc.) These also permit limited Stateful ParDos,
> such as for caching expensive network responses for re-use when processing
> state.
> # Dynamic functions (DynFns) which permit the use of closured global state,
> and dynamic input types which are generated at pipeline runtime.
> There's presently no way to generate a stateful function that relies on
> closured global state and needs to be aware of the bundle lifecycle in the
> worker harness. In short, there's no way to create a Dynamic Structural DoFn.
> To implement this, in particular, [graph.DynFn
> |http://example.com](https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/graph/fn.go#L70)
> will need to be modified so that multiple methods can be returned, likely as
> methods on a struct, so that
> [graph.NewFn](https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/graph/fn.go#L81)
> can populate the
> [graph.Fn.methods](https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/graph/fn.go#L40)
> map correctly.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
