kennknowles opened a new issue, #19096:
URL: https://github.com/apache/beam/issues/19096
Presently, the Go SDK permits 3 kinds of functions for use in ParDos.
- Native functions
- Structural DoFns with correctly named methods, that reflect the bundle
lifecycle, or combine lifecycle (Setup StartBundle, FinishBundle,
ProcessElement, Teardown etc.) These also permit limited Stateful ParDos, such
as for caching expensive network responses for re-use when processing state.
- Dynamic functions (DynFns) which permit the use of closured global state,
and dynamic input types which are generated at pipeline runtime.
There's presently no way to generate a stateful function that relies on
closured global state and needs to be aware of the bundle lifecycle in the
worker harness. In short, there's no way to create a Dynamic Structural DoFn.
To implement this, in particular, [graph.DynFn
](http://example.com)(https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/graph/fn.go#L70)
will need to be modified so that multiple methods can be returned, likely as
methods on a struct, so that
[graph.NewFn](https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/graph/fn.go#L81)
can populate the
[graph.Fn.methods](https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/graph/fn.go#L40)
map correctly.
Imported from Jira
[BEAM-4531](https://issues.apache.org/jira/browse/BEAM-4531). Original Jira may
contain additional context.
Reported by: lostluck.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
