Robert Burke created BEAM-5030:
----------------------------------
Summary: Consolidate defer overhead per bundle
Key: BEAM-5030
URL: https://issues.apache.org/jira/browse/BEAM-5030
Project: Beam
Issue Type: Sub-task
Components: sdk-go
Reporter: Robert Burke
Assignee: Robert Burke
At present, reflectx.CallNoPanic is invoked for every element, and addsÂ
recovery defers for each element. Defers are cheap, but not "per element"
cheap. In a simple pipeline (a single ParDo), this represents around ~3%
overhead for the pipeline, which is aggravated for every pardo layer.
Since the beam model's unit of re-triable computation is per bundle, the
recovery defers should be invoked per bundle only at the root. This is already
being done in
[exec/plan.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/plan.go#L112]
This could be as simple as calling Call directly in
[exec/fn.go|https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/fn.go#L99]
instead of with the reflectx wrapper as it is presently.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
