[
https://issues.apache.org/jira/browse/BEAM-10579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Beam JIRA Bot updated BEAM-10579:
---------------------------------
Labels: stale-assigned (was: )
> Go Dynamic Splitting, use SDF input coder for encoding elements.
> ----------------------------------------------------------------
>
> Key: BEAM-10579
> URL: https://issues.apache.org/jira/browse/BEAM-10579
> Project: Beam
> Issue Type: Bug
> Components: sdk-go
> Reporter: Daniel Oliveira
> Assignee: Daniel Oliveira
> Priority: P3
> Labels: stale-assigned
>
> Dynamic splits in SDF in Go are performed on
> [ProcessSizedElementsAndRestrictions|https://github.com/apache/beam/blob/3d987e18bae1bbce18915b0d805b1db8339e92e8/sdks/go/pkg/beam/core/runtime/exec/sdf.go#L217]
> transforms, and therefore when encoding the split elements for a
> SplitResponse (to eventually be sent as a BundleApplication proto) it should
> be using the input coder of the ProcessSizedElementsAndRestrictions. However,
> right now a shortcut is taken. Since a DataSource currently always precedes a
> ProcessSizedElementsAndRestrictions transform, and has identical input and
> output coders, we currently encode split elements with the existing
> DataSource input coders.
> However, this will eventually lead to issues if any other transforms are ever
> placed preceding the ProcessSizedElementsAndRestrictions, so it's important
> to eventually fix this to use the correct input coder.
> Implementation-wise, this requires adding code to create an input coder for
> the ProcessSizedElementsAndRestrictions transform based on its input type,
> and then to provide access for encoding elements with that coder (probably
> through the SplittableUnit interface).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
