Henning Rohde created BEAM-3316:
-----------------------------------
Summary: Consider: do not use graph.Graph for runtime bundle
representation
Key: BEAM-3316
URL: https://issues.apache.org/jira/browse/BEAM-3316
Project: Beam
Issue Type: Improvement
Components: sdk-go
Reporter: Henning Rohde
Assignee: Henning Rohde
Priority: Minor
The internal graph.Graph representation is used to unify user pipeline and the
runtime subgraph. Perhaps we should instead translate the bundle descriptors
directly to an execution plan (=
[]exec.Unit, but a encapsulating type seems useful). Aside: we would likely
also need to have them be re-usable across bundles serially.
Such a change would keep the construction-time and runtime worlds more apart
instead of trying to unify them and we could remove DataSource/DataSink from
the graph. The direct runner would then only translate a graph.Graph to []Unit
and not have an awkward injection point for a DataManager.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
