Both. There are parts which live outside the runner like fusion and other pipeline manipulation steps (up to the runner whether they want to use these components or implement it themselves).
There is still code which is specific that integrates those components onto the runners native execution plan. On Mon, Mar 26, 2018 at 1:21 PM Romain Manni-Bucau <[email protected]> wrote: > Hi Lukasz, > > Did you started to do it generically (ie outside a particular runner impl) > using pipeline pre visitor or so, or is it still hardcoded in runner? > > It is key to not bind it to runners since this is a transversal feature > which doesnt require more than the primitives the runner supports to impl > beam model. This enables portability to be maintained at once and without > impacting the runtime reliably. > > Romain > > Le 26 mars 2018 20:51, "Lukasz Cwik" <[email protected]> a écrit : > > A few people from the Beam community have been steadily making effort on > realizing the portability goal (all SDKs on all Runners). > > Last week on an experimental branch we were able to get Apache Flink to > run a Go pipeline and also a Python pipeline using those respective SDKs. > The pipelines were limited to running ParDo and GBK (no combiners, no > state, no timers, no ...). > > To continue this effort, I have started this video conference[1] and > document[2] specifically with the intention for people to join and help > hack on the portability effort in a tighter development cycle (note time > differences may mean nobody is on the call, if that is the case feel free > to use the slack channel[3]). Questions/discussions of note will be brought > back to the dev@ list to circulate information with a wider audience. > > Note that this meet up is ONLY about getting all SDKs to work on all > Runners. > > 1: https://s.apache.org/beam-portability-team-meet > 2: https://s.apache.org/beam-portability-team-doc > 3: https://the-asf.slack.com/messages/C9W769ZJ7/ > > >
