To those unfamiliar with these concepts, I generally conflate everything to
a "Runner" to keep things simple. Though, also mention "execution engine"
at times. Glad there appears to be concrete consensus on how we want to
talk about this. It will also help guide me in being consistent :-)
On
Thank you all for this productive conversation!
Interestingly enough, a usability study we ran for Apache Beam (more
details coming soon) pointed out that our documentation and website assume
that the readers will be already familiar with Data Processing basic
concepts such as engines, pipelines,
On Wed, Jan 6, 2021 at 12:28 PM Robert Burke wrote:
> +1 on consolidating and being consistent with our terms.
>
> I've always considered them (Runner/Engine) synonymous. From a user
> perspective, an engine without a runner isn't any good for their beam
> pipeline. That there's an adapter is an
+1 on consolidating and being consistent with our terms.
I've always considered them (Runner/Engine) synonymous. From a user
perspective, an engine without a runner isn't any good for their beam
pipeline. That there's an adapter is an implementation detail in some
instances. I do appreciate not
+1 to keeping the distinction between Runner and Engine as Kenn described,
and cleaning up the site with these in mind (I don't think the term engine
is widely used yet).
On Wed, Jan 6, 2021 at 11:15 AM Yichi Zhang wrote:
> I agree with what kenn said, in most cases I would refer to the term
>
I agree with what kenn said, in most cases I would refer to the term runner
as the adapter for translating user's pipeline code into a job
representation and submitting it to the execution engine. Though in some
cases they may still be used interchangeably such as direct runner?
On Wed, Jan 6,
+1 to distinguishing between runners and engines(spark/flink/dataflow).
Those terms are clear and make sense to me.
*~Vincent*
On Wed, Jan 6, 2021 at 11:02 AM Kenneth Knowles wrote:
> I personally try to always distinguish two concepts: the thing doing the
> computing (like Spark or Flink),
I personally try to always distinguish two concepts: the thing doing the
computing (like Spark or Flink), and the adapter for running a Beam
pipeline (like SparkRunner or FlinkRunner). I use the term "runner" to mean
the adapter, and have been trying to use the term "engine" to refer to the
thing
