I like this proposal. It makes pipeline.run() seem like a pretty normal async request, and easy to program with. It removes the implicit assumption in the prior design that main() is pretty much just "build and run a pipeline".
The part of this that I care about most is being able to write a program (not the pipeline, but the program that launches one or more pipelines) that has reasonable cross-runner behavior. One comment: On Wed, Jul 20, 2016 at 3:39 PM, Pei He <[email protected]> wrote: > > 4. PipelineRunner.run() should (but not required) do non-blocking runs > I think we can elaborate on this a little bit. Obviously there might be "blocking" in terms of, say, an HTTP round-trip to submit the job, but run() should never be non-terminating. For a test runner that finishes the pipeline quickly, I would be fine with run() just executing the pipeline, but the PipelineResult should still emulate the usual - just always returning a terminal status. It would be annoying to add waitToFinish() to the end of all our tests, but leaving a run() makes the tests only work with special blocking runner wrappers (and make them poor examples). A JUnit @Rule for test pipeline would hide all that, perhaps. Kenn
