(Minor Issue: I'd propose waitUntilDone and waitUntilRunning rather than waitToRunning which reads oddly)
The only reason to separate submission from waitUntilRunning would be if you wanted to kick off several pipelines in quick succession, then wait for them all to be running. For instance: PipelineResult p1Future = p1.run(); PipelineResult p2Future = p2.run(); ... p1Future.waitUntilRunning(); p2Future.waitUntilRunning(); ... In this setup, you can more quickly start several pipelines, but your main program would wait and report any errors before exiting. On Thu, Jul 21, 2016 at 12:41 PM Robert Bradshaw <[email protected]> wrote: > I'm in favor of the proposal. My only question is whether we need > PipelineResult.waitToRunning(), instead I'd propose that run() block > until the pipeline's running/successfully submitted (or failed). This > would simplify the API--we'd only have one kind of wait that makes > sense in all cases. > > What kinds of interactions would one want to have with the > PipelineResults before it's running? > > On Thu, Jul 21, 2016 at 12:24 PM, Thomas Groh <[email protected]> > wrote: > > TestPipeline is probably the one runner that can be expected to block, as > > certainly JUnit tests and likely other tests will run the Pipeline, and > > succeed, even if the PipelineRunner throws an exception. Luckily, this > can > > be added to TestPipeline.run(), which already has additional behavior > > associated with it (currently regarding the unwrapping of > AssertionErrors) > > > > On Thu, Jul 21, 2016 at 11:40 AM, Kenneth Knowles <[email protected] > > > > wrote: > > > >> I like this proposal. It makes pipeline.run() seem like a pretty normal > >> async request, and easy to program with. It removes the implicit > assumption > >> in the prior design that main() is pretty much just "build and run a > >> pipeline". > >> > >> The part of this that I care about most is being able to write a program > >> (not the pipeline, but the program that launches one or more pipelines) > >> that has reasonable cross-runner behavior. > >> > >> One comment: > >> > >> On Wed, Jul 20, 2016 at 3:39 PM, Pei He <[email protected]> > wrote: > >> > > >> > 4. PipelineRunner.run() should (but not required) do non-blocking runs > >> > > >> > >> I think we can elaborate on this a little bit. Obviously there might be > >> "blocking" in terms of, say, an HTTP round-trip to submit the job, but > >> run() should never be non-terminating. > >> > >> For a test runner that finishes the pipeline quickly, I would be fine > with > >> run() just executing the pipeline, but the PipelineResult should still > >> emulate the usual - just always returning a terminal status. It would be > >> annoying to add waitToFinish() to the end of all our tests, but leaving > a > >> run() makes the tests only work with special blocking runner wrappers > (and > >> make them poor examples). A JUnit @Rule for test pipeline would hide all > >> that, perhaps. > >> > >> > >> Kenn > >> >
