On Thu, Jul 21, 2016 at 12:47 PM, Ben Chambers
<bchamb...@google.com.invalid> wrote:
> (Minor Issue: I'd propose waitUntilDone and waitUntilRunning rather than
> waitToRunning which reads oddly)
+1 to Until. Finish is nice, because it implies both success and
failure. (Alternatively, waitUntilDone throws an exception on failure,
so when it returns normally it is always Done).
> The only reason to separate submission from waitUntilRunning would be if
> you wanted to kick off several pipelines in quick succession, then wait for
> them all to be running. For instance:
>
> PipelineResult p1Future = p1.run();
> PipelineResult p2Future = p2.run();
> ...
>
> p1Future.waitUntilRunning();
> p2Future.waitUntilRunning();
> ...
>
> In this setup, you can more quickly start several pipelines, but your main
> program would wait and report any errors before exiting.
I would argue in this case you should be using standard Java
threading, which isn't so bad
ExecutorService executor = Executors.newWhatever();
Future<PipelineResult> p1Future = executor.submit(() -> p1.run());
Future<PipelineResult> p2Future = executor.submit(() -> p2.run());
...
p1Future.get(); // to block for one
executor.awaitTermination(...); // to wait for all
The difference with waitUntilFinish() is that one often wants to
interact with the PipelineResult itself in the meantime. It's also
much simpler to have one wait method, rather than two (or, as
mentioned, arbitrary status sets (which gets messy if some of them are
not terminal)). If there is high demand, we could add the second one
later.
(Totally backwards incompatible, we could calls this p.launch() for
clarity, and maybe keep a run as run() { return
p.launch().waitUntilFinish(); }.)
> On Thu, Jul 21, 2016 at 12:41 PM Robert Bradshaw
> <rober...@google.com.invalid> wrote:
>
>> I'm in favor of the proposal. My only question is whether we need
>> PipelineResult.waitToRunning(), instead I'd propose that run() block
>> until the pipeline's running/successfully submitted (or failed). This
>> would simplify the API--we'd only have one kind of wait that makes
>> sense in all cases.
>>
>> What kinds of interactions would one want to have with the
>> PipelineResults before it's running?
>>
>> On Thu, Jul 21, 2016 at 12:24 PM, Thomas Groh <tg...@google.com.invalid>
>> wrote:
>> > TestPipeline is probably the one runner that can be expected to block, as
>> > certainly JUnit tests and likely other tests will run the Pipeline, and
>> > succeed, even if the PipelineRunner throws an exception. Luckily, this
>> can
>> > be added to TestPipeline.run(), which already has additional behavior
>> > associated with it (currently regarding the unwrapping of
>> AssertionErrors)
>> >
>> > On Thu, Jul 21, 2016 at 11:40 AM, Kenneth Knowles <k...@google.com.invalid
>> >
>> > wrote:
>> >
>> >> I like this proposal. It makes pipeline.run() seem like a pretty normal
>> >> async request, and easy to program with. It removes the implicit
>> assumption
>> >> in the prior design that main() is pretty much just "build and run a
>> >> pipeline".
>> >>
>> >> The part of this that I care about most is being able to write a program
>> >> (not the pipeline, but the program that launches one or more pipelines)
>> >> that has reasonable cross-runner behavior.
>> >>
>> >> One comment:
>> >>
>> >> On Wed, Jul 20, 2016 at 3:39 PM, Pei He <pe...@google.com.invalid>
>> wrote:
>> >> >
>> >> > 4. PipelineRunner.run() should (but not required) do non-blocking runs
>> >> >
>> >>
>> >> I think we can elaborate on this a little bit. Obviously there might be
>> >> "blocking" in terms of, say, an HTTP round-trip to submit the job, but
>> >> run() should never be non-terminating.
>> >>
>> >> For a test runner that finishes the pipeline quickly, I would be fine
>> with
>> >> run() just executing the pipeline, but the PipelineResult should still
>> >> emulate the usual - just always returning a terminal status. It would be
>> >> annoying to add waitToFinish() to the end of all our tests, but leaving
>> a
>> >> run() makes the tests only work with special blocking runner wrappers
>> (and
>> >> make them poor examples). A JUnit @Rule for test pipeline would hide all
>> >> that, perhaps.
>> >>
>> >>
>> >> Kenn
>> >>
>>
