Found the JIRA: https://issues.apache.org/jira/browse/BEAM-2506
TL;DR TestPipeline.run() would be a noop and only in the @Rule teardown it would run a single pipeline with all the tests in it. There are two big issues I can think of: - Which test failed? Eugene comments on the JIRA that PAssert site capture might make this acceptable - Some tests actually need to run a pipeline and interact with the job, like PubsubReadIT and PubsubJsonIT. This is fine - they don't really need TestPipeline at all. But they do need the automatic testing PipelineOptions, so we need to build that path. Kenn On Fri, Jun 29, 2018 at 7:23 PM Reuven Lax <[email protected]> wrote: > My run spent 5 hours and 40 minutes in the queue. I could easily see > enough people trying to run these tests to cause days of waiting time. > > > > On Fri, Jun 29, 2018 at 3:57 PM Jason Kuster <[email protected]> > wrote: > >> There's been a fairly long-standing discussion regarding merging >> ValidatesRunner tests in the same class together into one pipeline; this >> would give us more breathing room in terms of Dataflow job quota and may >> allow us to run multiple ValidatesRunner suites at the same time. >> >> On Fri, Jun 29, 2018 at 3:49 PM Lukasz Cwik <[email protected]> wrote: >> >>> I believe it was increased so we could run batch and streaming tests in >>> parallel but not enough to run multiple Dataflow VR runs in parallel. >>> >>> On Fri, Jun 29, 2018 at 3:43 PM Reuven Lax <[email protected]> wrote: >>> >>>> Didn't we recently have the quota increased? >>>> >>>> On Fri, Jun 29, 2018, 3:40 PM Lukasz Cwik <[email protected]> wrote: >>>> >>>>> Dataflow VR only runs one at a time due to Dataflow job quota capacity. >>>>> >>>>> On Fri, Jun 29, 2018 at 3:39 PM Andrew Pilloud <[email protected]> >>>>> wrote: >>>>> >>>>>> Looks like it is only running 1 instance of Dataflow ValidatesRunner at >>>>>> a time, and the job takes 2+ hours. The queue is a little backed up. >>>>>> https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle_PR/ >>>>>> >>>>>> Andrew >>>>>> >>>>>> On Fri, Jun 29, 2018 at 3:36 PM Lukasz Cwik <[email protected]> wrote: >>>>>> >>>>>>> I think the jobs aren't being scheduled or the status is failing to >>>>>>> be sent back to Github. >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Fri, Jun 29, 2018 at 3:22 PM Reuven Lax <[email protected]> wrote: >>>>>>> >>>>>>>> Well it's now moving on 2.5 hours, and the tests still haven't even >>>>>>>> started to execute. Something seems very wrong. >>>>>>>> >>>>>>>> On Fri, Jun 29, 2018 at 3:14 PM Andrew Pilloud <[email protected]> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I don't know what is going on, but I saw the same high latency in >>>>>>>>> jobs right after filtered pre-commit was turned on for the first >>>>>>>>> time. (I >>>>>>>>> don't have anything to suggest it is related, just a memorable time it >>>>>>>>> happened.) I'm also noticing the Jenkins UI is a bit laggy. >>>>>>>>> >>>>>>>>> Andrew >>>>>>>>> >>>>>>>>> On Fri, Jun 29, 2018 at 3:09 PM Reuven Lax <[email protected]> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> In addition to they delay in the trigger phrases, I'm seeing >>>>>>>>>> multi-hour (!) delays in running tests. >>>>>>>>>> >>>>>>>>>> For example: https://github.com/apache/beam/pull/5545 >>>>>>>>>> >>>>>>>>>> I triggered the Dataflow ValidatesRunner tests. The check showed >>>>>>>>>> up as yellow with no "Details" link, which generally means it had >>>>>>>>>> not yet >>>>>>>>>> been scheduled on Jenkins. It stayed like this for two hours, never >>>>>>>>>> even >>>>>>>>>> starting to run. Apparently there were a ton of idle Jenkins >>>>>>>>>> executors at >>>>>>>>>> the time, so it's not that all our Jenkins executors were busy. As >>>>>>>>>> of now, >>>>>>>>>> it still has not started running. >>>>>>>>>> >>>>>>>>>> Does anybody have any idea what's going on here? >>>>>>>>>> >>>>>>>>>> Reuven >>>>>>>>>> >>>>>>>>> >> >> -- >> ------- >> Jason Kuster >> Apache Beam / Google Cloud Dataflow >> >> See something? Say something. go/jasonkuster-feedback >> <https://goto.google.com/jasonkuster-feedback> >> >
