Re: [Mesa-dev] [Intel-gfx] gitlab.fd.o financial situation and impact on services

2020-04-05 Thread Peter Hutterer 
On Sat, Apr 04, 2020 at 11:16:08AM -0700, Rob Clark wrote:
> On Sat, Apr 4, 2020 at 10:47 AM Nicolas Dufresne  wrote:
> >
> > Le samedi 04 avril 2020 à 08:11 -0700, Rob Clark a écrit :
> > > On Fri, Apr 3, 2020 at 7:12 AM Michel Dänzer  wrote:
> > > > On 2020-03-01 6:46 a.m., Marek Olšák wrote:
> > > > > For Mesa, we could run CI only when Marge pushes, so that it's a 
> > > > > strictly
> > > > > pre-merge CI.
> > > >
> > > > Thanks for the suggestion! I implemented something like this for Mesa:
> > > >
> > > > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432
> > > >
> > >
> > > I wouldn't mind manually triggering pipelines, but unless there is
> > > some trick I'm not realizing, it is super cumbersome.  Ie. you have to
> > > click first the container jobs.. then wait.. then the build jobs..
> > > then wait some more.. and then finally the actual runners.  That would
> > > be a real step back in terms of usefulness of CI.. one might call it a
> > > regression :-(
> >
> > On GStreamer side we have moved some existing pipeline to manual mode.
> > As we use needs: between jobs, we could simply set the first job to
> > manual (in our case it's a single job called manifest in your case it
> > would be the N container jobs). This way you can have a manual pipeline
> > that is triggered in single (or fewer) clicks. Here's an example:
> >
> > https://gitlab.freedesktop.org/gstreamer/gstreamer/pipelines/128292
> >
> > That our post-merge pipelines, we only trigger then if we suspect a
> > problem.
> >
> 
> I'm not sure that would work for mesa since the hierarchy of jobs
> branches out pretty far.. ie. if I just clicked the arm64 build + test
> container jobs, and everything else ran automatically after that, it
> would end up running all the CI jobs for all the arm devices (or at
> least all the 64b ones)

generate your gitlab-ci from a template so each pipeline has its own job
dependency. The duplication won't hurt you if it's expanded through
templating and it gives you fine-grained running of the manual jobs.

We're using this in ci-templates/libevdev/libinput for the various
distributions and their versions so each distribution+version is effectively
its own pipeline. But we only need to maintain one job in the actual
template file.

https://freedesktop.pages.freedesktop.org/ci-templates/ci-fairy.html#templating-gitlab-ci-yml

Cheers,
   Peter
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [Mesa-dev] [Intel-gfx] gitlab.fd.o financial situation and impact on services

2020-04-05 Thread Peter Hutterer 
On Sat, Apr 04, 2020 at 08:11:23AM -0700, Rob Clark wrote:
> On Fri, Apr 3, 2020 at 7:12 AM Michel Dänzer  wrote:
> >
> > On 2020-03-01 6:46 a.m., Marek Olšák wrote:
> > > For Mesa, we could run CI only when Marge pushes, so that it's a strictly
> > > pre-merge CI.
> >
> > Thanks for the suggestion! I implemented something like this for Mesa:
> >
> > https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/4432
> >
> 
> I wouldn't mind manually triggering pipelines, but unless there is
> some trick I'm not realizing, it is super cumbersome.  Ie. you have to
> click first the container jobs.. then wait.. then the build jobs..
> then wait some more.. and then finally the actual runners.  That would
> be a real step back in terms of usefulness of CI.. one might call it a
> regression :-(

I *think* this should work though if you set up the right job dependencies.
very simple example:
https://gitlab.freedesktop.org/whot/ci-playground/pipelines/128601

job1 is "when:manual", job2 has "needs: job1", job3 has "needs: job2".
Nothing runs at first, if you trigger job1 it'll cascade down to job 2 and
3.

The main limit you have here are the stages - where a job is part of a stage
but does not have an explicit "needs:" it will wait for the previous stage
to complete. That will never happen if one job in that stage has a manual
dependency. See this pipeline as an example:
https://gitlab.freedesktop.org/whot/ci-playground/pipelines/128605

So basically: if you set up all your jobs with the correct "needs" you could
even have a noop stage for user interface purposes. Here's an example:
https://gitlab.freedesktop.org/whot/ci-playground/pipelines/128606

It has a UI stage with "test-arm" and "test-x86" manual jobs. It has other
stages with dependent jobs on those (cascading down) but it also has 
a set of autorun jobs that run independent of the manual triggers. When you
push, the autorun jobs run. When you trigger "test-arm" manually, it
triggers the various dependent jobs.

So I think what you want to do is possible, it just requires some tweaking
of the "needs" entries.

Cheers,
   Peter

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx