Re: [Mesa-dev] Running the CI pipeline on personal Mesa branches
On Mon, Dec 9, 2019 at 2:27 AM Michel Dänzer wrote: > > On 2019-12-07 12:56 a.m., Rob Clark wrote: > > > It would be nice if there was some way to setup some conservative > > filters to exclude groups of tests, ie. if only paths changed were > > under src/freedreno or src/gallium/drivers/freedreno, then no need to > > run panfrost CI, and visa versa. We probably shouldn't try to fine > > tune that *too* much at this risk of skipping tests that should have > > run, but seems like there should be same safe low hanging fruit to cut > > down on CI runs in common cases. > > Lots of people suggest something like this, but most of them probably > imagine it to be easier than it actually is. :) > > See https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2569 . > yeah, it does seem like it is more complex than it sounds.. I'm glad to see there is some work on that. There are a lot more CI jobs that I'd *like* to be able to run (maybe some can be scheduled jobs rather than per-MR).. for example modern adreno can be both an IMR and a tiler, so we really have two different render paths which could have their own bugs.. I'd love to be able to test both forcing tiling and sysmem render paths.. maybe don't have to happen on every MR, but freeing up CI runner bandwidth to run additional tests seems useful. BR, -R ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Running the CI pipeline on personal Mesa branches
On 2019-12-07 12:56 a.m., Rob Clark wrote: > On Fri, Dec 6, 2019 at 3:46 PM Bas Nieuwenhuizen > wrote: >> >> On Fri, Dec 6, 2019 at 10:49 AM Michel Dänzer wrote: >>> >>> >>> I just merged >>> https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2794 , which >>> affects people who want to run the CI pipeline on personal Mesa branches: >>> >>> Pushing changes to a personal branch now always creates a pipeline, but >>> none of the jobs in it run by default. (There are no longer any special >>> branch names affecting this, because creating MRs from such special >>> branches resulted in duplicate CI job runs) >>> >>> The container stage jobs can be triggered manually from the GitLab UI >>> (and maybe also via the GitLab API, for people who'd like to automate >>> this? I haven't looked into that). The build/test stage jobs run >>> automatically once all their dependencies have passed. >>> >>> As an example, in order to run one of the "piglit-*" test jobs, one has >>> to manually trigger the "x86_build" and "x86_test" jobs. >>> >>> The pipelines created for merge requests still run all jobs by default >>> as before. >>> >>> >>> The main motivation for these changes is to avoid wasting CI runner >>> resources. In that same spirit, please also cancel any unneeded >>> build/test jobs. This can be done already before those jobs start >>> running, e.g. while the container stage jobs run. >> >> No complaint about not running the pipelines by default in personal >> repositories, but expecting people to cancel automatically spawned CI >> jobs as normal part of their workflow seems incredibly fiddly and >> fragile to me. I'm not sure what you mean by fragile, but maybe my wording was too strong. It's not really "expected", I just want people to be aware that CI runner resources are finite, and wasting them affects others as well as oneself. >> Are we *that* constrained? We are a bit privileged here in Europe, as things are usually calm during our mornings. For people in North America, it tends to be pretty painful to merge an MR. (If we had a merge bot, that should mostly eliminate that pain tough) > It would be nice if there was some way to setup some conservative > filters to exclude groups of tests, ie. if only paths changed were > under src/freedreno or src/gallium/drivers/freedreno, then no need to > run panfrost CI, and visa versa. We probably shouldn't try to fine > tune that *too* much at this risk of skipping tests that should have > run, but seems like there should be same safe low hanging fruit to cut > down on CI runs in common cases. Lots of people suggest something like this, but most of them probably imagine it to be easier than it actually is. :) See https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2569 . -- Earthling Michel Dänzer | https://redhat.com Libre software enthusiast | Mesa and X developer ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Running the CI pipeline on personal Mesa branches
On Fri, Dec 6, 2019 at 3:56 PM Rob Clark wrote: > > On Fri, Dec 6, 2019 at 3:46 PM Bas Nieuwenhuizen > wrote: > > > > On Fri, Dec 6, 2019 at 10:49 AM Michel Dänzer wrote: > > > > > > > > > I just merged > > > https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2794 , which > > > affects people who want to run the CI pipeline on personal Mesa branches: > > > > > > Pushing changes to a personal branch now always creates a pipeline, but > > > none of the jobs in it run by default. (There are no longer any special > > > branch names affecting this, because creating MRs from such special > > > branches resulted in duplicate CI job runs) > > > > > > The container stage jobs can be triggered manually from the GitLab UI > > > (and maybe also via the GitLab API, for people who'd like to automate > > > this? I haven't looked into that). The build/test stage jobs run > > > automatically once all their dependencies have passed. > > > > > > As an example, in order to run one of the "piglit-*" test jobs, one has > > > to manually trigger the "x86_build" and "x86_test" jobs. > > > > > > The pipelines created for merge requests still run all jobs by default > > > as before. > > > > > > > > > The main motivation for these changes is to avoid wasting CI runner > > > resources. In that same spirit, please also cancel any unneeded > > > build/test jobs. This can be done already before those jobs start > > > running, e.g. while the container stage jobs run. > > > > No complaint about not running the pipelines by default in personal > > repositories, but expecting people to cancel automatically spawned CI > > jobs as normal part of their workflow seems incredibly fiddly and > > fragile to me. Are we *that* constrained? > > > > It would be nice if there was some way to setup some conservative > filters to exclude groups of tests, ie. if only paths changed were > under src/freedreno or src/gallium/drivers/freedreno, then no need to > run panfrost CI, and visa versa. We probably shouldn't try to fine > tune that *too* much at this risk of skipping tests that should have > run, but seems like there should be same safe low hanging fruit to cut > down on CI runs in common cases. > > I guess maybe that only helps if the bottleneck are the hw CI runners, > which might not be the case.. I'm not sure. Our bottleneck, every time I go check, is in x86_64 build/test job slots. ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Running the CI pipeline on personal Mesa branches
On Fri, Dec 6, 2019 at 3:46 PM Bas Nieuwenhuizen wrote: > > On Fri, Dec 6, 2019 at 10:49 AM Michel Dänzer wrote: > > > > > > I just merged > > https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2794 , which > > affects people who want to run the CI pipeline on personal Mesa branches: > > > > Pushing changes to a personal branch now always creates a pipeline, but > > none of the jobs in it run by default. (There are no longer any special > > branch names affecting this, because creating MRs from such special > > branches resulted in duplicate CI job runs) > > > > The container stage jobs can be triggered manually from the GitLab UI > > (and maybe also via the GitLab API, for people who'd like to automate > > this? I haven't looked into that). The build/test stage jobs run > > automatically once all their dependencies have passed. > > > > As an example, in order to run one of the "piglit-*" test jobs, one has > > to manually trigger the "x86_build" and "x86_test" jobs. > > > > The pipelines created for merge requests still run all jobs by default > > as before. > > > > > > The main motivation for these changes is to avoid wasting CI runner > > resources. In that same spirit, please also cancel any unneeded > > build/test jobs. This can be done already before those jobs start > > running, e.g. while the container stage jobs run. > > No complaint about not running the pipelines by default in personal > repositories, but expecting people to cancel automatically spawned CI > jobs as normal part of their workflow seems incredibly fiddly and > fragile to me. Are we *that* constrained? > It would be nice if there was some way to setup some conservative filters to exclude groups of tests, ie. if only paths changed were under src/freedreno or src/gallium/drivers/freedreno, then no need to run panfrost CI, and visa versa. We probably shouldn't try to fine tune that *too* much at this risk of skipping tests that should have run, but seems like there should be same safe low hanging fruit to cut down on CI runs in common cases. I guess maybe that only helps if the bottleneck are the hw CI runners, which might not be the case.. I'm not sure. BR, -R ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
Re: [Mesa-dev] Running the CI pipeline on personal Mesa branches
On Fri, Dec 6, 2019 at 10:49 AM Michel Dänzer wrote: > > > I just merged > https://gitlab.freedesktop.org/mesa/mesa/merge_requests/2794 , which > affects people who want to run the CI pipeline on personal Mesa branches: > > Pushing changes to a personal branch now always creates a pipeline, but > none of the jobs in it run by default. (There are no longer any special > branch names affecting this, because creating MRs from such special > branches resulted in duplicate CI job runs) > > The container stage jobs can be triggered manually from the GitLab UI > (and maybe also via the GitLab API, for people who'd like to automate > this? I haven't looked into that). The build/test stage jobs run > automatically once all their dependencies have passed. > > As an example, in order to run one of the "piglit-*" test jobs, one has > to manually trigger the "x86_build" and "x86_test" jobs. > > The pipelines created for merge requests still run all jobs by default > as before. > > > The main motivation for these changes is to avoid wasting CI runner > resources. In that same spirit, please also cancel any unneeded > build/test jobs. This can be done already before those jobs start > running, e.g. while the container stage jobs run. No complaint about not running the pipelines by default in personal repositories, but expecting people to cancel automatically spawned CI jobs as normal part of their workflow seems incredibly fiddly and fragile to me. Are we *that* constrained? > > > -- > Earthling Michel Dänzer | https://redhat.com > Libre software enthusiast | Mesa and X developer > ___ > mesa-dev mailing list > mesa-dev@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/mesa-dev ___ mesa-dev mailing list mesa-dev@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/mesa-dev
