Re: CI that doesn't run on MRs

2025-08-22 Thread Ben Cooksley
On Fri, Aug 22, 2025 at 10:52 AM David Edmundson 
wrote:

> >Before I make that change, which is really just removing the
> force-to-manual-for-Plasma, could folks please go through outstanding MRs
> and mark those that aren't ready to merge as draft?
>
> Before we make a change, lets discuss it a bit more.
>
> I'm 100% sure that draft is not an option that will work for us. We've
> already discussed it previously.
> We'll be back at the original state where CI is running on too many
> pointless things again.
>
> The current state is not perfect, but it's not worse than before. I
> don't want us to just flip-flop forever.
>

In that instance, do we just stick with what we currently have?

I do have some other ideas but that requires some development and there are
some other priorities that need to be addressed first.


>
> David
>

Thanks,
Ben


Re: CI that doesn't run on MRs

2025-08-21 Thread David Edmundson
>Before I make that change, which is really just removing the 
>force-to-manual-for-Plasma, could folks please go through outstanding MRs and 
>mark those that aren't ready to merge as draft?

Before we make a change, lets discuss it a bit more.

I'm 100% sure that draft is not an option that will work for us. We've
already discussed it previously.
We'll be back at the original state where CI is running on too many
pointless things again.

The current state is not perfect, but it's not worse than before. I
don't want us to just flip-flop forever.

David


Re: CI that doesn't run on MRs

2025-08-20 Thread Nate Graham

On 8/20/25 13:07, Ben Cooksley wrote:
On Thu, Aug 21, 2025 at 7:02 AM Nate Graham Just having a look at KWin at least, looks like the majority of MRs 
aren't draft.


Before I make that change, which is really just removing the force-to- 
manual-for-Plasma, could folks please go through outstanding MRs and 
mark those that aren't ready to merge as draft?



If we're going to do this, it will require some formal communication, 
documentation, and a transitional period.


It will also require some understanding that whether a merge request is 
"ready to merge" or not isn't black-and-white. When you submit your MR, 
you certainly think it's ready to merge. Then you get review comments 
and make changes. Who's to say the CI shouldn't run on the changes you 
just made, to make sure they didn't break anything?


I can see us marking merge requests as Draft if we *know* there's going 
to be heavy iteration over an extended period of time (for example 
https://invent.kde.org/plasma/plasma-workspace/-/merge_requests/5563), 
and that's what needs to be communicated broadly to align everyone's 
expectations.



Nate



Re: CI that doesn't run on MRs

2025-08-20 Thread Ben Cooksley
On Thu, Aug 21, 2025 at 7:02 AM Nate Graham  wrote:

>
>
> On 8/20/25 12:51, Ben Cooksley wrote:
> > On Tue, Aug 12, 2025 at 11:58 PM Nate Graham  > > wrote:
> >
> >
> >
> > On 8/11/25 04:58, Ben Cooksley wrote:
> >  > Unfortunately the variable in question we were trying to rely on,
> >  > CI_MERGE_REQUEST_APPROVED is exclusive to Enterprise Edition:
> >  >
> >  > ben@sapphire:~/workspace/gitlab> grep -ir
> CI_MERGE_REQUEST_APPROVED *
> >  > ee/app/models/ee/merge_request.rb:  variables.append(key:
> >  > 'CI_MERGE_REQUEST_APPROVED', value: approved?.to_s) if approved?
> >  >
> >  > As such that functionality is not available to us.
> >  >
> >  > This means the options available to us are:
> >  > - Using a custom label (accessible via CI_MERGE_REQUEST_LABELS)
> >
> > What does this look like?
> >
> >
> > It would be like any other label on a merge request.
> >
> > See Arch for instance on https://invent.kde.org/sysadmin/repo-
> > metadata/-/merge_requests/614  > metadata/-/merge_requests/614> - in this case the label would be
> > "Approved", "Ready" or something equivalent that means a bit more than
> > just "Run CI"
>
> Ah, that kind of label.
>
> I doubt that will work. We did a variant of this for approval before
> Gitlab CE backported the optional approvals feature, and it was a
> complete mess. Did not work at all. I think turning off the CI for draft
> merge requests would be a better approach if I had to choose between
> those options.
>

Just having a look at KWin at least, looks like the majority of MRs aren't
draft.

Before I make that change, which is really just removing the
force-to-manual-for-Plasma, could folks please go through outstanding MRs
and mark those that aren't ready to merge as draft?


> Nate
>

Thanks,
Ben


Re: CI that doesn't run on MRs

2025-08-20 Thread Nate Graham




On 8/20/25 12:51, Ben Cooksley wrote:
On Tue, Aug 12, 2025 at 11:58 PM Nate Graham > wrote:




On 8/11/25 04:58, Ben Cooksley wrote:
 > Unfortunately the variable in question we were trying to rely on,
 > CI_MERGE_REQUEST_APPROVED is exclusive to Enterprise Edition:
 >
 > ben@sapphire:~/workspace/gitlab> grep -ir CI_MERGE_REQUEST_APPROVED *
 > ee/app/models/ee/merge_request.rb:          variables.append(key:
 > 'CI_MERGE_REQUEST_APPROVED', value: approved?.to_s) if approved?
 >
 > As such that functionality is not available to us.
 >
 > This means the options available to us are:
 > - Using a custom label (accessible via CI_MERGE_REQUEST_LABELS)

What does this look like?


It would be like any other label on a merge request.

See Arch for instance on https://invent.kde.org/sysadmin/repo- 
metadata/-/merge_requests/614  - in this case the label would be 
"Approved", "Ready" or something equivalent that means a bit more than 
just "Run CI"


Ah, that kind of label.

I doubt that will work. We did a variant of this for approval before 
Gitlab CE backported the optional approvals feature, and it was a 
complete mess. Did not work at all. I think turning off the CI for draft 
merge requests would be a better approach if I had to choose between 
those options.


Nate


Re: CI that doesn't run on MRs

2025-08-20 Thread Ben Cooksley
On Tue, Aug 12, 2025 at 11:58 PM Nate Graham  wrote:

>
>
> On 8/11/25 04:58, Ben Cooksley wrote:
> > Unfortunately the variable in question we were trying to rely on,
> > CI_MERGE_REQUEST_APPROVED is exclusive to Enterprise Edition:
> >
> > ben@sapphire:~/workspace/gitlab> grep -ir CI_MERGE_REQUEST_APPROVED *
> > ee/app/models/ee/merge_request.rb:  variables.append(key:
> > 'CI_MERGE_REQUEST_APPROVED', value: approved?.to_s) if approved?
> >
> > As such that functionality is not available to us.
> >
> > This means the options available to us are:
> > - Using a custom label (accessible via CI_MERGE_REQUEST_LABELS)
>
> What does this look like?
>

It would be like any other label on a merge request.

See Arch for instance on
https://invent.kde.org/sysadmin/repo-metadata/-/merge_requests/614 - in
this case the label would be "Approved", "Ready" or something equivalent
that means a bit more than just "Run CI"


>
> Nate
>

Thanks,
Ben


Re: CI that doesn't run on MRs

2025-08-12 Thread Nate Graham




On 8/11/25 04:58, Ben Cooksley wrote:
Unfortunately the variable in question we were trying to rely on, 
CI_MERGE_REQUEST_APPROVED is exclusive to Enterprise Edition:


ben@sapphire:~/workspace/gitlab> grep -ir CI_MERGE_REQUEST_APPROVED *
ee/app/models/ee/merge_request.rb:          variables.append(key: 
'CI_MERGE_REQUEST_APPROVED', value: approved?.to_s) if approved?


As such that functionality is not available to us.

This means the options available to us are:
- Using a custom label (accessible via CI_MERGE_REQUEST_LABELS)


What does this look like?

Nate


Re: CI that doesn't run on MRs

2025-08-11 Thread Ben Cooksley
On Mon, Aug 11, 2025 at 2:31 AM Nate Graham  wrote:

> On 8/10/25 8:06 AM, Nate Graham wrote:
> > On 8/7/25 1:06 PM, Ben Cooksley wrote:
> >> On Fri, Aug 8, 2025 at 2:55 AM Nate Graham  >> > wrote:
> >
> >> Is this working? I'm not seeing it happen. When a Plasma-aligned
> >> merge
> >> request is approved and then rebased, pipelines still aren't
> starting
> >> automatically.
> >>
> >>
> >> Links to affected MRs would be helpful please :)
> >
> > Here's an example: https://invent.kde.org/plasma/plasma-workspace/-/
> > merge_requests/5743
>
> And another:
> https://invent.kde.org/plasma/plasma-desktop/-/merge_requests/3149
>
> None of them seem to be running all pipelines automatically after
> rebasing following an approval.
>

Unfortunately the variable in question we were trying to rely on,
CI_MERGE_REQUEST_APPROVED is exclusive to Enterprise Edition:

ben@sapphire:~/workspace/gitlab> grep -ir CI_MERGE_REQUEST_APPROVED *
ee/app/models/ee/merge_request.rb:  variables.append(key:
'CI_MERGE_REQUEST_APPROVED', value: approved?.to_s) if approved?

As such that functionality is not available to us.

This means the options available to us are:
- Using a custom label (accessible via CI_MERGE_REQUEST_LABELS)
- Making use of the functionality to mark merge requests as draft until
they're actually ready for merging (accessible via CI_MERGE_REQUEST_DRAFT)


> Nate
>

Thanks,
Ben


Re: CI that doesn't run on MRs

2025-08-10 Thread Nate Graham

On 8/10/25 8:06 AM, Nate Graham wrote:

On 8/7/25 1:06 PM, Ben Cooksley wrote:
On Fri, Aug 8, 2025 at 2:55 AM Nate Graham > wrote:


    Is this working? I'm not seeing it happen. When a Plasma-aligned 
merge

    request is approved and then rebased, pipelines still aren't starting
    automatically.


Links to affected MRs would be helpful please :)


Here's an example: https://invent.kde.org/plasma/plasma-workspace/-/ 
merge_requests/5743


And another: 
https://invent.kde.org/plasma/plasma-desktop/-/merge_requests/3149


None of them seem to be running all pipelines automatically after 
rebasing following an approval.


Nate


Re: CI that doesn't run on MRs

2025-08-10 Thread Nate Graham

On 8/7/25 1:06 PM, Ben Cooksley wrote:
On Fri, Aug 8, 2025 at 2:55 AM Nate Graham > wrote:



Is this working? I'm not seeing it happen. When a Plasma-aligned merge
request is approved and then rebased, pipelines still aren't starting
automatically.


Links to affected MRs would be helpful please :)


Here's an example: 
https://invent.kde.org/plasma/plasma-workspace/-/merge_requests/5743


Nate


Re: CI that doesn't run on MRs

2025-08-07 Thread Ben Cooksley
On Fri, Aug 8, 2025 at 2:55 AM Nate Graham  wrote:

> On 8/2/25 4:13 AM, Ben Cooksley wrote:
> > On Fri, Aug 1, 2025 at 12:23 AM David Edmundson
> > mailto:[email protected]>> wrote:
> >
> >  > We could set merge request pipelines to always run on an approved
> > merge request, so the next time it is rebased (which for highly
> > active projects should basically always be the case by the time
> > approve is ready to be pressed) the pipeline will kick off?
> >
> > That sounds a perfect compromise between our two extremes. ++
> >
> >
> > That change was made in 28f137680869d1af97f6595d5c0e0ff2e773a439.
> >
> > Please note pipelines are not re-generated just by a merge request being
> > approved - it will still need changes to the sources to happen in order
> > to trigger that. Therefore if you are happy with a merge request it
> > should be approved before you hit rebase.
>
> Is this working? I'm not seeing it happen. When a Plasma-aligned merge
> request is approved and then rebased, pipelines still aren't starting
> automatically.
>

Links to affected MRs would be helpful please :)


>
> Nate
>

Thanks,
Ben


Re: CI that doesn't run on MRs

2025-08-07 Thread Nate Graham

On 8/2/25 4:13 AM, Ben Cooksley wrote:
On Fri, Aug 1, 2025 at 12:23 AM David Edmundson 
mailto:[email protected]>> wrote:


 > We could set merge request pipelines to always run on an approved
merge request, so the next time it is rebased (which for highly
active projects should basically always be the case by the time
approve is ready to be pressed) the pipeline will kick off?

That sounds a perfect compromise between our two extremes. ++


That change was made in 28f137680869d1af97f6595d5c0e0ff2e773a439.

Please note pipelines are not re-generated just by a merge request being 
approved - it will still need changes to the sources to happen in order 
to trigger that. Therefore if you are happy with a merge request it 
should be approved before you hit rebase.


Is this working? I'm not seeing it happen. When a Plasma-aligned merge 
request is approved and then rebased, pipelines still aren't starting 
automatically.


Nate


Re: CI that doesn't run on MRs

2025-08-02 Thread Ben Cooksley
On Fri, Aug 1, 2025 at 12:23 AM David Edmundson 
wrote:

> > We could set merge request pipelines to always run on an approved merge
> request, so the next time it is rebased (which for highly active projects
> should basically always be the case by the time approve is ready to be
> pressed) the pipeline will kick off?
>
> That sounds a perfect compromise between our two extremes. ++
>

That change was made in 28f137680869d1af97f6595d5c0e0ff2e773a439.

Please note pipelines are not re-generated just by a merge request being
approved - it will still need changes to the sources to happen in order to
trigger that. Therefore if you are happy with a merge request it should be
approved before you hit rebase.


> David
>

Cheers,
Ben


Re: CI that doesn't run on MRs

2025-07-31 Thread David Edmundson
> We could set merge request pipelines to always run on an approved merge 
> request, so the next time it is rebased (which for highly active projects 
> should basically always be the case by the time approve is ready to be 
> pressed) the pipeline will kick off?

That sounds a perfect compromise between our two extremes. ++

David


Re: CI that doesn't run on MRs

2025-07-31 Thread Ben Cooksley
On Thu, Jul 31, 2025 at 7:30 AM Nate Graham  wrote:

> On 7/30/25 1:15 PM, Ben Cooksley wrote:
> > If memory serves correctly the general view from that thread could be
> > summarised as:
> > 1) Work branches were basically used to immediately open merge requests
> > in the majority of cases, so running CI on pushes to work branches was
> > simply a waste as CI would be (re-)run on the merge request.
> > 2) Merge requests were used as part of an iterative development
> > workflow, and as such CI runs on merge requests were in many cases also
> > useless because the MR was not yet ready to be merged and was still
> > under active development.
> >
> > Because merge requests that aren't actually ready for final review and
> > then merging cannot currently be distinguished from merge requests that
> > are not at all ready and are just collecting comments, the only way to
> > solve for (2) is to switch CI to manual on merge requests.
> >
> > Gitlab already has a mechanism for differentiating between those two
> > types of merge requests (marking as draft) however it is not used by
> > Plasma (or at least wasn't on the two pages of KWin reviews I quickly
> > skimmed)
> >
> > So this isn't really Gitlab's fault, it is how Plasma has opted to use
> > Gitlab.
>
> Every merge request (everywhere, not just in Plasma) is submitted in a
> state where the author believes it's ready for merging. The purpose of
> review is for others to challenge that belief and offer suggestions for
> improvement. Hence iteration, and avoiding that is impossible for all
> but the smallest and most trivial merge requests.
>

This runs counter to one of the statements in the earlier thread:

[quote]
>Realistically merge requests shouldn't be proposed until people are ready
to get something reviewed and merged in...

That's not realistic with how it is on the frontlines. The earlier we
share feedback the better, and reviews can take many many iterations.
[/quote]

I'm a bit confused here as to what is actually correct?


> You only see this process using a lot of resources in Plasma because
> Plasma is the largest and most active project in KDE. As I recall, there
> were similar complaints about Krita and Kstars, which are also large and
> active projects.
>

Both of those have attracted complaints for other reasons - because they
use custom workflows which then creates a maintenance burden.

Krita in particular has an entirely custom set of CI images, which includes
their own specific compiler toolchain, and has their own set of build
tooling that builds their specific set of dependencies built using CMake
toolchains they maintain.
This includes a patched Qt as well as an entire multimedia toolstack up to
and including ffmpeg as well as a Python build with bindings support.

Stats from the earlier thread:

time_used | job_count
--+---
3536:44:31.03921 | 45860

  full_path   |time_used | job_count
--+--+---
plasma/kwin  | 325:40:45.439134 |  2419
plasma/plasma-workspace  | 224:56:33.415042 |  1032
multimedia/kdenlive  | 195:29:20.224873 |   794
graphics/krita   | 189:03:12.118222 |   441
network/ruqola   | 180:30:09.57436  |   585
network/neochat  | 151:25:19.726593 |  1695
education/kstars | 129:53:37.649402 |   333
sysadmin/ci-management   | 116:32:43.423853 |   163
plasma/plasma-desktop| 116:07:05.90223  |   818
kde-linux/kde-linux-packages | 84:45:50.156624  |35


> Implicitly punishing projects for success and popularity is a rather
> perverse incentive, I hope you'll agree!
>

Agreed on that front.

We should strive to run things as efficiently as possible still, which is
something we can do better at.


>
>
>
> > If you were to adopt the draft flag on merge requests that are in that
> > iterative phase and not intended to be merged then we could look at
> > adjusting how it works.
>
> I think we're all mostly in agreement that running the pipeline n times
> for n pushes in large and active merge requests is something that
> doesn't make a lot of sense.
>
> The question is how to improve upon this without throwing the baby out
> with the bathwater.
>

Sounds good to me.


>
> Using the Draft flag for such merge requests once it becomes clear that
> they won't merge immediately is one option. Perhaps there are others,
> like running pipelines automatically only upon MR submission, and
> thereafter requiring a fully green pipeline as a prerequisite for
> merging and starting a pipeline automatically when someone clicks "Set
> to auto-merge". Lots of options here; we can brainstorm.
>

We could set merge request pipelines to always run on an approved merge
request, so the next time it is rebased (which for highly active projects
should basically always be the case by the time approve is ready to be

Re: CI that doesn't run on MRs

2025-07-30 Thread Nate Graham

On 7/30/25 1:15 PM, Ben Cooksley wrote:
If memory serves correctly the general view from that thread could be 
summarised as:
1) Work branches were basically used to immediately open merge requests 
in the majority of cases, so running CI on pushes to work branches was 
simply a waste as CI would be (re-)run on the merge request.
2) Merge requests were used as part of an iterative development 
workflow, and as such CI runs on merge requests were in many cases also 
useless because the MR was not yet ready to be merged and was still 
under active development.


Because merge requests that aren't actually ready for final review and 
then merging cannot currently be distinguished from merge requests that 
are not at all ready and are just collecting comments, the only way to 
solve for (2) is to switch CI to manual on merge requests.


Gitlab already has a mechanism for differentiating between those two 
types of merge requests (marking as draft) however it is not used by 
Plasma (or at least wasn't on the two pages of KWin reviews I quickly 
skimmed)


So this isn't really Gitlab's fault, it is how Plasma has opted to use 
Gitlab.


Every merge request (everywhere, not just in Plasma) is submitted in a 
state where the author believes it's ready for merging. The purpose of 
review is for others to challenge that belief and offer suggestions for 
improvement. Hence iteration, and avoiding that is impossible for all 
but the smallest and most trivial merge requests.


You only see this process using a lot of resources in Plasma because 
Plasma is the largest and most active project in KDE. As I recall, there 
were similar complaints about Krita and Kstars, which are also large and 
active projects.


Implicitly punishing projects for success and popularity is a rather 
perverse incentive, I hope you'll agree!




If you were to adopt the draft flag on merge requests that are in that 
iterative phase and not intended to be merged then we could look at 
adjusting how it works.


I think we're all mostly in agreement that running the pipeline n times 
for n pushes in large and active merge requests is something that 
doesn't make a lot of sense.


The question is how to improve upon this without throwing the baby out 
with the bathwater.


Using the Draft flag for such merge requests once it becomes clear that 
they won't merge immediately is one option. Perhaps there are others, 
like running pipelines automatically only upon MR submission, and 
thereafter requiring a fully green pipeline as a prerequisite for 
merging and starting a pipeline automatically when someone clicks "Set 
to auto-merge". Lots of options here; we can brainstorm.




Note that this won't do anything to solve broken tests, or tests that 
take too long - that is something only Plasma developers can correct.


There's plenty of blame to go around. We can and should improve this.

From another quarter, an improvement I can envision is not re-running 
the pipeline on a commit that came from a merge request that already ran 
the pipeline right before merging. This is wasted work in 100% of cases.



Nate



Re: CI that doesn't run on MRs

2025-07-30 Thread Nicolas Fella

On 7/30/25 9:15 PM, Ben Cooksley wrote:

On Thu, Jul 31, 2025 at 4:52 AM Harald Sitter  wrote:

On Wed, Jul 30, 2025 at 1:25 PM Ben Cooksley 
wrote:
>
> On Wed, Jul 30, 2025 at 7:18 PM Vlad Zahorodnii
 wrote:
>>
>> On 7/30/25 9:58 AM, Harald Sitter wrote:
>> > Hey
>> >
>> > So, I need to vent a bit. I am super unhappy with the manual CI
>> > trigger stuff. Half the time I forget.  When I don't forget I
want to
>> > press all the buttons, but I can't because of the prestage, I
usually
>> > sit there for 5 seconds, nothing happens because the prestage
takes
>> > more than 5 seconds, I get distracted and do something else
... and
>> > forget I wanted to land something at all.
>
>
> I was working on some merge requests for sysadmin/repo-metadata
just a few hours ago, which only contains basic linting jobs.
> They were picked up and fully processed in the space of ~30
seconds - total of 6 different linter jobs (so 6 containers
spawned in total).
>
> All seems quite reasonable for processing turnaround.
>
>>
>> >
>> > More and more I find myself not caring and merging without
pre-merge
>> > CI. Between the manual triggers, failing appium tests, and
conflicting
>> > merges I just can't be bothered with the CI. Surely that
can't be the
>> > goal here.
>> >
>> > Can we get back a CI that helps?
>>
>> Hi,
>>
>> Just in case, the automatic CI got disabled because kwin had
too many
>> pipelines and it takes about 10-15 minutes for a single pipeline to
>> complete.
>>
>> There was a long and somewhat heated discussion about it. We
asked to
>> disable automatic CI for kwin specifically, so when you push to
a work
>> branch that is not ready to be merged yet, no CI resources are
wasted.
>> However, instead, automatic CI got disabled for all Plasma
projects.
>> Sorry! :(
>
>
> Plasma in general was set to manual for all merge requests
because the commentary in that thread made it appear that the
workflows were more Plasma than just KWin. There are also issues
with Appium tests in Plasma Desktop / Workspace which mean
pipelines take an extremely long amount of time to complete/fail
(something on the order of 40-50 minutes if memory serves)
>
> The overuse of CI resources by Plasma could in part be mitigated
by marking merge requests that are not ready to merge as draft,
which seems to be a Gitlab feature that is not in use at all at
least if I take a cursory look at current pending KWin MRs. We
already set CI into manual mode for draft merge requests as a
global rule, as well as for work branches.

What confuses me right now is whether this is just plasma being an
active project or gitlab having poor behavior. It occurs to me that
the former would be solved by giving plasma dedicated plasma resources
to use, while the latter is a matter of hiring someone to engineer the
problem(s) with gitlab away.
Both solvable problems though.


If memory serves correctly the general view from that thread could be 
summarised as:
1) Work branches were basically used to immediately open merge 
requests in the majority of cases, so running CI on pushes to work 
branches was simply a waste as CI would be (re-)run on the merge request.


That part seems pretty uncontroversial.


2) Merge requests were used as part of an iterative development 
workflow, and as such CI runs on merge requests were in many cases 
also useless because the MR was not yet ready to be merged and was 
still under active development.


Because merge requests that aren't actually ready for final review and 
then merging cannot currently be distinguished from merge requests 
that are not at all ready and are just collecting comments, the only 
way to solve for (2) is to switch CI to manual on merge requests.


Gitlab already has a mechanism for differentiating between those two 
types of merge requests (marking as draft) however it is not used by 
Plasma (or at least wasn't on the two pages of KWin reviews I quickly 
skimmed)


So this isn't really Gitlab's fault, it is how Plasma has opted to use 
Gitlab.


That's not something that's unique to Plasma in any way. Sometimes you 
make a merge request and it quickly gets approved and merged, sometime 
it takes a few iterations until consensus is reached. That's how it 
works for any reasonably active project where it's not just one guy 
pushing straight to master.



If you were to adopt the draft flag on merge requests that are in that 
iterative phase and not intended to be merged then we could look at 
adjusting how it works.


Note that this won't do anything to solve broken tests, or tests that 
take too long - that is something only Plasma developers can correct.


Cheers,
Ben


Re: CI that doesn't run on MRs

2025-07-30 Thread Ben Cooksley
On Thu, Jul 31, 2025 at 4:52 AM Harald Sitter  wrote:

> On Wed, Jul 30, 2025 at 1:25 PM Ben Cooksley  wrote:
> >
> > On Wed, Jul 30, 2025 at 7:18 PM Vlad Zahorodnii 
> wrote:
> >>
> >> On 7/30/25 9:58 AM, Harald Sitter wrote:
> >> > Hey
> >> >
> >> > So, I need to vent a bit. I am super unhappy with the manual CI
> >> > trigger stuff. Half the time I forget.  When I don't forget I want to
> >> > press all the buttons, but I can't because of the prestage, I usually
> >> > sit there for 5 seconds, nothing happens because the prestage takes
> >> > more than 5 seconds, I get distracted and do something else ... and
> >> > forget I wanted to land something at all.
> >
> >
> > I was working on some merge requests for sysadmin/repo-metadata just a
> few hours ago, which only contains basic linting jobs.
> > They were picked up and fully processed in the space of ~30 seconds -
> total of 6 different linter jobs (so 6 containers spawned in total).
> >
> > All seems quite reasonable for processing turnaround.
> >
> >>
> >> >
> >> > More and more I find myself not caring and merging without pre-merge
> >> > CI. Between the manual triggers, failing appium tests, and conflicting
> >> > merges I just can't be bothered with the CI. Surely that can't be the
> >> > goal here.
> >> >
> >> > Can we get back a CI that helps?
> >>
> >> Hi,
> >>
> >> Just in case, the automatic CI got disabled because kwin had too many
> >> pipelines and it takes about 10-15 minutes for a single pipeline to
> >> complete.
> >>
> >> There was a long and somewhat heated discussion about it. We asked to
> >> disable automatic CI for kwin specifically, so when you push to a work
> >> branch that is not ready to be merged yet, no CI resources are wasted.
> >> However, instead, automatic CI got disabled for all Plasma projects.
> >> Sorry! :(
> >
> >
> > Plasma in general was set to manual for all merge requests because the
> commentary in that thread made it appear that the workflows were more
> Plasma than just KWin. There are also issues with Appium tests in Plasma
> Desktop / Workspace which mean pipelines take an extremely long amount of
> time to complete/fail (something on the order of 40-50 minutes if memory
> serves)
> >
> > The overuse of CI resources by Plasma could in part be mitigated by
> marking merge requests that are not ready to merge as draft, which seems to
> be a Gitlab feature that is not in use at all at least if I take a cursory
> look at current pending KWin MRs. We already set CI into manual mode for
> draft merge requests as a global rule, as well as for work branches.
>
> What confuses me right now is whether this is just plasma being an
> active project or gitlab having poor behavior. It occurs to me that
> the former would be solved by giving plasma dedicated plasma resources
> to use, while the latter is a matter of hiring someone to engineer the
> problem(s) with gitlab away.
> Both solvable problems though.
>

If memory serves correctly the general view from that thread could be
summarised as:
1) Work branches were basically used to immediately open merge requests in
the majority of cases, so running CI on pushes to work branches was simply
a waste as CI would be (re-)run on the merge request.
2) Merge requests were used as part of an iterative development workflow,
and as such CI runs on merge requests were in many cases also useless
because the MR was not yet ready to be merged and was still under active
development.

Because merge requests that aren't actually ready for final review and then
merging cannot currently be distinguished from merge requests that are not
at all ready and are just collecting comments, the only way to solve for
(2) is to switch CI to manual on merge requests.

Gitlab already has a mechanism for differentiating between those two types
of merge requests (marking as draft) however it is not used by Plasma (or
at least wasn't on the two pages of KWin reviews I quickly skimmed)

So this isn't really Gitlab's fault, it is how Plasma has opted to use
Gitlab.

If you were to adopt the draft flag on merge requests that are in that
iterative phase and not intended to be merged then we could look at
adjusting how it works.

Note that this won't do anything to solve broken tests, or tests that take
too long - that is something only Plasma developers can correct.

Cheers,
Ben


Re: CI that doesn't run on MRs

2025-07-30 Thread Harald Sitter
On Wed, Jul 30, 2025 at 1:25 PM Ben Cooksley  wrote:
>
> On Wed, Jul 30, 2025 at 7:18 PM Vlad Zahorodnii  
> wrote:
>>
>> On 7/30/25 9:58 AM, Harald Sitter wrote:
>> > Hey
>> >
>> > So, I need to vent a bit. I am super unhappy with the manual CI
>> > trigger stuff. Half the time I forget.  When I don't forget I want to
>> > press all the buttons, but I can't because of the prestage, I usually
>> > sit there for 5 seconds, nothing happens because the prestage takes
>> > more than 5 seconds, I get distracted and do something else ... and
>> > forget I wanted to land something at all.
>
>
> I was working on some merge requests for sysadmin/repo-metadata just a few 
> hours ago, which only contains basic linting jobs.
> They were picked up and fully processed in the space of ~30 seconds - total 
> of 6 different linter jobs (so 6 containers spawned in total).
>
> All seems quite reasonable for processing turnaround.
>
>>
>> >
>> > More and more I find myself not caring and merging without pre-merge
>> > CI. Between the manual triggers, failing appium tests, and conflicting
>> > merges I just can't be bothered with the CI. Surely that can't be the
>> > goal here.
>> >
>> > Can we get back a CI that helps?
>>
>> Hi,
>>
>> Just in case, the automatic CI got disabled because kwin had too many
>> pipelines and it takes about 10-15 minutes for a single pipeline to
>> complete.
>>
>> There was a long and somewhat heated discussion about it. We asked to
>> disable automatic CI for kwin specifically, so when you push to a work
>> branch that is not ready to be merged yet, no CI resources are wasted.
>> However, instead, automatic CI got disabled for all Plasma projects.
>> Sorry! :(
>
>
> Plasma in general was set to manual for all merge requests because the 
> commentary in that thread made it appear that the workflows were more Plasma 
> than just KWin. There are also issues with Appium tests in Plasma Desktop / 
> Workspace which mean pipelines take an extremely long amount of time to 
> complete/fail (something on the order of 40-50 minutes if memory serves)
>
> The overuse of CI resources by Plasma could in part be mitigated by marking 
> merge requests that are not ready to merge as draft, which seems to be a 
> Gitlab feature that is not in use at all at least if I take a cursory look at 
> current pending KWin MRs. We already set CI into manual mode for draft merge 
> requests as a global rule, as well as for work branches.

What confuses me right now is whether this is just plasma being an
active project or gitlab having poor behavior. It occurs to me that
the former would be solved by giving plasma dedicated plasma resources
to use, while the latter is a matter of hiring someone to engineer the
problem(s) with gitlab away.
Both solvable problems though.


Re: CI that doesn't run on MRs

2025-07-30 Thread Ben Cooksley
On Wed, Jul 30, 2025 at 7:18 PM Vlad Zahorodnii 
wrote:

> On 7/30/25 9:58 AM, Harald Sitter wrote:
> > Hey
> >
> > So, I need to vent a bit. I am super unhappy with the manual CI
> > trigger stuff. Half the time I forget.  When I don't forget I want to
> > press all the buttons, but I can't because of the prestage, I usually
> > sit there for 5 seconds, nothing happens because the prestage takes
> > more than 5 seconds, I get distracted and do something else ... and
> > forget I wanted to land something at all.
>

I was working on some merge requests for sysadmin/repo-metadata just a few
hours ago, which only contains basic linting jobs.
They were picked up and fully processed in the space of ~30 seconds - total
of 6 different linter jobs (so 6 containers spawned in total).

All seems quite reasonable for processing turnaround.


> >
> > More and more I find myself not caring and merging without pre-merge
> > CI. Between the manual triggers, failing appium tests, and conflicting
> > merges I just can't be bothered with the CI. Surely that can't be the
> > goal here.
> >
> > Can we get back a CI that helps?
>
> Hi,
>
> Just in case, the automatic CI got disabled because kwin had too many
> pipelines and it takes about 10-15 minutes for a single pipeline to
> complete.
>
> There was a long and somewhat heated discussion about it. We asked to
> disable automatic CI for kwin specifically, so when you push to a work
> branch that is not ready to be merged yet, no CI resources are wasted.
> However, instead, automatic CI got disabled for all Plasma projects.
> Sorry! :(
>

Plasma in general was set to manual for all merge requests because the
commentary in that thread made it appear that the workflows were more
Plasma than just KWin. There are also issues with Appium tests in Plasma
Desktop / Workspace which mean pipelines take an extremely long amount of
time to complete/fail (something on the order of 40-50 minutes if memory
serves)

The overuse of CI resources by Plasma could in part be mitigated by marking
merge requests that are not ready to merge as draft, which seems to be a
Gitlab feature that is not in use at all at least if I take a cursory look
at current pending KWin MRs. We already set CI into manual mode for draft
merge requests as a global rule, as well as for work branches.


>
> Regards,
> Vlad
>
>
Regards,
Ben


Re: CI that doesn't run on MRs

2025-07-30 Thread Vlad Zahorodnii

On 7/30/25 9:58 AM, Harald Sitter wrote:

Hey

So, I need to vent a bit. I am super unhappy with the manual CI
trigger stuff. Half the time I forget.  When I don't forget I want to
press all the buttons, but I can't because of the prestage, I usually
sit there for 5 seconds, nothing happens because the prestage takes
more than 5 seconds, I get distracted and do something else ... and
forget I wanted to land something at all.

More and more I find myself not caring and merging without pre-merge
CI. Between the manual triggers, failing appium tests, and conflicting
merges I just can't be bothered with the CI. Surely that can't be the
goal here.

Can we get back a CI that helps?


Hi,

Just in case, the automatic CI got disabled because kwin had too many 
pipelines and it takes about 10-15 minutes for a single pipeline to 
complete.


There was a long and somewhat heated discussion about it. We asked to 
disable automatic CI for kwin specifically, so when you push to a work 
branch that is not ready to be merged yet, no CI resources are wasted. 
However, instead, automatic CI got disabled for all Plasma projects. 
Sorry! :(


Regards,
Vlad