On 11 May 2026 10:42:39 pm ACST, Kristen McWilliam <[email protected]> wrote: >Hello, and welcome to the new Plasma CI Health Check! > >You may be familiar with the "KDE Gear projects with failing CI" >project/emails that Albert has been doing for some time — I will be doing >similar for Plasma, to ensure that our CI is healthy and improve our testing >strategy. > >We'll be looking at a few interesting metrics, for example whether: > >• CI is passing or failing on master > >If CI is failing on master its usually a bad sign, and something we want to >fix ASAP. > > >• Tests are being automatically retried > >Having the CI configured to auto retry tests is usually a sign of a workaround >for flaky tests, and flaky tests are bad tests; at best they are frustrating >and a waste of CI resources, at worst they give a false sense of security that >code is actually reliable. A test should either pass or fail — if it doesn't >do so reliably it should be fixed or removed. > >In the .kde-ci.yml this will look something like: > >```yaml >Options: > ctest-arguments: '--repeat until-pass:5' >``` > >We should prefer to remove this option and fix the tests. > > >• Tests are configured as required passing for the repo > >With this enabled, if tests fail in the CI the associated merge request will >show this as a nice obvious warning; if it isn't enabled, the MR will report >the pipeline is completely green — not good! > >We definitely want it to be obvious if the tests failed, otherwise people are >likely to merge it without even being aware there is an issue. > >In the .kde-ci.yml this will look like: > >```yaml >Options: > require-passing-tests-on: ['@all'] >``` > >Knowing if the tests are passing or failing is super important, so we need to >work towards this being enabled for all our repos. > > >--------------------------- > > >There will be additional things to track to assess our CI and test health, but >this is a good place to start. > >Here are our initial metrics: > >• 86 monitored repos >• 10 failing CI in master >• 3 configured for auto-repeating tests >• 14% of repos configured to require passing tests > > >Failing CI: > >• plasma/qqc2-breeze-style >• plasma/plasma-desktop >• plasma/kmenuedit >• plasma/discover >• plasma/lightdm-kde-greeter >• plasma/kde-gtk-config >• plasma/plasma-sdk >• plasma/plasma-systemmonitor >• plasma/latte-dock >• plasma-mobile/spacebar > > >Auto-retrying tests: > >• kwin-x11 >• kwin >• plasma-desktop > > >I hacked together a simple dashboard to help visualize the state of things, >which also includes links to the relevant CI run for each repo; if interested >in checking, it lives here for the moment: > >https://ci-healthcheck-5e5c80.local-kde.org/ > > >Thank you for helping to improve the reliability of our software, and reducing >wasted CI resources. > >-- >Cheers, > Kristen >
If that is your idea of a simple dashboard, mine are straight up Neanderthal. Great work!
