On Wed, Feb 4, 2026 at 9:07 PM Ben Cooksley <[email protected]> wrote:
> On Wed, Feb 4, 2026 at 8:25 PM Vlad Zahorodnii <[email protected]> > wrote: > >> Hello, >> > > HI Vlad, > > >> >> Over approximately the past weekend something happened in our CI and now >> it takes quite long time for tests to run. For example, in kwin, we have >> a test that used to run for about 20 seconds, and now it takes about 5 >> or so minutes to finish running. Speaking for kwin, there were no >> changes that could increase test run times so dramatically. >> >> January 26th: >> >> Start 61: kwin-testOutputChanges >> 61/158 Test #61: kwin-testOutputChanges >> ............................. Passed 19.36 sec >> >> January 29th: >> >> Start 61: kwin-testOutputChanges >> 61/158 Test #61: kwin-testOutputChanges >> ............................. Passed 43.93 sec >> >> January 30th: >> >> Start 61: kwin-testOutputChanges >> 61/158 Test #61: kwin-testOutputChanges >> ............................. Passed 45.91 sec >> >> Februrary 3rd: >> >> Start 61: kwin-testOutputChanges >> 61/158 Test #61: kwin-testOutputChanges >> ............................. Passed 254.19 sec >> >> FreeBSD appears to be fine. >> >> We suspect that test run times blew up due to enabling LSAN in various >> libraries (kwin itself has no LSAN enabled yet). The issue doesn't >> appear to be specific to only kwin, people reported that they've seen >> similar issues in other projects too. Maybe something else happened to >> CI that sysadmins will be able to clarify. >> > > Nothing else happened to CI recently aside from the enablement of LSAN. > > The underlying SUSE images were for Qt 6.10 at least last rebuilt on > January 25th, which is well before your "last good" date. > > The only change to CI between January 30th and February 3rd > was fast_unwind_on_malloc=0 being added by default, even though it is > primarily for the benefit of LSAN. > I've now made changes to only set fast_unwind_on_malloc=0 if LSAN is > explicitly enabled for a repository - hard to tell if that will fix the > issue though as KWin takes a while to build. > For the record, as per https://invent.kde.org/plasma/kwin/-/jobs/3958895 which completed moments ago: Start 61: kwin-testOutputChanges 61/158 Test #61: kwin-testOutputChanges ............................. Passed 19.58 sec > > >> >> Either way, the current state of CI is not great. Hypothetically, test >> timeouts can be increased but QSignalSpy's have hardcoded timeouts that >> can be too low for the current CI. And in case of kwin, 5 minutes for a >> test is simply not a workable thing. >> >> Regards, >> Vlad >> > > Thanks, > Ben > Cheers, Ben
