Am 03.06.25 um 11:42 schrieb Ben Cooksley:
On Tue, Jun 3, 2025 at 9:03 AM Albert Astals Cid <aa...@kde.org> wrote:
El dilluns, 2 de juny del 2025, a les 13:39:21 (Hora d’estiu d’Europa
central), Ben Cooksley va escriure:
> Hi all,
>
> For some time now we have had a variety of issues with our
Docker/Podman
> based CI builds. These have included the lack of GUI test support on
> Windows, periodic crashes on FreeBSD, poor IO performance of Windows
> builds, issues supporting builds for Flatpak and Snaps and
inability to
> support either builds or tests where elevated privileges or
system session
> resources are needed.
>
> In addition to this we've had issues where Linux CI builds have the
> capability to trigger OOM events on the CI hosts, which in turn
takes out
> Windows and (less often) FreeBSD builders. While this does not
occur too
> often, it does happen from time to time and eventually
negatively impacts
> the build queue for those platforms.
>
> The need to have dedicated VMs for FreeBSD and Windows on our
builders also
> makes setting up of a CI build node for KDE software a more
complicated and
> time intensive task than it otherwise needs to be (and means
that the
> amount of systems to care for increases by 3 for every CI node
we add).
>
> While individually relatively minor, together these issues more than
> justify making a significant change to the way we run our CI
system - in
> this case transitioning from container based builds to VM based
builds.
>
> These builds will still take place on dedicated hardware that we
control,
> however instead of taking place within a container (managed by
Podman on
> Linux and FreeBSD, or Docker on Windows) they will instead take
place
> within a VM using a copy-on-write disk image.
> VM based builds will unfortunately take a little longer to start
(it takes
> ~10 seconds for a VM from any of Linux, FreeBSD or Windows to
boot on my
> personal system) however the benefits we gain should more than
outweigh
> this small downside.
>
> This has been under development for the past couple of weeks and
is now
> reaching the point where the only remaining steps are to get it
integrated
> with the Gitlab CI agent (gitlab-runner) for which prototype code is
> already in place, and complete porting of our images over. Once that
> happens a complete rebuild of all of our builders will be swiftly
> undertaken to transition them completely over to the new VM based
> infrastructure.
>
> Specs wise, at this time it is planned for each spawned standard
VM to be
> provided with 2/3's of the system CPU cores (so 12 cores), 16GB
RAM and
> 100GB of disk space (although some of that will be occupied by
the system
> image - approximately 10GB for standard Linux builds and ~30GB
or so for
> Windows builds). There will be a higher resource tier available
for certain
> builds however that will be on request only and would need to be
justified
> (such as Craft needing to build QtWebEngine).
>
> As launching VMs is not the most efficient approach for all
workloads,
> limited support for running Docker containers will be preserved,
however
> this support is primarily intended for running linters, sanity
checks and
> website builds, and is not intended for running general CI/CD
builds.
>
> The tooling used by the CI nodes to run VMs is something that
should be
> fairly trivial for people to run on their own local system
should they wish
> to run any of those images (say for FreeBSD or Android),
although you will
> need to setup libvirt yourself (SUSE has very good instructions
for this,
> Debian less so as their instructions lack installing the
packages needed to
> provide UEFI and TPM support). The tooling itself was merged
this evening
> to sysadmin/ci-images (vm-common/ folder) and can be used with
the VM
> images found at https://storage.kde.org/vm-images/
>
> There is however one downside to this - Qt 5 support.
>
> Over the past few months distributions have been steadily
removing packages
> and other supporting infrastructure needed to keep Qt 5 builds
alive. In
> the case of Windows, support for the entire Qt 5 tree has been
unmaintained
> for some time. For FreeBSD and SUSE a significant number of
packages have
> been removed - which in the case of SUSE also includes packages
needed to
> support the building of KJS. Accordingly, because builds of
Frameworks are
> a first stepping stone to support building anything else, it
will not be
> possible for us to produce Qt 5 based VM build images for any of
the 3
> platforms.
>
> We will therefore have to remove Qt 5 support from the CI system
with the
> transition to VM based CI.
From previous discussions I had the impression this was only for
things that
wanted to create packages and not for "want to have CI to
compile/run tests".
Can you confirm you are proposng a total annihilation of Qt5
support in our
CI?
At the time we had that discussion it was still possible to build some
of the Qt 5 images, however that is no longer the case - all of them
now fail to build.
In the case of the suse-qt515 image, the removal of libpcre in SUSE
means it is no longer possible to build KJS.
Consequently, we're no longer able to build all Frameworks (making
'kf5' branch CI for Frameworks non-functional) so there isn't much
point in looking further to support Qt 5 on Linux.
Not that I have much sympathy left for Qt5, but wouldn't it be possible
to exclude kjs and keep the remaining frameworks? I don't think any of
the unported projects need kjs anyway.
For FreeBSD the story is much the same as SUSE - packages are being
removed as apps upgrade to Qt 6 and the Qt 5 version of libraries
becomes surplus to requirements.
For Windows the continued operation of that CI has only been possible
because our existing images are still around - new ones cannot be built.
That has been the case for a significant amount of time now, and it is
not worth the investment to fix it as everyone who works on Windows
has moved on to Qt 6.
In essence there is little we can do to keep this alive -
distributions are removing support so we must follow suit.
The correct course of action is to accelerate the porting of the
remaining applications, not to delay and keep Qt 5 alive.
Cheers,
Albert
Regards,
Ben
>
> Please let me know if there are any questions on the above.
>
> Thanks,
> Ben
