On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela <nos...@vuorela.dk> wrote:
> On 2024-02-22, Nate Graham <n...@kde.org> wrote: > > I've started pondering post-megarelease projects. We've spent so long on > > porting and bugfixing that I think it might be useful to shift gears to > > feature work, and I'd like to brainstorm potential large-scale projects > > and gauge the level of interest in putting resources into them soon. > > A bit more from the devops end that I'd love to see people tackle: > > - Ensure frameworks and app unit tests interacting with windows can run > on Windows. > More details: The following fails on our windows CI > https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp > I find it weird that we are spending resources on putting things in > the windows store and making apps available on windows, but we can't > actually have passing tests in our CI. > This unfortunately will not be easy to solve. One of the key things that we've learned out of doing CI, as has been showcased by FreeBSD in particular, is that the builders need to be ephemeral - that is only around for the build in question that is being run. We're currently accomplishing this by using containers - via Podman (for Linux/Android/FreeBSD) and Docker (for Windows). Containers also offer us the advantage of allowing people to easily reproduce the CI environment on their local system without too much trouble. For Windows however, Microsoft has limited the container stack to not allow anything GUI related to work. The underlying libraries may be there, but the equivalent display server components are not operational. To complicate things further, on Windows certain permissions are restricted to the interactive console and are not possible to do as either a scheduled task or as a system service. Usage of existing Windows automation frameworks such as Powershell Remoting or SSH will therefore not work if we want things to perfectly replicate a end user environment - because those will run the command(s) as part of a non-interactive session (even if the user we connect as is the same one logged in on the desktop console). Resolving this will therefore not only require running full sized Windows VMs (on an ephemeral basis to avoid the recently resolved issues that used to plague FreeBSD), but would also need the VM to automatically login and spawn an agent as part of the interactive desktop session that we would connect to in order to run the CI tests. The spawning of a VM would require refactoring the setup of our poor CI workers (again - after they were only just recently completely rebuilt to allow the transition to Podman to fix flatpak-builder) to make use of something like: - https://forum.gitlab.com/t/custom-executor-into-windows-vm-using-linux-kvm-libvirt/72713/5 - https://docs.gitlab.com/runner/executors/custom_examples/libvirt.html We would still have to write the agent that receives our commands (something like https://gist.github.com/cschwede/3e2c025408ab4af531651098331cce45 maybe) We would still have to solve the issue of how to share disk images among our nodes so they were built (ideally would be able to use Gitlab's Container Registry for this, which is something Tart can do - Tart being a virtualisation management utility for ARM based Macs), as well as automating the installation of the various OSes we need to run this way. See https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/automate-windows-setup?view=windows-11 for some notes on that for Windows. The big downside to all of that of course is that the worker nodes would take longer to startup, and it would make sharing build artifacts much more difficult and/or less efficient. > > - Find a way to run unit tests on android CI. > Likewise, this is very non-trivial - from my understanding our tooling currently for building Android software is centered around it all being cross compiled rather than being built on the native architecture it will be run on. Naturally tests cannot be run (unless you emulate, which I guess we could consider...) if the CPU architectures are not compatible. Even if you emulate though, I imagine we would need to provide a full Android system setup (including all of it's relevant bits of system infrastructure, such as it's display server DisplayFlinger) in order for tests to truly be of use. The path of least resistance is probably by making use of Google's existing Android emulator - although i've no idea how you would tie that into CI. We would need to have our build chains ready to build on a native system before we could think about this I think. Building Android x86/x64 wouldn't cover things off properly due as it won't reflect the actual CPUs being used on end-user devices (and ARM processors can expose issues that don't happen on x86/x64 based systems) > > - Make autotests guarding on all our CI's. > > - Clazy and clang-tidy and cppcheck on all our repositories in CI > > /Sune > Hopefully the above is food for thought as to why we don't have the above - don't mean to say it can't be done, but it certainly is not trivial to accomplish... Cheers, Ben
