On Sat, Feb 24, 2024 at 9:21 PM Volker Krause <vkra...@kde.org> wrote:
> On Saturday, 24 February 2024 04:31:49 CET Ben Cooksley wrote: > > On Fri, Feb 23, 2024 at 11:12 PM Sune Vuorela <nos...@vuorela.dk> wrote: > > > On 2024-02-22, Nate Graham <n...@kde.org> wrote: > > > > I've started pondering post-megarelease projects. We've spent so > long on > > > > porting and bugfixing that I think it might be useful to shift gears > to > > > > feature work, and I'd like to brainstorm potential large-scale > projects > > > > and gauge the level of interest in putting resources into them soon. > > > > > > A bit more from the devops end that I'd love to see people tackle: > > > - Ensure frameworks and app unit tests interacting with windows can > run > > > > > > on Windows. > > > More details: The following fails on our windows CI > > > > https://invent.kde.org/sune/windows-test-thingie/-/blob/master/main.cpp > > > I find it weird that we are spending resources on putting things in > > > the windows store and making apps available on windows, but we can't > > > actually have passing tests in our CI. > > > > This unfortunately will not be easy to solve. > > And that's fine, we are in dreaming territory here anyway :) > > > One of the key things that we've learned out of doing CI, as has been > > showcased by FreeBSD in particular, is that the builders need to be > > ephemeral - that is only around for the build in question that is being > run. > > We're currently accomplishing this by using containers - via Podman (for > > Linux/Android/FreeBSD) and Docker (for Windows). > > > > Containers also offer us the advantage of allowing people to easily > > reproduce the CI environment on their local system without too much > trouble. > > > > For Windows however, Microsoft has limited the container stack to not > allow > > anything GUI related to work. The underlying libraries may be there, but > > the equivalent display server components are not operational. > > > > To complicate things further, on Windows certain permissions are > restricted > > to the interactive console and are not possible to do as either a > scheduled > > task or as a system service. > > Usage of existing Windows automation frameworks such as Powershell > Remoting > > or SSH will therefore not work if we want things to perfectly replicate a > > end user environment - because those will run the command(s) as part of a > > non-interactive session (even if the user we connect as is the same one > > logged in on the desktop console). > > > > Resolving this will therefore not only require running full sized Windows > > VMs (on an ephemeral basis to avoid the recently resolved issues that > used > > to plague FreeBSD), but would also need the VM to automatically login and > > spawn an agent as part of the interactive desktop session that we would > > connect to in order to run the CI tests. > > > > The spawning of a VM would require refactoring the setup of our poor CI > > workers (again - after they were only just recently completely rebuilt to > > allow the transition to Podman to fix flatpak-builder) to make use of > > something like: > > - > > > https://forum.gitlab.com/t/custom-executor-into-windows-vm-using-linux-kvm-l > > ibvirt/72713/5 - > > https://docs.gitlab.com/runner/executors/custom_examples/libvirt.html > > > > We would still have to write the agent that receives our commands > > (something like > > https://gist.github.com/cschwede/3e2c025408ab4af531651098331cce45 maybe) > > > > We would still have to solve the issue of how to share disk images among > > our nodes so they were built (ideally would be able to use Gitlab's > > Container Registry for this, which is something Tart can do - Tart being > a > > virtualisation management utility for ARM based Macs), as well as > > automating the installation of the various OSes we need to run this way. > > See > > > https://learn.microsoft.com/en-us/windows-hardware/manufacture/desktop/autom > > ate-windows-setup?view=windows-11 for some notes on that for Windows. > > > > The big downside to all of that of course is that the worker nodes would > > take longer to startup, and it would make sharing build artifacts much > more > > difficult and/or less efficient. > > > > > - Find a way to run unit tests on android CI. > > > > Likewise, this is very non-trivial - from my understanding our tooling > > currently for building Android software is centered around it all being > > cross compiled rather than being built on the native architecture it will > > be run on. > > Naturally tests cannot be run (unless you emulate, which I guess we could > > consider...) if the CPU architectures are not compatible. > > > > Even if you emulate though, I imagine we would need to provide a full > > Android system setup (including all of it's relevant bits of system > > infrastructure, such as it's display server DisplayFlinger) in order for > > tests to truly be of use. > > The path of least resistance is probably by making use of Google's > existing > > Android emulator - although i've no idea how you would tie that into CI. > > Right, the Android emulator is probably the first thing to look at for > this. > That alone isn't enough though, we can't just copy over a unit test > executable > and run it there, this needs to be wrapped in an APK (and which > potentially > needs to be given permissions, as we don't want system UI dialogs popping > up, > etc). > > The Qt CI seems to be doing something like this meanwhile, might be worth > investigating how this is done there. > > > We would need to have our build chains ready to build on a native system > > before we could think about this I think. Building Android x86/x64 > wouldn't > > cover things off properly due as it won't reflect the actual CPUs being > > used on end-user devices (and ARM processors can expose issues that don't > > happen on x86/x64 based systems) > > For Android I'd rate that less relevant actually, we'd get by far the most > things covered with x86_64 as well. I'm only aware of one ARM-specific > runtime > issue recently, and that was ARM32 only even. The vast majority of > problems > are either related to APK packaging or interaction with (changing) Android > platform infrastructure, all of which is platform-independent. And the > majority of ARM-specific issues we actually encounter during building. > > ARM64 native builds nevertheless belong on this wishlist, but IMHO more > importantly for ARM Flatpaks for the mobile apps. > I already have a ticket for that - https://phabricator.kde.org/T17120 Hardware will likely be provided via a CAX31 / CAX41 ( https://www.hetzner.com/cloud/) as the number of builds isn't likely to be huge and therefore cannot justify a RX170 ( https://www.hetzner.com/dedicated-rootserver/matrix-rx/) (which would make more sense if all our Android builds were ARM native) At this stage i'd like to get us comfortably through into the post-Binary Factory world before anything further takes place though (some teething issues have popped up that weren't immediately evident prior to it's decommissioning that will need resolving). >From my side, the next big thing to start breaking down and building via CI would be docs.kde.org. > > > > - Make autotests guarding on all our CI's. > > > > > > - Clazy and clang-tidy and cppcheck on all our repositories in CI > > > > > > /Sune > > > > Hopefully the above is food for thought as to why we don't have the > above - > > don't mean to say it can't be done, but it certainly is not trivial to > > accomplish... > > Sure, and likewise the wishlist isn't meant as criticism of what we have, > on > the contrary, IMHO our CI/CD system has made huge steps forward with the > migration to Gitlab and to container runners as well as the new signing > service, but now we are hooked for more ;-) > > Yes, those are ambitious ideas, but they would significantly help > application > development I think, and assuming Nate's original question was also about > where the e.V. might want to invest resources into going forward, our > CI/CD > infrastructure would IMHO be a good area to (continue to) invest in. > > Regards, > Volker > Cheers, Ben
