For those that couldn't attend. It was a longer than usual meeting. But we will continue the discussions on list of course.
- New hardware and how forgejo action runners can use it - Servers, VMs and services - 2026 infrastructure goals and budget plans for paid staff On Fri, Mar 13, 2026 at 02:30:10AM +0100, Mark Wielaard wrote: > Last month we finalized our hardware refresh cycle and moved all > services off bare metal servers onto VMs (for x86_64 hardware) which > greatly simplifies backups, redundancy and moving services around. For > hardware we should be good for the next couple of years now and thanks > to your generous donations our hardware refresh fund is back up > strenght so we are able to replace any server in case they might > break. We now also have two extra forgejo action runners (sw3runner1: > 40 vcpus, 256GB mem, 500GB disk and sw3runner2: 16 vcpus, 96GB mem, > 420GB disk) which can be used in forge workflows. = New hardware and how forgejo action runners can use it The new forge action runners sw3runner1 and sw3runner2 seem to pick up existing workflows. But that isn't really taking advantage of the new 40/16 vcpu setups. It doesn't look like we have global stats, but this is for gcc: https://forge.sourceware.org/gcc/gcc-TEST/actions There's workflows and there are actions. Workflows use pre-made images. Actions can make them on the fly it seems. We are currently using workflows and it isn't clear how in forgejo one can create images on the fly (from container files as the buildbot does). For elfutils the workflow uses an image/action https://forge.sourceware.org/elfutils/elfutils/src/branch/main/.forgejo/workflows Our runners are in a isolated VM https://forgejo.org/docs/latest/admin/actions/docker-access/#forgejo-runner-in-a-vm which spawn containers for actions/workflows. When you use sourceware-debian-runner it will be debian stable, when using sourceware-fedora-runner it will be fedora latest. The runner adds labels: https://forgejo.org/docs/latest/admin/actions/#choosing-labels There is an increase in forge spam signups but a) they seem to never activate their account. And b) if they do they are restricted, so that is fine. We don't have a good workflow for "real" signups. Those people should be added to a Contributor group for a project to be able to participate. Those people are still restricted though, so they can only contribute to one specific project. It might be nice if we could white-list certain domains so that folk signing up through those had fewer hoops to jump through. It could be automated with the api. https://forge.sourceware.org/api/swagger/#/admin/adminEditUser = Servers, VMs and services Discussion about all the different servers we have now and which services, backups and redundant services run where. There is the osuosl openstack cluster where snapshots runs in a VM. There is sourceware-builder3 which has 4 VMs. sw3runner1, sw3runner2, sw3bb1 and sw3bb2. There is server1 which has 8 VMs, currently "sourceware", forge-stage, debuginfod, and forge (the new one) with slots for inbox, builder, patchwork and bunsen. There is the RH OSPO openstack cluster where the current forge still runs (already synced with the new one). Then there are server2 which is setup for running VMs but doesn't yet. And server3 which should have a sourceware hot backup VM (or vice versa server2/server3; server2 has more ram) Mark promises an ascii art drawing. = 2026 infrastructure goals and budget plans for paid staff We are setup pretty nice for hardware and hosting now. And our hardware refresh fund is also back up to about $8000+. We should be good for the next couple of years now. So for this year we would like to get a budget setup to help some of the volunteer admins to get paid staff help to finalize some of our (security) plans https://sourceware.org/sourceware-security-vision.html#plans There are two questions here. First is if the plans are complete are we missing a broad category? Second what it would take to hire paid staff/contractors to do some of the work specified (we could of course do everything ourselves, but that might take much longer) Discussion about how at Fosdem we started the process to estimate how many hours each task would take, what level of expertise someone would need and how much they cost. e.g. different levels of contractors or (junior) adminstrators. The hours/days it would take the volunteer admins, and factor for someone not yet familiar with setups. Next up we'll do a community survey, like we did last year, to see which services are most important and what priority which plans have. And setup a meeting with the SFC staff to finalize the budget, publish it and get a grant or corporate donation to execute. > Keep Sourceware worry-free, friendly and independent by donating > https://sourceware.org/donate.html support our fiscal sponser SFC > https://sfconservancy.org/sustainer and/or support OSUOSL for > hosting Free Software projects https://osuosl.org/donate/ > > Do you or your company want to sponsor Sourceware plans financially > https://sourceware.org/sourceware-security-vision.html#plans > donate hardware or services then contact us at [email protected]
