On 2/3/26 11:43 AM, Felix Huettner via dev wrote:
Hi everyone, the following is a not-yet-working preparation for running the OVN CI on custom hosted runners. The problem with normal Github Actions is that the CI Job is run directly on the CI runner. For the public runners they are spawned and afterwards destroyed by github themselves. For custom CI runners that makes things quite complicated as there is no nice implementation of it. However github actions also supports running the actions within containers. This brings the benefit that we have finally a well known system where we start from and that the building host will not be filled with trash.
I don't think trash is a *huge* problem (there are known solutions - including marketplace actions you can pull, like https://github.com/marketplace/actions/free-disk-space-ubuntu). But I agree that pinning a container to an exact version of all dependencies instead of letting GitHub decide for you is a good idea.
It comes with a price, e.g. managing the image repository. (Beyond initial setup, I think it's just making sure admin creds are not leaked.)
Also it seems that custom runners natively support this which would make custom runners significantly easier to use. The below patch is a initial idea (but not yet working) of a migration of the "Build and Test" CI job. While a few tests work okish (like the first 40 tests or so) anything that spawns an ovsdb has issues with the "appctl exit" command not terminating. I have not yet found out why.
You need an init system to harvest processes properly. I think if you revert the entrypoint change, tests will work again.
If anyone has done something similar in the past and has some idea what might be wrong here i would be interested in help.
You may check my `debian` branch where I use trixie container for .deb build: https://github.com/booxter/ovn/tree/debian (I've just sent the branch series for review, so you can pull it from patchwork too).
My current alternative plan is to run the "build-linux" job directly with the image that "prepare-container" generates. Maybe this would make things easier (however we then need a temporary docker registry). If you want to try this out you can just push it to a branch on your fork of the github repo. Thanks a lot, Felix Signed-off-by: Felix Huettner <[email protected]> --- .ci/ci.sh | 6 ++- .ci/linux-util.sh | 10 +--- .github/workflows/test.yml | 63 ++++++++++++++------------ utilities/containers/ubuntu/Dockerfile | 2 +- 4 files changed, 39 insertions(+), 42 deletions(-) diff --git a/.ci/ci.sh b/.ci/ci.sh index 3640d3243..23c343925 100755 --- a/.ci/ci.sh +++ b/.ci/ci.sh @@ -171,11 +171,13 @@ fi CONTAINER_ID="$($CONTAINER_CMD run --privileged -d \ --pids-limit=-1 \ --security-opt apparmor=unconfined \ + --cgroupns=host \ + --cgroups=no-conmon \ --env ASAN_OPTIONS=$ASAN_OPTIONS \ - -v /lib/modules/$(uname -r):/lib/modules/$(uname -r):ro \ + -v /host/lib/modules/$(uname -r):/lib/modules/$(uname -r):ro \ -v $OVN_PATH:$CONTAINER_WORKSPACE/ovn:Z \ -v $OVS_PATH:$CONTAINER_WORKSPACE/ovs:Z \ - $IMAGE_NAME)" + $IMAGE_NAME tail -f /dev/null)" trap remove_container EXIT copy_sources_to_workdir diff --git a/.ci/linux-util.sh b/.ci/linux-util.sh index b5bd1f8c9..e4f5da377 100755 --- a/.ci/linux-util.sh +++ b/.ci/linux-util.sh @@ -36,16 +36,8 @@ function fix_etc_hosts() cp /etc/hosts ./hosts.bak sed -E -n \ '/^[[:space:]]*(#.*|[0-9a-fA-F:.]+([[:space:]]+[a-zA-Z0-9.-]+)+|)$/p' \ - ./hosts.bak | sudo tee /etc/hosts + ./hosts.bak | tee /etc/hosts diff -u ./hosts.bak /etc/hosts || true } -# Workaround until https://github.com/actions/runner-images/issues/10015 -# is resolved in some way. -function disable_apparmor() -{ - # https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/2093797 - sudo aa-teardown || true - sudo systemctl disable --now apparmor.service -} diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index b6e461129..822810eb7 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -22,31 +22,32 @@ jobs: # +-------+-------------------+-------------------+ # | !main | Builds - Ubuntu | xxxxxxxxxxxxxxxxx | # +-------+-------------------+-------------------+ + defaults: + run: + shell: bash env: - DEPENDENCIES: podman + DEPENDENCIES: podman make name: Prepare container if: github.repository_owner == 'ovn-org' || github.event_name != 'schedule' runs-on: ubuntu-24.04 + container: + image: ubuntu:24.04 + options: --privileged steps: - uses: actions/checkout@v4 - name: Update APT cache - run: sudo apt update + run: apt update - name: Install dependencies - run: sudo apt install -y ${{ env.DEPENDENCIES }} + run: apt install -y ${{ env.DEPENDENCIES }} - name: Fix /etc/hosts file run: | . .ci/linux-util.sh fix_etc_hosts - - name: Disable apparmor - run: | - . .ci/linux-util.sh - disable_apparmor - - name: Choose image distro if: github.event_name == 'push' || github.event_name == 'pull_request' run: | @@ -72,15 +73,17 @@ jobs: - name: Export image run: podman save -o /tmp/image.tar --format oci-archive ovn-org/ovn-tests - - name: Cache image - id: image_cache - uses: actions/cache@v4 + - name: Upload image + uses: actions/upload-artifact@v4 with: + name: image-tar path: /tmp/image.tar - key: ${{ github.sha }}/${{ github.event_name }} build-linux: needs: [prepare-container] + defaults: + run: + shell: bash env: ARCH: ${{ matrix.cfg.arch }} CC: ${{ matrix.cfg.compiler }} @@ -91,9 +94,15 @@ jobs: TEST_RANGE: ${{ matrix.cfg.test_range }} SANITIZERS: ${{ matrix.cfg.sanitizers }} UNSTABLE: ${{ matrix.cfg.unstable }} + DEPENDENCIES: build-essential git podman name: linux ${{ join(matrix.cfg.*, ' ') }} runs-on: ubuntu-24.04 + container: + image: ubuntu:24.04 + options: --privileged + volumes: + - /lib/modules:/host/lib/modules strategy: fail-fast: false @@ -126,11 +135,17 @@ jobs: - { arch: x86, compiler: gcc, opts: --disable-ssl } steps: + - name: Update APT cache + run: apt update + + - name: Install dependencies + run: apt install -y ${{ env.DEPENDENCIES }} + - name: system-level-dependencies if: ${{ startsWith(matrix.cfg.testsuite, 'system-test') }} run: | - sudo apt update - sudo apt -y install linux-modules-extra-$(uname -r) + apt update + apt -y install linux-modules-extra-$(uname -r) - name: checkout if: github.event_name == 'push' || github.event_name == 'pull_request' @@ -166,30 +181,18 @@ jobs: . .ci/linux-util.sh fix_etc_hosts - - name: Disable apparmor - run: | - . .ci/linux-util.sh - disable_apparmor - - - name: image cache - id: image_cache - uses: actions/cache@v4 + - name: download image artifact + uses: actions/download-artifact@v4 with: - path: /tmp/image.tar - key: ${{ github.sha }}/${{ github.event_name }} + name: image-tar + path: /tmp - name: load image run: | - sudo podman load -i /tmp/image.tar podman load -i /tmp/image.tar rm -rf /tmp/image.tar - name: build - if: ${{ startsWith(matrix.cfg.testsuite, 'system-test') }} - run: sudo -E ./.ci/ci.sh --archive-logs --timeout=2h - - - name: build - if: ${{ !startsWith(matrix.cfg.testsuite, 'system-test') }} run: ./.ci/ci.sh --archive-logs --timeout=2h - name: upload logs on failure diff --git a/utilities/containers/ubuntu/Dockerfile b/utilities/containers/ubuntu/Dockerfile index bf64974de..01e8b969e 100755 --- a/utilities/containers/ubuntu/Dockerfile +++ b/utilities/containers/ubuntu/Dockerfile @@ -66,4 +66,4 @@ ENV TZ Etc/UTC RUN /tmp/prepare.sh -CMD ["/sbin/init"] +CMD ["/bin/sh"] base-commit: 081d6e6d8dd5645a2b651621ec401f5aa2aa7f08 -- 2.43.0 _______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
_______________________________________________ dev mailing list [email protected] https://mail.openvswitch.org/mailman/listinfo/ovs-dev
