Re: [ovs-dev] [PATCH ovn v3 1/3] ci: Build container image before very job

2023-12-07 Thread Dumitru Ceara
On 11/23/23 13:38, Ales Musil wrote:
> Build the image before every job to allow more
> fine-grained dependency pinning. This is especially
> useful for stable branches as they might need to stay
> on specific distribution or Python version.
> 
> This also allows us to switch to Ubuntu instead
> of Fedora in the Cirrus CI for consistency reasons.
> 
> Signed-off-by: Ales Musil 
> ---
> v3: Fix the Cirrus CI architecture to ensure it is really running on ARM64.
> Add comment why do we need separate task in the Cirrus CI.
> ---

Looks good to me, thanks!

Acked-by: Dumitru Ceara 

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev


[ovs-dev] [PATCH ovn v3 1/3] ci: Build container image before very job

2023-11-23 Thread Ales Musil
Build the image before every job to allow more
fine-grained dependency pinning. This is especially
useful for stable branches as they might need to stay
on specific distribution or Python version.

This also allows us to switch to Ubuntu instead
of Fedora in the Cirrus CI for consistency reasons.

Signed-off-by: Ales Musil 
---
v3: Fix the Cirrus CI architecture to ensure it is really running on ARM64.
Add comment why do we need separate task in the Cirrus CI.
---
 .cirrus.yml| 44 ++--
 .github/workflows/test.yml | 46 +++---
 2 files changed, 80 insertions(+), 10 deletions(-)

diff --git a/.cirrus.yml b/.cirrus.yml
index 2d3824972..5453bf93d 100644
--- a/.cirrus.yml
+++ b/.cirrus.yml
@@ -1,15 +1,34 @@
-arm_unit_tests_task:
+compute_engine_instance:
+  image_project: ubuntu-os-cloud
+  image: family/ubuntu-2304-arm64
+  architecture: arm64
+  platform: linux
+  memory: 4G
+
+# Run separate task for the image build, so it's running only once outside
+# the test matrix.
+build_image_task:
+  install_dependencies_script:
+- sudo apt update
+- sudo apt install -y podman make
+
+  build_container_script:
+- cd utilities/containers
+- make ubuntu
+- podman save -o /tmp/image.tar ovn-org/ovn-tests:ubuntu
+
+  upload_image_script:
+- curl -s -X POST -T /tmp/image.tar 
http://$CIRRUS_HTTP_CACHE_HOST/${CIRRUS_CHANGE_IN_REPO}
 
-  arm_container:
-image: ghcr.io/ovn-org/ovn-tests:fedora
-memory: 4G
-cpu: 2
+arm_unit_tests_task:
+  depends_on:
+- build_image
 
   env:
-ARCH: aarch64
 CIRRUS_CLONE_SUBMODULES: true
 PATH: ${HOME}/bin:${HOME}/.local/bin:${PATH}
 RECHECK: yes
+IMAGE_NAME: ovn-org/ovn-tests:ubuntu
 matrix:
   - CC: gcc
 TESTSUITE: test
@@ -34,5 +53,16 @@ arm_unit_tests_task:
 
   name: ARM64 ${CC} ${TESTSUITE} ${TEST_RANGE}
 
+  install_dependencies_script:
+- sudo apt update
+- sudo apt install -y podman
+
+  download_cache_script:
+- curl http://$CIRRUS_HTTP_CACHE_HOST/${CIRRUS_CHANGE_IN_REPO} -o 
/tmp/image.tar
+
+  load_image_script:
+- podman load -i /tmp/image.tar
+- rm -rf /tmp/image.tar
+
   build_script:
-- ./.ci/linux-build.sh
+- ./.ci/ci.sh --archive-logs
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
index b3fcb57fc..fdbc8f5f5 100644
--- a/.github/workflows/test.yml
+++ b/.github/workflows/test.yml
@@ -80,10 +80,38 @@ jobs:
   if: steps.dpdk_cache.outputs.cache-hit != 'true'
   run: ./.ci/dpdk-build.sh
 
+  prepare-container:
+env:
+  DEPENDENCIES: podman
+name: Prepare container
+runs-on: ubuntu-22.04
+
+steps:
+  - uses: actions/checkout@v3
+
+  - name: Update APT cache
+run: sudo apt update
+
+  - name: Install dependencies
+run: sudo apt install -y ${{ env.DEPENDENCIES }}
+
+  - name: Build container
+run: make ubuntu
+working-directory: utilities/containers
+
+  - name: Export image
+run: podman save -o /tmp/image.tar ovn-org/ovn-tests
+
+  - name: Cache image
+id: image_cache
+uses: actions/cache@v3
+with:
+  path: /tmp/image.tar
+  key: ${{ github.sha }}
+
   build-linux:
-needs: build-dpdk
+needs: [build-dpdk, prepare-container]
 env:
-  IMAGE_NAME:  ghcr.io/ovn-org/ovn-tests:ubuntu
   ARCH:${{ matrix.cfg.arch }}
   CC:  ${{ matrix.cfg.compiler }}
   DPDK:${{ matrix.cfg.dpdk }}
@@ -152,13 +180,25 @@ jobs:
 sort -V | tail -1)
   working-directory: ovs
 
-- name: cache
+- name: cache dpdk
   if: matrix.cfg.dpdk != ''
   uses: actions/cache@v3
   with:
 path: dpdk-dir
 key: ${{ needs.build-dpdk.outputs.dpdk_key }}
 
+- name: image cache
+  uses: actions/cache@v3
+  with:
+path: /tmp/image.tar
+key: ${{ github.sha }}
+
+- name: load image
+  run: |
+sudo podman load -i /tmp/image.tar
+podman load -i /tmp/image.tar
+rm -rf /tmp/image.tar
+
 - name: build
   if: ${{ startsWith(matrix.cfg.testsuite, 'system-test') }}
   run: sudo -E ./.ci/ci.sh --archive-logs
-- 
2.42.0

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev