On Wed, Jul 12, 2023 at 11:32 PM Ilya Maximets <[email protected]> wrote:
>
> On 6/27/23 12:11, Frode Nordahl wrote:
> > The autopkgtests [0][1] are relevant in an upstream context
> > because an Open vSwitch contributor may want to have a quick
> > way of running the upstream system testsuites on recent
> > Debian/Ubuntu releases in an automated and contained manner.
> >
> > During the Debian/Ubuntu/upstream package source sync work [2], a
> > relatively naive autopkgtest was added. It had been around since
> > Open vSwitch was initially packaged many years ago.
> >
> > Replace the autopkgtest with a test that runs all the upstream
> > system testsuites instead.
> >
> > To run the autopkgtest, take a look at [1] for prerequisites then:
> >
> > ./boot.sh && \
> > ./configure \
> > --prefix=/usr \
> > --localstatedir=/var \
> > --sysconfdir=/etc \
> > --with-dpdk=shared && \
> > make debian-source
> > autopkgtest \
> > --env DEB_BUILD_OPTIONS="nocheck parallel=32" \
> > openvswitch_3.1.90-1.dsc \
> > -- qemu \
> > --cpus 32 \
> > --ram-size 8129 \
> > autopkgtest-mantic-amd64.img
> >
> > This patch also adds Cirrus CI jobs that run these tests.
> >
> > 0: https://wiki.debian.org/ContinuousIntegration/autopkgtest
> > 1: https://packaging.ubuntu.com/html/auto-pkg-test.html
> > 2: https://mail.openvswitch.org/pipermail/ovs-dev/2022-July/396219.html
> >
> > Signed-off-by: Frode Nordahl <[email protected]>
>
> Hi, Frode. Thanks for the patch!
> Though it looks like there are 3 separate changes:
> - make debian-source
> - autopkgtest
> - CI
>
> Could you split the patch in 3?
Yes, that makes sense.
> I also find it a bit confusing that DPDK test is running AF_XDP
> testsuite as well. May not be needed?
I added that mostly to confirm that building a packaged binary that
supported both DPDK and AF_XDP works as expected, as I missed
supporting that in an earlier iteration of this series. I guess now
that we have established that it works, we don't really need to keep
validating that for every run. Will remove it.
> Also, it looks like tests are taking less than half of the runtime
> in CI. Is there a way to re-use the build for multiple tests maybe?
> I'm not sure.
There is usually some way to share artifacts across runs, I'll check
what Cirrus can provide us with here. Thank you for the suggestion.
> > ---
> > .cirrus.yml | 102 ++++++++++++++
> > .gitignore | 1 +
> > Documentation/intro/install/debian.rst | 23 +++-
> > debian/automake.mk | 33 ++++-
> > debian/rules | 5 +
> > debian/tests/afxdp | 1 +
> > debian/tests/control | 41 +++++-
> > debian/tests/dpdk | 46 +------
> > debian/tests/kmod | 1 +
> > debian/tests/offloads | 1 +
> > debian/tests/openflow.py | 66 ---------
> > debian/tests/run-tests.sh | 183 +++++++++++++++++++++++++
> > debian/tests/userspace | 1 +
> > debian/tests/vanilla | 29 ----
> > 14 files changed, 380 insertions(+), 153 deletions(-)
> > create mode 120000 debian/tests/afxdp
> > mode change 100755 => 120000 debian/tests/dpdk
> > create mode 120000 debian/tests/kmod
> > create mode 120000 debian/tests/offloads
> > delete mode 100755 debian/tests/openflow.py
> > create mode 100755 debian/tests/run-tests.sh
> > create mode 120000 debian/tests/userspace
> > delete mode 100755 debian/tests/vanilla
> >
> > diff --git a/.cirrus.yml b/.cirrus.yml
> > index 952d96431..7d3ae86d9 100644
> > --- a/.cirrus.yml
> > +++ b/.cirrus.yml
> > @@ -32,3 +32,105 @@ freebsd_build_task:
> > check_script:
> > - gmake -j8 check TESTSUITEFLAGS=-j8 RECHECK=yes
> > || { cat ./tests/testsuite.log; exit 1; }
> > +
> > +# This tests both the upstream OVS Debian packaging recipe and executes
> > all the
> > +# system testsuites on Ubuntu.
> > +ubuntu_autopkgtest_latest_task:
> > +
> > + compute_engine_instance:
> > + image_project: ubuntu-os-cloud
> > + image: family/ubuntu-2304-amd64
> > + platform: linux
> > + cpu: 4
> > + memory: 4G
> > +
> > + env:
> > + DEPENDENCIES: |
> > + build-essential fakeroot dpkg-dev devscripts equivs autopkgtest
> > autodep8
> > + libbpf-dev libxdp-dev
> > + DPDK: shared
> > + matrix:
> > + TEST_NAME: afxdp
> > + TEST_NAME: dpdk
> > + TEST_NAME: kmod
> > + # Pending resolution of https://launchpad.net/bugs/2020677
> > + # TEST_NAME: offloads
> > + TEST_NAME: userspace
> > +
> > + prepare_script: |
> > + apt-get update
> > + if [ "${DPDK}" = no ]; then
> > + echo "Skipping DPDK dependency."
> > + else
> > + apt-get -y install libdpdk-dev
> > + fi
> > + apt-get --yes install linux-headers-$(uname -r) ${DEPENDENCIES}
> > +
> > + configure_script: |
> > + ./boot.sh
> > + ./configure \
> > + --prefix=/usr \
> > + --localstatedir=/var \
> > + --sysconfdir=/etc \
> > + --with-dpdk=${DPDK}
> > +
> > + build_script:
> > + - make debian-source
> > +
> > + check_script: |
> > + if [ "${DPDK}" = "no" ]; then
> > + deb_dpdk_build_opt="nodpdk"
> > + else
> > + deb_dpdk_build_opt=""
> > + fi
> > + autopkgtest \
> > + --env DEB_BUILD_OPTIONS="nocheck parallel=8 ${deb_dpdk_build_opt}"
> > \
> > + --test-name $TEST_NAME \
> > + *.dsc \
> > + -- null
> > +
> > +ubuntu_autopkgtest_lts_task:
> > +
> > + compute_engine_instance:
> > + image_project: ubuntu-os-cloud
> > + image: family/ubuntu-2204-lts
> > + platform: linux
> > + cpu: 4
> > + memory: 4G
> > +
> > + env:
> > + DEPENDENCIES: |
> > + build-essential fakeroot dpkg-dev devscripts equivs autopkgtest
> > autodep8
> > + DPDK: no
> > + matrix:
> > + TEST_NAME: kmod
> > + # Pending resolution of https://launchpad.net/bugs/2020677
> > + # TEST_NAME: offloads
> > + TEST_NAME: userspace
> > +
> > + prepare_script: |
> > + apt-get update
> > + apt-get --yes install linux-headers-$(uname -r) ${DEPENDENCIES}
> > +
> > + configure_script: |
> > + ./boot.sh
> > + ./configure \
> > + --prefix=/usr \
> > + --localstatedir=/var \
> > + --sysconfdir=/etc \
> > + --with-dpdk=${DPDK}
> > +
> > + build_script:
> > + - make debian-source
> > +
> > + check_script: |
> > + if [ "${DPDK}" = "no" ]; then
> > + deb_dpdk_build_opt="nodpdk"
> > + else
> > + deb_dpdk_build_opt=""
> > + fi
> > + autopkgtest \
> > + --env DEB_BUILD_OPTIONS="nocheck parallel=8 ${deb_dpdk_build_opt}"
> > \
> > + --test-name $TEST_NAME \
> > + *.dsc \
> > + -- null
>
> There seems to be some duplication here. Maybe we can just define
> a matrix with 6 sets of variables? Each set with an image family
> and a test name? Maybe also CPU number. The only test that needs
> 4 cores is DPDK. And we have 16 cores available in parallel. We
> can run 5 jobs with 2 cores each and 1 job with 4 cores. This way
> we'll be able to run all the jobs in parallel saving a noticeable
> amount of time.
>
> What do you think?
Yes, I did not like the duplication either, but after staring at the
Cirrus CI docs for a while, I did not really find a good way to do it,
and I also wanted feedback on the idea before spending more time on
it.
> We may also try to use the templating functionality to avoid
> duplication at least:
>
> https://cirrus-ci.org/guide/tips-and-tricks/#sharing-configuration-between-tasks
Ah, thank you for this pointer, I did not find that the last time I
looked at it.
I'll come up with an improved version and send a new proposal, thank
you for your feedback, much appreciated!
--
Frode Nordahl
> Best regards, Ilya Maximets.
_______________________________________________
dev mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
