Re: [Qemu-devel] [PATCH v3 07/20] Acceptance tests: look for target architecture in test tags first
On Fri, Mar 01, 2019 at 11:37:19AM +0100, Cornelia Huck wrote:
> On Wed, 20 Feb 2019 19:57:40 -0500
> Cleber Rosa wrote:
>
> > A test can, optionally, be tagged for one or many architectures. If a
> > test has been tagged for a single architecture, there's a high chance
> > that the test won't run on other architectures. This changes the
> > default order of choosing a default target architecture to use based
> > on the 'arch' tag value first.
> >
> > The precedence order is for choosing a QEMU binary to use for a test
> > is now:
> >
> > * qemu_bin parameter
> > * arch parameter
> > * arch tag value (for example, x86_64 if ":avocado: tags=arch:x86_64
> >is used)
> >
> > This means that if one runs:
> >
> > $ avocado run -p qemu_bin=/usr/bin/qemu-system-x86_64 test.py
> >
> > No arch parameter or tag will influence the selection of the QEMU
> > target binary. If one runs:
> >
> > $ avocado run -p arch=ppc64 test.py
> >
> > The target binary selection mechanism will attempt to find a binary
> > such as "ppc64-softmmu/qemu-system-ppc64". And finally, if one runs
> > a test that is tagged (in its docstring) with "arch:aarch64":
> >
> > $ avocado run aarch64.py
> >
> > The target binary selection mechanism will attempt to find a binary
> > such as "aarch64-softmmu/qemu-system-aarch64".
> >
> > At this time, no provision is made to cancel the execution of tests if
> > the arch parameter given (manually) does not match the test "arch"
> > tag, but it may be a useful default behavior to be added in the
> > future.
>
> Would it be useful as well to cancel a test if the passed-in qemu_bin
> parameter clashed with the tagged architecture?
>
It would... in fact, a much older version of this proposal did that.
I'm waiting for the most basic things to get in, before adding yet
another "special" behavior.
Thanks for the review and suggestion.
- Cleber.
> >
> > Signed-off-by: Cleber Rosa
> > ---
> > docs/devel/testing.rst| 4 +++-
> > tests/acceptance/avocado_qemu/__init__.py | 7 ++-
> > 2 files changed, 9 insertions(+), 2 deletions(-)
> >
> > diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> > index 6035db1b44..87bcf8ef43 100644
> > --- a/docs/devel/testing.rst
> > +++ b/docs/devel/testing.rst
> > @@ -702,7 +702,9 @@ A test may, for instance, use the same value when
> > selecting the
> > architecture of a kernel or disk image to boot a VM with.
> >
> > The ``arch`` attribute will be set to the test parameter of the same
> > -name, and if one is not given explicitly, it will be set to ``None``.
> > +name. If one is not given explicitly, it will either be set to
> > +``None``, or, if the test is tagged with one (and only one)
> > +``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
> >
> > qemu_bin
> >
> > diff --git a/tests/acceptance/avocado_qemu/__init__.py
> > b/tests/acceptance/avocado_qemu/__init__.py
> > index f580582602..9e98d113cb 100644
> > --- a/tests/acceptance/avocado_qemu/__init__.py
> > +++ b/tests/acceptance/avocado_qemu/__init__.py
> > @@ -53,7 +53,12 @@ def pick_default_qemu_bin(arch=None):
> > class Test(avocado.Test):
> > def setUp(self):
> > self.vm = None
> > -self.arch = self.params.get('arch')
> > +arches = self.tags.get('arch', [])
> > +if len(arches) == 1:
> > +arch = arches.pop()
> > +else:
> > +arch = None
> > +self.arch = self.params.get('arch', default=arch)
> > default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
> > self.qemu_bin = self.params.get('qemu_bin',
> > default=default_qemu_bin)
>
> Reviewed-by: Cornelia Huck
Re: [Qemu-devel] [PATCH v3 07/20] Acceptance tests: look for target architecture in test tags first
On Wed, 20 Feb 2019 19:57:40 -0500
Cleber Rosa wrote:
> A test can, optionally, be tagged for one or many architectures. If a
> test has been tagged for a single architecture, there's a high chance
> that the test won't run on other architectures. This changes the
> default order of choosing a default target architecture to use based
> on the 'arch' tag value first.
>
> The precedence order is for choosing a QEMU binary to use for a test
> is now:
>
> * qemu_bin parameter
> * arch parameter
> * arch tag value (for example, x86_64 if ":avocado: tags=arch:x86_64
>is used)
>
> This means that if one runs:
>
> $ avocado run -p qemu_bin=/usr/bin/qemu-system-x86_64 test.py
>
> No arch parameter or tag will influence the selection of the QEMU
> target binary. If one runs:
>
> $ avocado run -p arch=ppc64 test.py
>
> The target binary selection mechanism will attempt to find a binary
> such as "ppc64-softmmu/qemu-system-ppc64". And finally, if one runs
> a test that is tagged (in its docstring) with "arch:aarch64":
>
> $ avocado run aarch64.py
>
> The target binary selection mechanism will attempt to find a binary
> such as "aarch64-softmmu/qemu-system-aarch64".
>
> At this time, no provision is made to cancel the execution of tests if
> the arch parameter given (manually) does not match the test "arch"
> tag, but it may be a useful default behavior to be added in the
> future.
Would it be useful as well to cancel a test if the passed-in qemu_bin
parameter clashed with the tagged architecture?
>
> Signed-off-by: Cleber Rosa
> ---
> docs/devel/testing.rst| 4 +++-
> tests/acceptance/avocado_qemu/__init__.py | 7 ++-
> 2 files changed, 9 insertions(+), 2 deletions(-)
>
> diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
> index 6035db1b44..87bcf8ef43 100644
> --- a/docs/devel/testing.rst
> +++ b/docs/devel/testing.rst
> @@ -702,7 +702,9 @@ A test may, for instance, use the same value when
> selecting the
> architecture of a kernel or disk image to boot a VM with.
>
> The ``arch`` attribute will be set to the test parameter of the same
> -name, and if one is not given explicitly, it will be set to ``None``.
> +name. If one is not given explicitly, it will either be set to
> +``None``, or, if the test is tagged with one (and only one)
> +``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
>
> qemu_bin
>
> diff --git a/tests/acceptance/avocado_qemu/__init__.py
> b/tests/acceptance/avocado_qemu/__init__.py
> index f580582602..9e98d113cb 100644
> --- a/tests/acceptance/avocado_qemu/__init__.py
> +++ b/tests/acceptance/avocado_qemu/__init__.py
> @@ -53,7 +53,12 @@ def pick_default_qemu_bin(arch=None):
> class Test(avocado.Test):
> def setUp(self):
> self.vm = None
> -self.arch = self.params.get('arch')
> +arches = self.tags.get('arch', [])
> +if len(arches) == 1:
> +arch = arches.pop()
> +else:
> +arch = None
> +self.arch = self.params.get('arch', default=arch)
> default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
> self.qemu_bin = self.params.get('qemu_bin',
> default=default_qemu_bin)
Reviewed-by: Cornelia Huck
[Qemu-devel] [PATCH v3 07/20] Acceptance tests: look for target architecture in test tags first
A test can, optionally, be tagged for one or many architectures. If a
test has been tagged for a single architecture, there's a high chance
that the test won't run on other architectures. This changes the
default order of choosing a default target architecture to use based
on the 'arch' tag value first.
The precedence order is for choosing a QEMU binary to use for a test
is now:
* qemu_bin parameter
* arch parameter
* arch tag value (for example, x86_64 if ":avocado: tags=arch:x86_64
is used)
This means that if one runs:
$ avocado run -p qemu_bin=/usr/bin/qemu-system-x86_64 test.py
No arch parameter or tag will influence the selection of the QEMU
target binary. If one runs:
$ avocado run -p arch=ppc64 test.py
The target binary selection mechanism will attempt to find a binary
such as "ppc64-softmmu/qemu-system-ppc64". And finally, if one runs
a test that is tagged (in its docstring) with "arch:aarch64":
$ avocado run aarch64.py
The target binary selection mechanism will attempt to find a binary
such as "aarch64-softmmu/qemu-system-aarch64".
At this time, no provision is made to cancel the execution of tests if
the arch parameter given (manually) does not match the test "arch"
tag, but it may be a useful default behavior to be added in the
future.
Signed-off-by: Cleber Rosa
---
docs/devel/testing.rst| 4 +++-
tests/acceptance/avocado_qemu/__init__.py | 7 ++-
2 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 6035db1b44..87bcf8ef43 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -702,7 +702,9 @@ A test may, for instance, use the same value when selecting
the
architecture of a kernel or disk image to boot a VM with.
The ``arch`` attribute will be set to the test parameter of the same
-name, and if one is not given explicitly, it will be set to ``None``.
+name. If one is not given explicitly, it will either be set to
+``None``, or, if the test is tagged with one (and only one)
+``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
qemu_bin
diff --git a/tests/acceptance/avocado_qemu/__init__.py
b/tests/acceptance/avocado_qemu/__init__.py
index f580582602..9e98d113cb 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -53,7 +53,12 @@ def pick_default_qemu_bin(arch=None):
class Test(avocado.Test):
def setUp(self):
self.vm = None
-self.arch = self.params.get('arch')
+arches = self.tags.get('arch', [])
+if len(arches) == 1:
+arch = arches.pop()
+else:
+arch = None
+self.arch = self.params.get('arch', default=arch)
default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
self.qemu_bin = self.params.get('qemu_bin',
default=default_qemu_bin)
--
2.20.1
