On Wed, Oct 17, 2018 at 04:09:51PM -0300, Eduardo Habkost wrote:
> On Wed, Oct 17, 2018 at 07:40:51PM +0100, Peter Maydell wrote:
> > On 17 October 2018 at 18:38, Cleber Rosa <cr...@redhat.com> wrote:
> > >
> > >
> > > On 10/17/18 12:29 PM, Eduardo Habkost wrote:
> > >> On Wed, Oct 17, 2018 at 01:34:41PM +0100, Peter Maydell wrote:
> > >>> So, why does the test code need to care? It's not clear
> > >>> from the patch... My expectation would be that you'd
> > >>> just test all the testable target architectures,
> > >>> regardless of what the host architecture is.
> > >>
> > >> I tend to agree. Maybe the right solution is to get rid of the
> > >> os.uname(). I think the default should be testing all QEMU
> > >> binaries that were built, and the host architecture shouldn't
> > >> matter.
> >
> > Yes, looking at os.uname() also seems like an odd thing
> > for the tests to be doing here. The test framework
> > should be as far as possible host-architecture agnostic.
> > (For some of the KVM cases there probably is a need to
> > care, but those are exceptions, not the rule.)
> >
> > > I'm in favor of exercising all built targets, but that seems to me to be
> > > on another layer, above the test themselves. This change is about the
> > > behavior of a test when not told about the target arch (and thus binary)
> > > it should use.
> >
> > At that level, I think the right answer is "tell the user
> > they need to specify the qemu executable they are trying to test".
> > In particular, there is no guarantee that the user has actually
> > built the executable for the target that corresponds to the
> > host, so it doesn't work to try to default to that anyway.
>
> Agreed. However, I don't see when exactly this message would be
> triggered. Cleber, on which use cases do you expect
> pick_default_qemu_bin() to be called?
>
> In an ideal world, any testing runner/tool should be able to
> automatically test all binaries by default. Can Avocado help us
> do that? (If not, we could just do it inside a
> ./tests/acceptance/run script).
>
> --
> Eduardo
>
Why don't we add a variants file to QEMU's tree listing all known possible paths
for QEMU binary that can result from a build? For example:
$ cat tests/acceptance/variants
qemu_bin:
- ppc-softmmu/qemu-system-ppc
- ppc64-softmmu/qemu-system-ppc64
- x86_64-softmmu/qemu-system-x86_64
Then avocado could multiplex these variants file and call ./tests/acceptance/run
for each value of qemu_bin. `run` script could skip and return if $qemu_bin
doesn't exist. This approach also allows user forcing a value of qemu_bin when
calling `run` manually, for example:
./tests/acceptance/run --qemu_bin=/path/to/your/qemu-system-blah ...
This ./tests/acceptance/run can serve as an entry point to run all the tests.
If more parameters are considered mandatory in the future, the logic can be
placed there.
--
Murilo
