Re: [PATCH v7 3/8] Acceptance tests: use avocado tags for machine type

2019-11-12 Thread Philippe Mathieu-Daudé 

On 11/12/19 2:59 AM, Cleber Rosa wrote:

On Fri, Nov 08, 2019 at 02:20:45PM +0100, Philippe Mathieu-Daudé wrote:

On 11/4/19 4:13 PM, Cleber Rosa wrote:

   """
-self.vm.set_machine('none')
   self.vm.add_args('-S')
   self.vm.launch()
diff --git a/tests/acceptance/linux_initrd.py b/tests/acceptance/linux_initrd.py
index c61d9826a4..3a0ff7b098 100644
--- a/tests/acceptance/linux_initrd.py
+++ b/tests/acceptance/linux_initrd.py
@@ -20,6 +20,7 @@ class LinuxInitrd(Test):
   Checks QEMU evaluates correctly the initrd file passed as -initrd option.
   :avocado: tags=arch:x86_64
+:avocado: tags=machine:pc


For some tests we can run on multiple machines (here q35), I was tempted to
use multiple tags. How could I do that now?



I missed this comment: you can add many tag values here to *classify*
the test as being "q35 machine type capable".

But, Avocado will only run a test multiple times with a varianter
plugin active.  In that case, a "machine" *parameter* with different
values will be passed to the tests.  This tag value is being used
as a default value for the parameter, so it has a lower precedence.

We have a pending task[1] to create an initial CIT file for arch and
machine types.

CC'ing Jan Richter, who is supposed to start working on it soon.

- Cleber.

[1] - 
https://trello.com/c/1wvzcxHY/105-create-cit-parameter-for-acceptance-tests


Good news, thanks for the trello link.

Re: [PATCH v7 3/8] Acceptance tests: use avocado tags for machine type

2019-11-12 Thread Philippe Mathieu-Daudé 

On 11/11/19 10:49 PM, Cleber Rosa wrote:

On Fri, Nov 08, 2019 at 02:20:45PM +0100, Philippe Mathieu-Daudé wrote:

@@ -310,7 +302,7 @@ class BootLinuxConsole(Test):
   def test_arm_emcraft_sf2(self):
   """
   :avocado: tags=arch:arm
-:avocado: tags=machine:emcraft_sf2
+:avocado: tags=machine:emcraft-sf2


Maybe add a comment about this change, "Since avocado 72(?) we can ... so
use ..."



You mean on this specific test docstring?  I'm confused if there's a


No! Just in the commit description :)


special reason for doing it here, of if you're suggesting adding a
similar command to all tag entries which make use of the extended
character set (which I think would be too verbose, repetitve, and hard
to maintain).


diff --git a/tests/acceptance/cpu_queries.py b/tests/acceptance/cpu_queries.py
index af47d2795a..293dccb89a 100644
--- a/tests/acceptance/cpu_queries.py
+++ b/tests/acceptance/cpu_queries.py
@@ -20,8 +20,8 @@ class QueryCPUModelExpansion(Test):
   def test(self):
   """
   :avocado: tags=arch:x86_64
+:avocado: tags=machine:none


Not to confuse with None :)



Yep! :)

- Cleber.

Re: [PATCH v7 3/8] Acceptance tests: use avocado tags for machine type

2019-11-11 Thread Cleber Rosa 
On Fri, Nov 08, 2019 at 02:20:45PM +0100, Philippe Mathieu-Daudé wrote:
> On 11/4/19 4:13 PM, Cleber Rosa wrote:
> >   """
> > -self.vm.set_machine('none')
> >   self.vm.add_args('-S')
> >   self.vm.launch()
> > diff --git a/tests/acceptance/linux_initrd.py 
> > b/tests/acceptance/linux_initrd.py
> > index c61d9826a4..3a0ff7b098 100644
> > --- a/tests/acceptance/linux_initrd.py
> > +++ b/tests/acceptance/linux_initrd.py
> > @@ -20,6 +20,7 @@ class LinuxInitrd(Test):
> >   Checks QEMU evaluates correctly the initrd file passed as -initrd 
> > option.
> >   :avocado: tags=arch:x86_64
> > +:avocado: tags=machine:pc
> 
> For some tests we can run on multiple machines (here q35), I was tempted to
> use multiple tags. How could I do that now?
>

I missed this comment: you can add many tag values here to *classify*
the test as being "q35 machine type capable".

But, Avocado will only run a test multiple times with a varianter
plugin active.  In that case, a "machine" *parameter* with different
values will be passed to the tests.  This tag value is being used
as a default value for the parameter, so it has a lower precedence.

We have a pending task[1] to create an initial CIT file for arch and
machine types.

CC'ing Jan Richter, who is supposed to start working on it soon.

- Cleber.

[1] - 
https://trello.com/c/1wvzcxHY/105-create-cit-parameter-for-acceptance-tests

Re: [PATCH v7 3/8] Acceptance tests: use avocado tags for machine type

2019-11-11 Thread Cleber Rosa 
On Fri, Nov 08, 2019 at 02:20:45PM +0100, Philippe Mathieu-Daudé wrote:
> > @@ -310,7 +302,7 @@ class BootLinuxConsole(Test):
> >   def test_arm_emcraft_sf2(self):
> >   """
> >   :avocado: tags=arch:arm
> > -:avocado: tags=machine:emcraft_sf2
> > +:avocado: tags=machine:emcraft-sf2
> 
> Maybe add a comment about this change, "Since avocado 72(?) we can ... so
> use ..."
> 

You mean on this specific test docstring?  I'm confused if there's a
special reason for doing it here, of if you're suggesting adding a
similar command to all tag entries which make use of the extended
character set (which I think would be too verbose, repetitve, and hard
to maintain).

> > diff --git a/tests/acceptance/cpu_queries.py 
> > b/tests/acceptance/cpu_queries.py
> > index af47d2795a..293dccb89a 100644
> > --- a/tests/acceptance/cpu_queries.py
> > +++ b/tests/acceptance/cpu_queries.py
> > @@ -20,8 +20,8 @@ class QueryCPUModelExpansion(Test):
> >   def test(self):
> >   """
> >   :avocado: tags=arch:x86_64
> > +:avocado: tags=machine:none
> 
> Not to confuse with None :)
> 

Yep! :)

- Cleber.

Re: [PATCH v7 3/8] Acceptance tests: use avocado tags for machine type

2019-11-08 Thread Philippe Mathieu-Daudé 

On 11/4/19 4:13 PM, Cleber Rosa wrote:

The same way the arch tag is being used as a fallback for the arch
parameter, let's do the same for QEMU's machine and avoid some boiler
plate code.

Signed-off-by: Cleber Rosa 
---
  docs/devel/testing.rst | 18 
  tests/acceptance/avocado_qemu/__init__.py  |  5 ++
  tests/acceptance/boot_linux_console.py | 19 +---
  tests/acceptance/cpu_queries.py|  2 +-
  tests/acceptance/linux_initrd.py   |  2 +-
  tests/acceptance/linux_ssh_mips_malta.py   |  5 --
  tests/acceptance/machine_m68k_nextcube.py  | 21 ++---
  tests/acceptance/machine_sparc_leon3.py|  3 +-
  tests/acceptance/ppc_prep_40p.py   |  3 --
  tests/acceptance/x86_cpu_model_versions.py | 53 --
  10 files changed, 72 insertions(+), 59 deletions(-)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 8e981e062d..27f286858a 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -746,6 +746,17 @@ name.  If one is not given explicitly, it will either be 
set to
  ``None``, or, if the test is tagged with one (and only one)
  ``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
  
+machine

+~~~
+
+The machine type that will be set to all QEMUMachine instances created
+by the test.
+
+The ``machine`` attribute will be set to the test parameter of the same
+name.  If one is not given explicitly, it will either be set to
+``None``, or, if the test is tagged with one (and only one)
+``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
+
  qemu_bin
  
  
@@ -781,6 +792,13 @@ architecture of a kernel or disk image to boot a VM with.

  This parameter has a direct relation with the ``arch`` attribute.  If
  not given, it will default to None.
  
+machine

+~~~
+
+The machine type that will be set to all QEMUMachine instances created
+by the test.
+
+
  qemu_bin
  
  
diff --git a/tests/acceptance/avocado_qemu/__init__.py b/tests/acceptance/avocado_qemu/__init__.py

index e676d9c4e7..6618ea67c1 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -115,6 +115,9 @@ class Test(avocado.Test):
  self.arch = self.params.get('arch',
  default=self._get_unique_tag_val('arch'))
  
+self.machine = self.params.get('machine',

+   
default=self._get_unique_tag_val('machine'))
+
  default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
  self.qemu_bin = self.params.get('qemu_bin',
  default=default_qemu_bin)
@@ -136,6 +139,8 @@ class Test(avocado.Test):
  name = str(uuid.uuid4())
  if self._vms.get(name) is None:
  self._vms[name] = self._new_vm(*args)
+if self.machine is not None:
+self._vms[name].set_machine(self.machine)
  return self._vms[name]
  
  def tearDown(self):

diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 7e41cebd47..6732cc59ca 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -62,7 +62,6 @@ class BootLinuxConsole(Test):
  kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
  kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
  
-self.vm.set_machine('pc')

  self.vm.set_console()
  kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 
'console=ttyS0'
  self.vm.add_args('-kernel', kernel_path,
@@ -85,7 +84,6 @@ class BootLinuxConsole(Test):
  kernel_path = self.extract_from_deb(deb_path,
  
'/boot/vmlinux-2.6.32-5-4kc-malta')
  
-self.vm.set_machine('malta')

  self.vm.set_console()
  kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 
'console=ttyS0'
  self.vm.add_args('-kernel', kernel_path,
@@ -118,7 +116,6 @@ class BootLinuxConsole(Test):
  kernel_path = self.extract_from_deb(deb_path,
  
'/boot/vmlinux-2.6.32-5-5kc-malta')
  
-self.vm.set_machine('malta')

  self.vm.set_console()
  kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 
'console=ttyS0'
  self.vm.add_args('-kernel', kernel_path,
@@ -148,7 +145,6 @@ class BootLinuxConsole(Test):
  initrd_path = self.workdir + "rootfs.cpio"
  archive.gzip_uncompress(initrd_path_gz, initrd_path)
  
-self.vm.set_machine('malta')

  self.vm.set_console()
  kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
 + 'console=ttyS0 console=tty '
@@ -188,7 +184,6 @@ class BootLinuxConsole(Test):
  initrd_path = self.workdir + "rootfs.cpio"
  archive.gzip_uncompress(initrd_path_gz, initrd_path)
  
-

[PATCH v7 3/8] Acceptance tests: use avocado tags for machine type

2019-11-04 Thread Cleber Rosa 
The same way the arch tag is being used as a fallback for the arch
parameter, let's do the same for QEMU's machine and avoid some boiler
plate code.

Signed-off-by: Cleber Rosa 
---
 docs/devel/testing.rst | 18 
 tests/acceptance/avocado_qemu/__init__.py  |  5 ++
 tests/acceptance/boot_linux_console.py | 19 +---
 tests/acceptance/cpu_queries.py|  2 +-
 tests/acceptance/linux_initrd.py   |  2 +-
 tests/acceptance/linux_ssh_mips_malta.py   |  5 --
 tests/acceptance/machine_m68k_nextcube.py  | 21 ++---
 tests/acceptance/machine_sparc_leon3.py|  3 +-
 tests/acceptance/ppc_prep_40p.py   |  3 --
 tests/acceptance/x86_cpu_model_versions.py | 53 --
 10 files changed, 72 insertions(+), 59 deletions(-)

diff --git a/docs/devel/testing.rst b/docs/devel/testing.rst
index 8e981e062d..27f286858a 100644
--- a/docs/devel/testing.rst
+++ b/docs/devel/testing.rst
@@ -746,6 +746,17 @@ name.  If one is not given explicitly, it will either be 
set to
 ``None``, or, if the test is tagged with one (and only one)
 ``:avocado: tags=arch:VALUE`` tag, it will be set to ``VALUE``.
 
+machine
+~~~
+
+The machine type that will be set to all QEMUMachine instances created
+by the test.
+
+The ``machine`` attribute will be set to the test parameter of the same
+name.  If one is not given explicitly, it will either be set to
+``None``, or, if the test is tagged with one (and only one)
+``:avocado: tags=machine:VALUE`` tag, it will be set to ``VALUE``.
+
 qemu_bin
 
 
@@ -781,6 +792,13 @@ architecture of a kernel or disk image to boot a VM with.
 This parameter has a direct relation with the ``arch`` attribute.  If
 not given, it will default to None.
 
+machine
+~~~
+
+The machine type that will be set to all QEMUMachine instances created
+by the test.
+
+
 qemu_bin
 
 
diff --git a/tests/acceptance/avocado_qemu/__init__.py 
b/tests/acceptance/avocado_qemu/__init__.py
index e676d9c4e7..6618ea67c1 100644
--- a/tests/acceptance/avocado_qemu/__init__.py
+++ b/tests/acceptance/avocado_qemu/__init__.py
@@ -115,6 +115,9 @@ class Test(avocado.Test):
 self.arch = self.params.get('arch',
 default=self._get_unique_tag_val('arch'))
 
+self.machine = self.params.get('machine',
+   
default=self._get_unique_tag_val('machine'))
+
 default_qemu_bin = pick_default_qemu_bin(arch=self.arch)
 self.qemu_bin = self.params.get('qemu_bin',
 default=default_qemu_bin)
@@ -136,6 +139,8 @@ class Test(avocado.Test):
 name = str(uuid.uuid4())
 if self._vms.get(name) is None:
 self._vms[name] = self._new_vm(*args)
+if self.machine is not None:
+self._vms[name].set_machine(self.machine)
 return self._vms[name]
 
 def tearDown(self):
diff --git a/tests/acceptance/boot_linux_console.py 
b/tests/acceptance/boot_linux_console.py
index 7e41cebd47..6732cc59ca 100644
--- a/tests/acceptance/boot_linux_console.py
+++ b/tests/acceptance/boot_linux_console.py
@@ -62,7 +62,6 @@ class BootLinuxConsole(Test):
 kernel_hash = '23bebd2680757891cf7adedb033532163a792495'
 kernel_path = self.fetch_asset(kernel_url, asset_hash=kernel_hash)
 
-self.vm.set_machine('pc')
 self.vm.set_console()
 kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
 self.vm.add_args('-kernel', kernel_path,
@@ -85,7 +84,6 @@ class BootLinuxConsole(Test):
 kernel_path = self.extract_from_deb(deb_path,
 '/boot/vmlinux-2.6.32-5-4kc-malta')
 
-self.vm.set_machine('malta')
 self.vm.set_console()
 kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
 self.vm.add_args('-kernel', kernel_path,
@@ -118,7 +116,6 @@ class BootLinuxConsole(Test):
 kernel_path = self.extract_from_deb(deb_path,
 '/boot/vmlinux-2.6.32-5-5kc-malta')
 
-self.vm.set_machine('malta')
 self.vm.set_console()
 kernel_command_line = self.KERNEL_COMMON_COMMAND_LINE + 'console=ttyS0'
 self.vm.add_args('-kernel', kernel_path,
@@ -148,7 +145,6 @@ class BootLinuxConsole(Test):
 initrd_path = self.workdir + "rootfs.cpio"
 archive.gzip_uncompress(initrd_path_gz, initrd_path)
 
-self.vm.set_machine('malta')
 self.vm.set_console()
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE
+ 'console=ttyS0 console=tty '
@@ -188,7 +184,6 @@ class BootLinuxConsole(Test):
 initrd_path = self.workdir + "rootfs.cpio"
 archive.gzip_uncompress(initrd_path_gz, initrd_path)
 
-self.vm.set_machine('malta')
 self.vm.set_console()
 kernel_command_line = (self.KERNEL_COMMON_COMMAND_LINE