On 05/25/2018 02:08 AM, Fam Zheng wrote:
> On Thu, 05/24 20:58, Cleber Rosa wrote:
>> TL;DR
>> =====
>>
>> Another version, with a minimalist approach, to the acceptance tests
>> infrastructure for QEMU, based on the Avocado Testing Framework.
>>
>> Background
>> ==========
>>
>> The previous version, still considered an RFC, was sent to the list by
>> Eduardo[1] was based on the work held in Amador's branch[2]. After
>> reviewing in under a different light, including the experiences
>> done and reported by Philippe[3].
>>
>> Differences from previous versions
>> ==================================
>>
>> The main difference is that this series include only the minimal
>> changes deemed necessary to have a starting point. I like to think
>> that it's better connected to the QEMU community and project needs,
>> and will hopefully allow for a more organic growth.
>>
>> Since this version has less features than the previous versions,
>> provided it's accepted, these are the next probable development tasks:
>>
>> * Provide a simple variants mechanism to allow the same tests to be
>> run under different targets, machine models and devices (present on
>> the previous versions as a "YAML to Mux" file with architecture
>> definitions)
>> * Implement QEMUMachine migration support (present on the previous
>> version in the "avocado_qemu.test._VM" class)
>> * Implement Guest OS image selection and download (mostly an Avocado
>> feature, paired with a parameter convention and cloud-init support
>> code)
>> * Implement interactive support for Guest OS sessions (present on
>> the previous versions, supported by the aexpect Python module)
>>
>> Even though this version shares very little (if any) code with the
>> previous versions, the following is a list of noteworthy changes:
>>
>> * Tests directory is now "tests/acceptance" (was "tests/avocado")
>> * Base test class is now "avocado_qemu.Test" (was
>> "avocado_qemu.test.QemuTest")
>> * Base test class is now hosted in "avocado_qemu/__init__.py" (was
>> "avocado_qemu/test.py")
>> * Direct use of "qemu.QEMUMachine", that is, the
>> avocado_qemu.test._VM class is gone
>> * avocado_qemu.Test won't search for QEMU binaries on $PATH. To use
>> QEMU binary on a custom system location it's necessary to use the
>> "qemu_bin" parameter
>> * Example test in README.rst is distributed as a real test
>> ("test_version.py")
>> * A new "Linux boot console" test, loosely modeled after Phillipe's
>> use case
>
> I like the direction this series is leading us to. Making it easy to write
> 'correct' and 'effective' tests is the most important thing IMO, and the few
> tests you add here look promising.
>
The soft targets that these adjectives ("easy", "correct" and
"effective") bring, are kind of hard to get right, but I'm really glad
you like what you see.
> I'm also curious to see a test case involving a guest OS image. (I'm also
> open to
> a re-implementation of tests/vm/basevm.py with Avocado, if you find it
> worthwhile.)
>
We had such tests in the previous incarnations of this work[1]. But by
doing it in smaller steps, I believe we can achieve a better result,
specially when it comes to the exposed API.
- Cleber.
[1] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03466.html
> Fam
>
>>
>> Commit summary
>> ==============
>>
>> Cleber Rosa (5):
>> Add functional/acceptance tests infrastructure
>> scripts/qemu.py: allow adding to the list of extra arguments
>> Acceptance tests: add quick VNC tests
>> scripts/qemu.py: introduce set_console() method
>> Acceptance tests: add Linux kernel boot and console checking test
>>
>> scripts/qemu.py | 103 +++++++++++++++-
>> scripts/test_qemu.py | 176
>> +++++++++++++++++++++++++++
>> tests/acceptance/README.rst | 141 +++++++++++++++++++++
>> tests/acceptance/avocado_qemu/__init__.py | 45 +++++++
>> tests/acceptance/test_boot_linux_console.py | 37 ++++++
>> tests/acceptance/test_version.py | 13 ++
>> tests/acceptance/test_vnc.py | 50 ++++++++
>> 7 files changed, 564 insertions(+), 1 deletion(-)
>> create mode 100644 scripts/test_qemu.py
>> create mode 100644 tests/acceptance/README.rst
>> create mode 100644 tests/acceptance/avocado_qemu/__init__.py
>> create mode 100644 tests/acceptance/test_boot_linux_console.py
>> create mode 100644 tests/acceptance/test_version.py
>> create mode 100644 tests/acceptance/test_vnc.py
>>
>> ---
>>
>> [1] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03443.html
>> [2] https://github.com/apahim/qemu/commits/avocado_qemu
>> [3] https://lists.gnu.org/archive/html/qemu-devel/2018-04/msg03076.html
>>
>
