On Wed, 2023-12-13 at 20:30 +0000, Eilís 'pidge' Ní Fhlannagáin wrote:
> This takes the work rburton did on image screenshot testing and
> expands it.
>
> Right now this only works for qemux86-64. Some standardization
> of screensize/resolution needs to happen with runqemu params for other
> machines. There is an issue in qemux86-64 (and possibly others) where
> the screenswitch icon is only half present. This causes the test to
> fail.
>
> This test takes a screendump of a qemu image (for now, just
> core-image-sato for qemux86-64), and compares it to an image we
> have on record. Some normalisation of the different qemu configs
> need to happen to be able to support all machines. Example, the
> qemuarm64 screen size is much larger than the qemux86-64.
>
> The image we have on record contains a blanked out clock. We do
> the same blanking out process for the screenshot, so the images should
> have zero differences. If they do, we fail.
>
> In order to enable this test, you will need meta-openembedded/meta-oe in
> your bblayers.conf and the following in local.conf:
>
> IMAGE_CLASSES += "testimage"
> TEST_SUITES = "login"
> IMAGE_INSTALL:append = " python3-qemu-qmp "
> TESTIMAGEDEPENDS:append:qemuall = " imagemagick-native:do_populate_sysroot "
>
> Signed-off-by: Eilís 'pidge' Ní Fhlannagáin <[email protected]>
> Co-authored-by: Ross Burton <[email protected]>
> Co-authored-by: Eilís 'pidge' Ní Fhlannagáin <[email protected]>
> ---
> .../core-image-sato-qemux86-64.png | Bin 0 -> 46986 bytes
> meta/lib/oeqa/runtime/cases/login.py | 34 ++++++++++++++++++
> 2 files changed, 34 insertions(+)
> create mode 100644 meta/files/image-tests/core-image-sato-qemux86-64.png
> create mode 100644 meta/lib/oeqa/runtime/cases/login.py
>
> diff --git a/meta/lib/oeqa/runtime/cases/login.py
> b/meta/lib/oeqa/runtime/cases/login.py
> new file mode 100644
> index 00000000000..9e351fdeb29
> --- /dev/null
> +++ b/meta/lib/oeqa/runtime/cases/login.py
> @@ -0,0 +1,34 @@
> +#
> +# Copyright OpenEmbedded Contributors
> +#
> +# SPDX-License-Identifier: MIT
> +#
> +
> +import subprocess
> +from oeqa.runtime.case import OERuntimeTestCase
> +import tempfile
> +from oeqa.runtime.decorator.package import OEHasPackage
> +
> +class LoginTest(OERuntimeTestCase):
> +
> + @OEHasPackage(['python3-qemu-qmp'])
This only works if we have python3-qemu-qmp in the target? I thought
the monitor was external?
> + def test_screenshot(self):
> + if self.td.get('MACHINE') != "qemux86-64":
> + self.fail
So we're going to throw failures on all other machines? Wouldn't a skip
be more appropriate? Can we provide images for other machines?
Have a look at things like:
@skipIfNotMachine("qemux86-64", "tests are qemux86-64 specific currently")
> +
> + if bb.utils.which(os.getenv('PATH'), "convert") is not None and
> bb.utils.which(os.getenv('PATH'), "compare") is not None:
This works for nativesdk-imagemagick but it would need convert+compare
in HOSTTOOLS to work for a host binary. I'm torn on the best way to
find that otherwise.
> + with tempfile.NamedTemporaryFile(prefix="oeqa-screenshot-login",
> suffix=".png") as t:
> + ret = self.target.runner.run_monitor("screendump",
> args={"filename": t.name, "format":"png"})
> + # Use the meta-oe version of convert, along with it's suffix
> + cmd = "convert.im7 {0} -fill white -draw 'rectangle 600,10
> 640,22' {1}".format(t.name, t.name)
> + proc = subprocess.Popen(cmd, shell=True,
> stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> + output, error = proc.communicate()
You probably want subprocess.check_result()
> +
> + # Use the meta-oe version of compare, along with it's suffix
> + cmd = "compare.im7 -metric MSE {0}
> {1}/meta/files/image-tests/core-image-sato-{2}.png /dev/null".format(t.name,
> self.td.get('COREBASE'), self.td.get('MACHINE'))
> + proc = subprocess.Popen(cmd, shell=True,
> stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> + output, error = proc.communicate()
and check_output()
> + diff=float(error.decode('utf-8').replace("(",
> "").replace(")","").split()[1])
> + self.assertEqual(0, diff, "Screenshot diff is %s." %
> (str(diff)))
> + else:
> + self.fail
Wouldn't:
self.skipTest("could not find convert tool from imagemagick")
be better?
Right now I can't merge this. With skip()s I can at least merge whilst
we sort the other pieces.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#192322):
https://lists.openembedded.org/g/openembedded-core/message/192322
Mute This Topic: https://lists.openembedded.org/mt/103157570/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
