On Wed, 2022-09-07 at 20:27 +0530, [email protected] wrote:
> From: pgowda <[email protected]>
>
> The patch implements Rust testing framework similar to other selftest,
> specifically the gcc selftest in OE. It uses the client and server
> based method to test the binaries for cross-target on the image.
> The test framework is a wrapper around the Rust build system as ./x.py
> test. It tests many functionalities of Rust distribution like tools,
> documentation, libraries, packages, tools, Cargo, Crater etc.
> Please refer the following link for detailed description of Rust
> testing:-
> https://rustc-dev-guide.rust-lang.org/tests/intro.html#tool-tests
>
> To support the rust tests in oe-core, the following functions were
> added:-
> setup_cargo_environment(): Build bootstrap and some early stage tools.
> do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
> do_configure(): To generate config.toml
> do_compile(): To build "remote-test-server" for qemu target image.
>
> Approximate Number of Tests Run in the Rust Testsuite :- 18000
> Approximate Number of Tests that FAIL in bitbake environment :- 100-150
> Normally majority of the testcases are present in major folder "test/"
> It contributes to more than 80% of the testcases present in Rust test
> framework. These tests pass as expected on any Rust versions without
> much fuss. The tests that fail are of less important and contribute to
> less than 2% of the total testcases. These minor tests are observed to
> work on some versions and fail on others. They have to be added, ignored
> or excluded for different versions as per the behavior.
> These tests have been ignored or excluded in the Rust selftest
> environment to generate success of completing the testsuite.
>
> These tests work in parallel mode even in the skipped test mode as
> expected. Although the patch to disable tests is large, it is very simple
> in that it only disables tests. When updating to a newer version of Rust,
> the patch can usually be ported in a day.
>
> Signed-off-by: pgowda <[email protected]>
> Signed-off-by: Vinay Kumar <[email protected]>
> ---
> meta/lib/oeqa/selftest/cases/rust.py | 54 ++
> meta/recipes-devtools/rust/rust-source.inc | 3 +-
> meta/recipes-devtools/rust/rust.inc | 1 +
> .../rust/rust/rust-oe-selftest.patch | 508 ++++++++++++++++++
> meta/recipes-devtools/rust/rust_1.63.0.bb | 5 +
> 5 files changed, 570 insertions(+), 1 deletion(-)
> create mode 100644 meta/lib/oeqa/selftest/cases/rust.py
> create mode 100644 meta/recipes-devtools/rust/rust/rust-oe-selftest.patch
I ran this through some testing on the autobuilder. The first time was
ok, it only ran on qemux86 and qemux86-64 as it was only enabled for
toolchain-system but ran successfully.
On the second run I added @OETestTag("toolchain-user") as well which
meant it ran for arm/mips/powerpc. It looks like there was one worrying
intermittent failure:
qemux86-64 intermittent issue:
https://autobuilder.yoctoproject.org/typhoon/#/builders/73/builds/5788/steps/19/logs/stdio
and then some arch specific failures:
qemuppc failed in the test:
https://autobuilder.yoctoproject.org/typhoon/#/builders/63/builds/5790/steps/19/logs/stdio
qemumips64 failed in the test:
https://autobuilder.yoctoproject.org/typhoon/#/builders/74/builds/5786/steps/20/logs/stdio
qemumips failed in the test:
https://autobuilder.yoctoproject.org/typhoon/#/builders/60/builds/5804/steps/20/logs/stdio
qemuarm64, qemuarm and qemux86 passed on this run.
We need to understand and fix that qemux86-64 intermittent issue before
this can merge. I wasn't sure what we want to do with mips and ppc.
Cheers,
Richard
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#170453):
https://lists.openembedded.org/g/openembedded-core/message/170453
Mute This Topic: https://lists.openembedded.org/mt/93526674/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
