Hi Naveen,
On 2022-02-23 07:22, Pgowda wrote:
The patch implements Oe-selftest framework for Rust test.
I think you need a bit more of an introduction to
the patch. I assume that you and Vinay based this
in part on the gcc.py test suite so you might say so and
explain what is different and why.
Additionally, people will not be familiar with the rust test suite so
you should try to summarize that in a few sentences if possible.
I have some suggestion below but in addition to that maybe you can
include a link in this commit log to, and outline how this would differ
from:
https://rustc-dev-guide.rust-lang.org/tests/intro.html
without discussing the implementation.
Some of the functions are as follows:-
Rather than just listing the functions, say:
To support the rust tests in oe-core, the following
functions were added:
setup_cargo_environment(): Build bootstrap and some early stage tools.
do_rust_setup_snapshot(): Install the snapshot version of rust binaries.
do_configure(): To generate config.toml
do_compile(): To build "remote-test-server" for qemutarget image.
The python file builds remote-test-server and executes rust testing
remotely using background ssh. It adds the necessary test environment
and variables to run the rust oe-selftest.
Print the results in case of failure of runCmd().
The patch has been run and tested for X86, X86_64, Arm & Arm64 targets
on Ubuntu-18 successfully.
Most of the tests excluded in rust.py are due to the following error:-
These tests will be included in Oe-selftest after fixing the issue.
===============================================================
"build-st/tmp/work/core2-64-poky-linux/rust-testsuite/1.58.1-r0/
rustc-1.58.1-src/build/bootstrap/debug/rustc -vV` (exit status: 101)
--- stderr
thread 'main' panicked at '
kind: NotFound, message: "No such file or directory" }',
src/bootstrap/bin/rustc.rs:175:37
===============================================================
In this commit log, add a summary of APPROXIMATELY how many tests are
run when building
rust outside of a bitbake environment and compare that to:
1. tests run before skipping the broken ones
2. tests run with the patch below that skips higher level modules that
are broken.
i.e.
on ubuntu, there are 12000 tests and 11,555 pass
before the disable patch, 9000 tests ran and 7777 passed
with the patch, 5500 test ran and they all passed.
I just want the reader (myself) to have an understanding of
how much we have disabled. If you have exact numbers, that's great
but I'm NOT asking you to spend hours or days gathering that data.
Signed-off-by: pgowda <[email protected]>
Signed-off-by: Vinay Kumar <[email protected]>
---
meta/lib/oeqa/selftest/cases/rust.py | 57 +
meta/recipes-devtools/rust/rust-testsuite.inc | 163 +++
.../rust-testsuite/rust-oe-selftest.patch | 995 ++++++++++++++++++
.../rust/rust-testsuite_1.58.1.bb | 3 +
4 files changed, 1218 insertions(+)
create mode 100644 meta/lib/oeqa/selftest/cases/rust.py
create mode 100644 meta/recipes-devtools/rust/rust-testsuite.inc
create mode 100644
meta/recipes-devtools/rust/rust-testsuite/rust-oe-selftest.patch
create mode 100644 meta/recipes-devtools/rust/rust-testsuite_1.58.1.bb
diff --git a/meta/lib/oeqa/selftest/cases/rust.py
b/meta/lib/oeqa/selftest/cases/rust.py
new file mode 100644
index 0000000000..1379b09af1
--- /dev/null
+++ b/meta/lib/oeqa/selftest/cases/rust.py
@@ -0,0 +1,57 @@
+# SPDX-License-Identifier: MIT
+import os
+import subprocess
+from oeqa.core.decorator import OETestTag
+from oeqa.core.case import OEPTestResultTestCase
+from oeqa.selftest.case import OESelftestTestCase
+from oeqa.utils.commands import runCmd, bitbake, get_bb_var, get_bb_vars,
runqemu, Command
+from oeqa.utils.sshcontrol import SSHControl
+
+# Total time taken for testing is of about 2hr 20min, with PARALLEL_MAKE set
to 40 number of jobs.
+class RustSelfTestBase(OESelftestTestCase, OEPTestResultTestCase):
+
+ def run_check_emulated(self, *args, **kwargs):
+ # build remote-test-server before image build
+ recipe = "rust-testsuite"
+ bitbake("{} -c compile".format(recipe))
+ builddir = get_bb_var("B", "rust-testsuite")
+ # build core-image-minimal with required packages
+ default_installed_packages = ["libgcc", "libstdc++", "libatomic",
"libgomp"]
+ features = []
+ features.append('IMAGE_FEATURES += "ssh-server-openssh"')
+ features.append('CORE_IMAGE_EXTRA_INSTALL += "{0}"'.format("
".join(default_installed_packages)))
+ self.write_config("\n".join(features))
+ bitbake("core-image-minimal")
+ # wrap the execution with a qemu instance
+ with runqemu("core-image-minimal", runqemuparams = "nographic",
qemuparams = "-m 512") as qemu:
meta/lib/oeqa/selftest/cases/gcc.py has a quick test:
# validate that SSH is working
Given that you are about to run a long test, that seems worthwhile
even if your next step is to copy over the test server.
+ # Copy remote-test-server to image through scp
+ ssh = SSHControl(ip=qemu.ip, logfile=qemu.sshlog,
user="root")
+ ssh.copy_to(builddir + "/" +
"build/x86_64-unknown-linux-gnu/stage1-tools-bin/remote-test-server","~/")
+ # Execute remote-test-server on image through
background ssh
+ command = '~/remote-test-server -v remote'
+ sshrun=subprocess.Popen(("ssh", '-o',
'UserKnownHostsFile=/dev/null', '-o', 'StrictHostKeyChecking=no', '-f', "root@%s" %
qemu.ip, command),
+ shell=False,
+ stdout=subprocess.PIPE,
+ stderr=subprocess.PIPE)
+ # Get the values of variables.
+ targetsys = get_bb_var("TARGET_SYS", "rust-testsuite")
+ rustlibpath = get_bb_var("STAGING_LIBDIR_NATIVE",
"rust-testsuite")
+ tmpdir = get_bb_var("TMPDIR", "rust-testsuite")
+
+ # Exclude the test folders that error out while building
+ # Need to fix these errors and include them for testing
+ testargs = "--exclude src/test/rustdoc-json --exclude
src/test/run-make-fulldeps --exclude src/tools/tidy --exclude src/tools/rustfmt --exclude
src/tools/rustdoc-themes --exclude src/tools/rust-demangler --exclude src/tools/rls
--exclude src/tools/miri --exclude src/tools/linkchecker --exclude
src/tools/expand-yaml-anchors --exclude src/tools/compiletest --exclude src/tools/clippy
--exclude src/tools/cargotest --exclude src/tools/cargo --exclude src/test/rustdoc-gui
--exclude src/rustdoc-json-types --exclude src/tools/rustdoc --exclude src/librustdoc
--exclude src/doc/unstable-book --exclude src/doc/rustdoc --exclude src/doc/rustc
--exclude compiler/rustc --exclude library/panic_abort --exclude library/panic_unwind
--no-doc --no-fail-fast --bless"
Can you add a comment in the commit log listing why these are excluded?
It seems that they are extra features that we haven't worked on
supporting yet so if that's right, just say so.
+
+ # Set path for target-poky-linux-gcc, RUST_TARGET_PATH
and hosttools.
+ cmd = " export PATH=%s/../bin:$PATH;" % rustlibpath
+ cmd = cmd + " export
PATH=%s/../bin/%s:%s/hosttools:$PATH;" % (rustlibpath, targetsys, tmpdir)
+ cmd = cmd + " export RUST_TARGET_PATH=%s/rustlib;" %
rustlibpath
+ # Trigger testing.
+ cmd = cmd + " export TEST_DEVICE_ADDR=\"%s:12345\";" %
qemu.ip
+ cmd = cmd + " cd %s; python3 src/bootstrap/bootstrap.py
test %s --target %s ;" % (builddir, testargs, targetsys)
+ result = runCmd(cmd)
+
+@OETestTag("toolchain-system")
+class RustSelfTestSystemEmulated(RustSelfTestBase):
+ def test_rust(self):
+ self.run_check_emulated("rust")
I haven't actually worked with it but in:
meta/lib/oeqa/selftest/cases/gcc.py
the tests can be run using qemu-system or qemu-user mode.
It seems that either mode is possible for the gcc tests.
Nathan, Richard,
Should we support both modes and which is the default
if you know offhand.
Are these gcc tests actually run in parallel in practice?
We should look into what parallelism is possible in these
rust tests but first we need to make it work even in this degraded
(skipped tests) mode.
No comments on the rest of the patch so I've removed that.
I assume that you have moved the patch from one release to another
so just out of curiosity, is that minutes, an hour, many hours or days
of work
to fix up when we next have to update the recipe?
Good work, hopefully it's almost ready to be merged.
Since all I've asked from you is changes to the commit log,
I hope you can provide that quickly as v4 on this thread, and
then people CCed here can provide additional comments.
--
# Randy MacLeod
# Wind River Linux
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#162288):
https://lists.openembedded.org/g/openembedded-core/message/162288
Mute This Topic: https://lists.openembedded.org/mt/89339083/21656
Group Owner: [email protected]
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub
[[email protected]]
-=-=-=-=-=-=-=-=-=-=-=-
