Hi Brandon,
This sounds like a very useful! Thanks for looking into it!
On 27/02/2017 17:51, Potter, Brandon wrote:
https://github.com/linux-test-project/ltp/tree/master/testcases/kernel/syscalls
I was considering writing Linux test cases for SE Mode to work out (potential
but more likely probable) bugs in the implementation of some of the system
calls, but stumbled across a source base that already has tests for Linux. What
is the process for getting tests added to the regression system for the new git
repository? (If the information already exists, I apologize. Please point me to
the right documentation.)
There are basically two problems we need to solve for the new tests.
First, we need to distribute test/reference binaries. We don't have a
good flow for that. I think most of the current kernels used for testing
are distributed in a tar-ball from the main download page on gem5.org.
Git (and HG) really dislikes large binaries, so we'll need some other
solution. We could potentially use something like git-lfs where large
binaries are stored outside of the normal git repo.
Your best option for now is probably to create a tar ball with the
required SE-mode binaries that users can extract into ${M5_PATH}/test-progs.
Once binaries have a home, we just need to wire up to the test system.
This is currently done by creating a directory a directory per test case
in tests/quick/se/ and adding a test.py script in that directory. On top
of that, you need a tests/quick/se/NAME/ref/arch/os directory with
reference files (functional tests only contain a place holder file
called EMPTY). This is clearly not ideal for a large number of short
functional tests.
The short-term solution to to just create the configuration and push it
for review on gerrit. A nicer long-term solution would be to add a new
way to configure functional SE-mode tests. I imagine most of these tests
will just a have a simple pass/fail conditions and very simple command
lines that can be reused for all supported architectures. You'll most
likely need to add a new test type and discovery mechanism in
tests/testing/tests.py if you want to go down this route.
We only support a subset of the system calls that are tested for in LTP. So
ideally, we could pull the source, build it, and choose a subset of the tests
that we'd need. As other syscalls are implemented in gem5 in the future, we can
add the LTP tests into the regressions to test them without needing to worry
about always adding in our own.
I am a little concerned about getting cross-compiled versions of the binaries
so that we can test each ISA. The build machine would need a cross-compiler to
actually guarantee that the ISAs have coverage. (I am not sure that the
compilers even support ALPHA although maybe they still do.)
I'm tempted to say that you shouldn't worry about anything other than
x86. The respective arch maintainers should make sure to compile their
test binaries for their own architectures.
I believe that the license is GPLv2 for LTP, but we are not including it into
our code so I do not think that it matters; it would just be a regression
target for SE Mode. If this actually matters, someone should speak up.
Adding GPL code to gem5's main repo seems like a blocker to me. However,
as you already noted, that shouldn't be a problem since we don't need to
store the test binaries in the main repo.
Cheers,
Andreas
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended recipient,
please notify the sender immediately and do not disclose the contents to any
other person, use it for any purpose, or store or copy the information in any
medium. Thank you.
_______________________________________________
gem5-dev mailing list
[email protected]
http://m5sim.org/mailman/listinfo/gem5-dev
