On Mon, Jan 28, 2019 at 12:14 AM Sudhir Sharma <[email protected]> wrote:
>
> Added json file of converted manual runtime testcases(compliance test cases)
>
> Signed-off-by: Sudhir Sharma <[email protected]>
> ---
> meta/lib/oeqa/manual/compliance-test.json | 142
> ++++++++++++++++++++++++++++++
> 1 file changed, 142 insertions(+)
> create mode 100644 meta/lib/oeqa/manual/compliance-test.json
>
> diff --git a/meta/lib/oeqa/manual/compliance-test.json
> b/meta/lib/oeqa/manual/compliance-test.json
> new file mode 100644
> index 0000000..7adf8a1
> --- /dev/null
> +++ b/meta/lib/oeqa/manual/compliance-test.json
> @@ -0,0 +1,142 @@
> +[
> + {
> + "test": {
> + "@alias": "configuration.configuration.LTP_subset_test_suite",
> + "author": [
> + {
> + "email": "[email protected]",
> + "name": "[email protected]"
do we have to hard code email addresses ? maybe we can insert them via
environment.
> + }
> + ],
> + "execution": {
> + "1": {
> + "action": "For real hardware, run following component,
> \nsyscalls \nfs \nfsx \ndio \nio \nmm \nipc \nsched \nmath \nnptl \npty
> \nadmin_tools \ntimers \ncommands \n\nFor QEMU, run following component
> \nsyscalls \nmm \nipc \nsched \nmath \nnptl \npty \nadmin_tools \ncommands
> \n\nRun Instructions: \nLTP download:
> http://sourceforge.net/projects/ltp/files/LTP%20Source/ltp-20120401/ltp-full-20120401.bz2/download
> \n\n(link is outdated, always use the last version released or the one
> found in the image) \n\n\n\nbuild steps: refer to http://ltp.sourceforge.net
> \n\nRun steps:",
> + "expected_results": ""
> + },
> + "2": {
> + "action": "Build LTP with toolchain or in sdk image. Or
> use a sato-sdk image which has LTP already included in /opt/ltp",
> + "expected_results": ""
> + },
> + "3": {
> + "action": "For QEMU, create the qemu target with \"-m
> 512\", which makes some memory stress cases pass. For some issues, we could
> only set 128M for qemuarm and 256M for qemumips.",
> + "expected_results": ""
> + },
> + "4": {
> + "action": "Copy LTP folder into target, for example,
> /opt/ltp if you have built it yourself. Modify the default scenario file
> \"scenario_groups/default\", remove test suites not to be tested",
> + "expected_results": ""
> + },
> + "5": {
> + "action": "Comment runtests/sched: hackbench, which is
> not suitable to run in emulators. Reminder (comment it also for Sugarbay
> Devices).",
> + "expected_results": ""
> + },
> + "6": {
> + "action": "Comment oom01, oom02, oom03, oom04 in
> runtest/mm, which consume lots of memory",
> + "expected_results": ""
> + },
> + "7": {
> + "action": "From /opt/ltp run: ./runltp -p -l
> result-M2-20101218.log -C result-M2-20101218.fail -d /opt/ltp/tmp &>
> result-M2-20101218.fulllog \n\n",
> + "expected_results": "Check the result on wiki,
> https://wiki.yoctoproject.org/wiki/LTP_result, there should be no regression
> failure met."
> + }
> + },
> + "summary": "LTP_subset_test_suite"
> + }
> + },
> + {
> + "test": {
> + "@alias": "configuration.configuration.POSIX_subset_test_suite",
> + "author": [
> + {
> + "email": "[email protected]",
> + "name": "[email protected]"
> + }
> + ],
> + "execution": {
> + "1": {
> + "action": "In a sato-sdk image go to /opt/ltp or get
> latest LTP sourcecode, download location is
> http://sourceforge.net/projects/ltp/files/LTP%20Source/ and install it.",
> + "expected_results": ""
> + },
> + "2": {
> + "action": "Go into the folder of LTP, and
> posix_testsuite is under testcases/open_posix_testsuite/",
> + "expected_results": ""
> + },
> + "3": {
> + "action": "Run connmand: make generate-makefiles",
> + "expected_results": ""
> + },
> + "4": {
> + "action": "Run connmand: make conformance-all",
> + "expected_results": ""
> + },
> + "5": {
> + "action": "Run connmand: make conformance-test (this
> step may show errors, ignore them)",
> + "expected_results": ""
> + },
> + "6": {
> + "action": "Run connmand: make tools-all",
> + "expected_results": ""
> + },
> + "7": {
> + "action": "Run connmand: sh posix.sh > posix.log,
> posix.sh as below: \n \n#!/bin/sh \n./bin/run-posix-option-group-test.sh AIO
> \n./bin/run-posix-option-group-test.sh MEM
> \n./bin/run-posix-option-group-test.sh MSG
> \n./bin/run-posix-option-group-test.sh SEM
> \n./bin/run-posix-option-group-test.sh SIG
> \n./bin/run-posix-option-group-test.sh THR
> \n./bin/run-posix-option-group-test.sh TMR
> \n./bin/run-posix-option-group-test.sh TPS \n \n",
> + "expected_results": ""
> + },
> + "8": {
> + "action": "Check the posix.log after testing is
> finished",
> + "expected_results": "Compare the test result on wiki,
> https://wiki.yoctoproject.org/wiki/Posix_result, there should be no more
> regression failures met."
> + }
> + },
> + "summary": "POSIX_subset_test_suite"
> + }
> + },
> + {
> + "test": {
> + "@alias": "configuration.configuration.LSB_subset_test_suite",
> + "author": [
> + {
> + "email": "[email protected]",
> + "name": "[email protected]"
> + }
> + ],
> + "execution": {
> + "1": {
> + "action": "Get lsd-sdk image and install it on target
> device or start the image(if it is QEMU) with option \"-m 512M\"",
> + "expected_results": ""
> + },
> + "2": {
> + "action": "Comment in /opt/lsb-test/session any tests
> you don't want to run.",
> + "expected_results": ""
> + },
> + "3": {
> + "action": "Run /usr/bin/LSB_Test.sh which should
> download the LSB suite and set it up. Some packages may fail to download
> because their location changed on ftp.linuxfoundation.org. You need to
> manually update /opt/lsb-test/packages_list",
> + "expected_results": ""
> + },
> + "4": {
> + "action": "Tests should start automatically, you can use
> the web interface to reconfigure the setup. ",
> + "expected_results": "Check the result on wiki
> https://wiki.yoctoproject.org/wiki/LSB_Result No regression failures should
> be met."
> + }
> + },
> + "summary": "LSB_subset_test_suite"
> + }
> + },
> + {
> + "test": {
> + "@alias":
> "General_Runtime.General_Runtime.stress_test_-_Genericx86-64",
> + "author": [
> + {
> + "email": "[email protected]",
> + "name": "[email protected]"
> + }
> + ],
> + "execution": {
> + "1": {
> + "action": "Bootup with core-image-lsb-sdk image",
> + "expected_results": ""
> + },
> + "2": {
> + "action": "Execute the crashme test with below command
> \n\n./opt/ltp/runltp f crashme",
> + "expected_results": "The stress testing should not make
> the target crash. Check CPU usage and basic functionality of the system after
> the tests are over. "
> + }
> + },
> + "summary": "stress_test_-_Genericx86-64"
> + }
> + }
> +]
> \ No newline at end of file
> --
> 2.7.4
>
> --
> _______________________________________________
> Openembedded-core mailing list
> [email protected]
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
