On 08/27/2018 08:57 PM, Randy MacLeod wrote:
On 08/27/2018 06:17 PM, Yang Wang wrote:
Not sure if it's worth to run the Ptest on QEMU though, I also run
Ptest on SIMICS simulators, thousands of tests didn't get run, looks
like the result was not good as well.
Now my nightly Ptest runs on x86 device and gets consistent result
every day:
2018-08-27T06:26 -- 2018-08-27T09:52
Passed: 40518
Failed: 289
Skipped: 1876
Consistent results are good and > 90% pass rate is very good.
What are the stats using qemux86-64 and/or simics?
I don't expect that qemu results would be as close to real hardware
as Simics but it is quite good and freely available.
Actually, Ptest has 37 test suites as far as I know, different test
suites spent different time on QEMU and hardware, here is a list of
Ptest suites and their case number and spent time for running:
# Suite Name Suite Location Case # Time to
Run Case # Time to Run
qemu-x86-64
intel-xeon-core2
1 acl /lib64/acl/ptest/run-ptest 2 1m
380 1m
2 attr /lib64/attr/ptest/run-ptest 143 1m
143 1m
3 bash /lib64/bash/ptest/run-ptest 79 8m
79 4m
4 bluez5 /lib64/bluez5/ptest/run-ptest 7 6m
7 6m
5 bzip2 /lib64/bzip2/ptest/run-ptest 6 1m
6 1m
6 dbus-test /lib64/dbus-test/ptest/run-ptest 15 3m
15 1m
7 diffutils /lib64/diffutils/ptest/run-ptest 20 1m
20 2m
8 e2fsprogs /lib64/e2fsprogs/ptest/run-ptest 147 9m
335 10m
9 ethtool /lib64/ethtool/ptest/run-ptest 2 1m
2 1m
10 flex /lib64/flex/ptest/run-ptest 114 3m
114 1m
11 gawk /lib64/gawk/ptest/run-ptest 300 3m
298 2m
12 gdbm /lib64/gdbm/ptest/run-ptest 30 2m
25 2m
13 glib-2.0 /lib64/glib-2.0/ptest/run-ptest 62 14m
220 6m
14 gzip /lib64/gzip/ptest/run-ptest 51 4m
18 1m
15 kbd /lib64/kbd/ptest/run-ptest 15 1m
7 1m
16 libevent /lib64/libevent/ptest/run-ptest 22 6m
1 3m
17 libpcre /lib64/libpcre/ptest/run-ptest 34 3m
3 1m
18 libxml2 /lib64/libxml2/ptest/run-ptest 1 1m
0 1m
19 lzo /lib64/lzo/ptest/run-ptest 75 8m
5 3m
20 mdadm /lib64/mdadm/ptest/run-ptest 6m
6m
21 nettle /lib64/nettle/ptest/run-ptest 90 3m
90 3m
22 numactl /lib64/numactl/ptest/run-ptest 8 3m
23 openssh /lib64/openssh/ptest/run-ptest 13 27m
47 52m
24 openssl /lib64/openssl/ptest/run-ptest 87 47m
56 15m
25 parted /lib64/parted/ptest/run-ptest 64 5m
38 10m
26 perl /lib64/perl/ptest/run-ptest 101 20m
2440 47m
27 perl5 /lib64/perl5/ptest/run-ptest 110 17m
2406 29m
28 python /lib64/python/ptest/run-ptest 10961 1h
5m 32323 20m
29 rsyslog /lib64/rsyslog/ptest/run-ptest 2200 3h
37m 25 8m
30 sed /lib64/sed/ptest/run-ptest 147 1m
86 3m
31 slang /lib64/slang/ptest/run-ptest 96 1m
32 strace /lib64/strace/ptest/run-ptest 1557 1h
13m 431 6m
33 systemd /lib64/systemd/ptest/run-ptest 305 9m
155 3m
34 tcl /lib64/tcl/ptest/run-ptest 869
53m 206 6m
35 tcpdump /lib64/tcpdump/ptest/run-ptest 451 7m
413 3m
36 util-linux /lib64/util-linux/ptest/run-ptest 516 25m
514 13m
37 zlib /lib64/zlib/ptest/run-ptest 2 1m
2 1m
Overall 18080 10h
30m 40415 4h 15m
As you can see, running subset of them on QEMU could be a solution if
people do not want to spend too much time on it or simulator is the
preferred test device.
Thanks,
-Yang
Thanks,
Yang Wang
--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core
