Does the speed improve if system Qemu is configured to use several cpu cores (I think this doesn’t happen by default in yocto)?
Alex > On 24 Jul 2019, at 18.23, Nathan Rossi <[email protected]> wrote: > >> On Thu, 25 Jul 2019 at 00:06, Mark Hatle <[email protected]> wrote: >> >>> On 7/24/19 8:55 AM, [email protected] wrote: >>>> On Wed, 2019-07-24 at 22:30 +1000, Nathan Rossi wrote: >>>> So I only hit one major issue with the oe-core qemu* machines I >>>> tested on when running under system emulation, specifically with >>>> aarch64/qemuarm64. >>>> >>>> One (or more) test in the "c" suite of gcc causes qemu to crash due >>>> to what appears to be an internal qemu issue. >>>> >>>> qemu outputted: >>>> qemu-4.0.0/tcg/tcg.c:3952: tcg_gen_code: Assertion >>>> `s->gen_insn_end_off[num_insns] == off' failed. >>>> >>>> The test that caused this was: >>>> gcc.target/aarch64/advsimd-intrinsics/vldX.c -O2 execution test >>> >>> Ok, interesting. I'd hope that issue could be resolved if it started >>> causing problems for us, or we could skip that test as a workaround I >>> guess. >> >> When we have done similar testing like this, we have skipped these tests. >> Much >> of the time they are in corner case instructions and not something that >> really >> gives relevant results. >> >> But with that said, we should check if QEMU is already aware of this failure >> and >> if not report it to them. > > Something I forgot to note was that this specific test runs fine under > qemu user. > >> >> (The GNU tests also test various instructions that may not be available on >> all >> processors, we've seen this many times on IA32 machines. They just assume >> all >> instructions are available, even if running on an older CPU. So this is also >> something to be aware of when interpreting the results. -- but QEMU shouldn't >> crash!) > > Yes this is something I noticed with -mhard-float/-msoft-float on arm. > >> >> --Mark >> >>>> Just an update here, managed to get the results for this. As you will >>>> see below, running some of these tests is very slow on qemu system >>>> emulation. Though kvm did give a decent boost in performance. >>>> >>>> Note: qemuarm64 (sys) is missing gcc results because one of the gcc >>>> tests crashes qemu. >>> >>> The results were wrapped and hard to read so I unwrapped them (added >>> here for others): > > Sorry about that. > >>> >>> | g++ | gcc | glibc | >>> libatomic | libgomp | libitm | libstdc++-v3 >>> qemuarm (usr) | 365/128416 | 469/123905 | 65/ 5130 | >>> 0/ 49 | 0/ 2515 | 0/ 46 | 23/12782 >>> qemuarm (sys) | 365/128416 | 468/123874 | 47/ 5130 | >>> 0/ 49 | 0/ 2515 | 18/ 46 | 48/12790 >>> qemux86-64 (usr) | 457/131913 | 589/135169 | 1423/ 5991 | >>> 0/ 54 | 0/ 2522 | 0/ 46 | 1/13008 >>> qemux86-64 (sys) | 418/131913 | 519/135221 | 1418/ 5991 | >>> 0/ 54 | 1/ 2522 | 18/ 46 | 51/13010 >>> qemux86-64 (sys+kvm) | 418/131913 | 519/135415 | 40/ 5991 | >>> 0/ 54 | 1/ 2522 | 18/ 46 | 46/13010 >>> qemuarm64 (usr) | 364/128977 | 459/130904 | 75/ 5882 | >>> 0/ 54 | 0/ 2515 | 0/ 46 | 1/12789 >>> qemuarm64 (sys) | 364/128977 | | 43/ 5882 | >>> 0/ 54 | 0/ 2515 | 18/ 46 | 62/12791 >>> qemuppc (usr) | 6747/128636 | 18336/116624 | 1220/ 5110 | >>> 0/ 49 | 2/ 2515 | 0/ 46 | 33/12996 >>> qemuppc (sys) | 383/129056 | 800/119824 | 1188/ 5110 | >>> 0/ 49 | 2/ 2515 | 18/ 46 | 34/12998 >>> qemuriscv64 (usr) | 376/128427 | 460/106399 | 86/ 5847 | >>> 0/ 54 | 4/ 2508 | | 1/12748 >>> qemuriscv64 (sys) | 376/128427 | 458/106451 | 53/ 5847 | >>> 0/ 54 | 0/ 2508 | | 52/12750 >>> >>> | g++ | gcc | glibc | >>> libatomic | libgomp | libitm | libstdc++-v3 >>> qemuarm (usr) | 9m 24s | 15m 3s | 37m 10s | >>> 8s | 6m 52s | 8s | 1h 24s >>> qemuarm (sys) | 3h 58m 30s | 12h 21m 44s | 5h 36m 53s | >>> 55s | 45m 57s | 53s | 12h 16m 11s >>> qemux86-64 (usr) | 8m 22s | 15m 48s | 36m 52s | >>> 8s | 6m 1s | 8s | 34m 42s >>> qemux86-64 (sys) | 5h 38m 27s | 15h 15m 40s | 5h 54m 42s | >>> 1m 8s | 45m 52s | 55s | 3h 26m 11s >>> qemux86-64 (sys+kvm) | 16m 22s | 56m 44s | 2h 29m 45s | >>> 25s | 16m 58s | 21s | 19m 20s >>> qemuarm64 (usr) | 8m 34s | 16m 15s | 44m 25s | >>> 8s | 6m 23s | 8s | 35m 38s >>> qemuarm64 (sys) | 4h 2m 53s | | 6h 2m 39s | >>> 1m 7s | 44m 47s | 53s | 3h 9m 37s >>> qemuppc (usr) | 6m 54s | 10m 47s | 32m 50s | >>> 6s | 6m 22s | 7s | 34m 25s >>> qemuppc (sys) | 5h 46m 23s | 16h 16m 10s | 4h 10m 6s | >>> 1m 16s | 1h 3m 11s | 1m 12s | 4h 32m 45s >>> qemuriscv64 (usr) | 6m 54s | 10m 23s | 36m 50s | >>> 7s | 9m 38s | | 33m 13s >>> qemuriscv64 (sys) | 2h 19m 24s | 6h 27m 37s | 4h 23m 43s | >>> 47s | 31m 47s | | 1h 52m 18s >>> >>> This makes very interesting reading, thanks! >>> >>> I'm quite amazed how much faster user mode qemu is at running the tests >>> against a system kvm qemu. The accuracy of sys, usr and sys+kvm looks >>> questionable in different places. > > The speed user mode qemu gets comes from multi-threaded execution. > Whilst kvm gets better single threaded execution performance over user > mode qemu, more threads of tests can run with user mode. I probably > should have mentioned all the above runs were on an 8 thread system. > So on a bigger build machine user mode qemu probably gets even better > scaling. > >>> >>> There isn't a clear answer here although its obvious qemuppc user mode >>> emulation is bad. The usermode testing is clearly the winner speed wise >>> by a long margin. I would like to understand why though as KVM should >>> be reasonable... > > Just given some inspection of the systems and host while running, it > appears that two of the major performances hits were the overheads in > SSH (e.g. encryption, etc) and the use of multiple threads trying to > run things on the single core target. > > I will look into the test failures with user mode and try to resolve > some of them. If its possible to have similar results between usr/sys > then it would help make the choice between them a lot easier. > Resolving some of the test failures should also help to bring the > results in line with expectation. > > Regards, > Nathan > >>> >>> Cheers, >>> >>> Richard >>> >>> >>> >>> >>> >> > -- > _______________________________________________ > Openembedded-core mailing list > [email protected] > http://lists.openembedded.org/mailman/listinfo/openembedded-core -- _______________________________________________ Openembedded-core mailing list [email protected] http://lists.openembedded.org/mailman/listinfo/openembedded-core
