Hi, first of all: I'm really really happy to see yet another person run our automated test suite! :)))
Kill Your TV wrote (18 Aug 2014 19:15:20 GMT) : > * Even with 9GB assigned to a "level0-guest", the memory erasure > test fails to complete due to failures allocating memory. How much > (available) memory is (generally) required to be able to run the full > test suite successfully? I give 10240000 KiB to my tester guest VM. > I probably need more RAM in order to run everything. We can help, and you might want to talk to me privately about it. In the meantime, adding swap to that guest VM should work, although it'll of course slow things down. > * Which tests, if any, are expected to fail or even pass inconsistently? > (I already know about #7060) See also #6092, which sometimes breaks the USB installer tests badly (especially since many tests in there depend on previous ones to have passed successfully). Otherwise, I've nothing else in mind, but if you report specific failures, then maybe I'll remember :] > * Are there any tests which are known to only run (or only run well) on > 'bare metal'? Many tests haven't been run in a nested virt. setup, so if you don't have fast nested virt., then possibly some timeouts will be reached. We can work this around by bumping the timeouts. > * Generally speaking, how long does running the full suite take? (on > good or average hardware, or 'nested vm' or 'bare metal'...whatever > can be answered is fine) I've never clocked it precisely, but I think it takes about 2-3 hours in a fast nested virt. environment (Haswell i7-4600U, SSD). IIRC a little bit more on a slower box (ThinkPad X201, i5-560M), but running on bare metal. At least 2-3 times slower in nested virt. on that X201. I'd be delighted to see other results, especially: * on AMD where supposedly nested virt. might be faster * on desktop CPUs (instead of low-voltage mobile ones) * on normal-voltage mobile CPUs * with more RAM allocated, so that the temporary directory can have a tmpfs mounted onto it, which should make the I/O-heavy tests (USB installer, memory erasure) — which take most of the full run time here — substantially faster * with more vCPUs allocated to the Tails system under test (wanna give #6729 a try? :) Rationale: these results would 1. help us decide what hardware we want to run our tests automatically post-commit (#7631); 2. help us decide what hardware we should put into the hands of developers working on the test suite, or who need/want to run it often (e.g. to do TDD). It would be good if, in both usecases, we could run the current test suite in less than one hour... especially since we have plans to add many more automated tests. Cheers, -- intrigeri _______________________________________________ Tails-dev mailing list [email protected] https://mailman.boum.org/listinfo/tails-dev To unsubscribe from this list, send an empty email to [email protected].
