On 31/07/2021 00.04, Cleber Rosa wrote:
On Fri, Jul 30, 2021 at 11:43 AM Peter Maydell <peter.mayd...@linaro.org> wrote:
On Fri, 30 Jul 2021 at 16:12, Peter Maydell <peter.mayd...@linaro.org> wrote:
"make check-acceptance" takes way way too long. I just did a run
on an arm-and-aarch64-targets-only debug build and it took over
half an hour, and this despite it skipping or cancelling 26 out
of 58 tests!
I think that ~10 minutes runtime is reasonable. 30 is not;
ideally no individual test would take more than a minute or so.
Side note, can check-acceptance run multiple tests in parallel?
Yes, it can, but it's not currently enabled to do so, but I'm planning
to. As a matter of fact, Yesterday I was trying out Avocado's
parallel capable runner on a GitLab CI pipeline[1] and it went well.
Was this one of the shared gitlab CI runners? ... well, those feature only a
single CPU, so the run was likely not very different compared to a single run.
But the environment on GitLab CI is fluid, and I bet there's already
some level of overcommit of (at least) CPUs there. The only pipeline
I ran there with tests running in parallel, resulted in some jobs with
improvements, and others with regressions in runtime. Additionally,
lack of adequate resources can make more tests time out, and thus give
out false negatives.
It certainly does not make sense to enable parallel tests for the shared
runners there.
Thomas
