On Sat, Jul 31, 2021 at 2:40 AM Thomas Huth <th...@redhat.com> wrote: > > On 31/07/2021 00.04, Cleber Rosa wrote: > > On Fri, Jul 30, 2021 at 11:43 AM Peter Maydell <peter.mayd...@linaro.org> > > wrote: > >> > >> On Fri, 30 Jul 2021 at 16:12, Peter Maydell <peter.mayd...@linaro.org> > >> wrote: > >>> > >>> "make check-acceptance" takes way way too long. I just did a run > >>> on an arm-and-aarch64-targets-only debug build and it took over > >>> half an hour, and this despite it skipping or cancelling 26 out > >>> of 58 tests! > >>> > >>> I think that ~10 minutes runtime is reasonable. 30 is not; > >>> ideally no individual test would take more than a minute or so. > >> > >> Side note, can check-acceptance run multiple tests in parallel? > > > > Yes, it can, but it's not currently enabled to do so, but I'm planning > > to. As a matter of fact, Yesterday I was trying out Avocado's > > parallel capable runner on a GitLab CI pipeline[1] and it went well. > > Was this one of the shared gitlab CI runners? ... well, those feature only a > single CPU, so the run was likely not very different compared to a single run. >
Yes, the two pipeline executions I referred to were run in the shared GitLab CI runners. I was testing two things: 1. Possible caveats/issues with the parallel Avocado runner (AKA "nrunner") and the Acceptance tests (first pipeline linked, with "max parallel tasks" set to 1) 2. Any possible gains/losses with running with "max parallel tasks" set to 2 (second pipeline linked) > > But the environment on GitLab CI is fluid, and I bet there's already > > some level of overcommit of (at least) CPUs there. The only pipeline > > I ran there with tests running in parallel, resulted in some jobs with > > improvements, and others with regressions in runtime. Additionally, > > lack of adequate resources can make more tests time out, and thus give > > out false negatives. > > It certainly does not make sense to enable parallel tests for the shared > runners there. > > Thomas > > There could be gains on scenario #2 if there's considerable I/O wait on some tests. That's why I mention that previous experiences mixing the acceptance tests with the iotests were very interesting. But you're right, with only acceptance tests, mostly CPU bound, there was no clear gain. Best, - Cleber.