On 11/23/2016 07:07 AM, Zubair Lutfullah Kakakhel wrote: > Hi, > > Thank you for your comprehensive reply! > > Comments inline. > > On 11/22/2016 02:11 PM, Cleber Rosa wrote: >> On 11/22/2016 07:53 AM, Zubair Lutfullah Kakakhel wrote: >>> Hi, >>> >> >> Hi Zubair, >> >>> There are quite a few threads about this and a trello card >>> https://trello.com/c/xNeR2slj/255-support-running-tests-in-parallel >>> >>> And the discussion leads to a complex multi-host RFC. >>> https://www.redhat.com/archives/avocado-devel/2016-March/msg00025.html >>> >>> Our requirement is simpler. >>> All we wanted to do is run disjoint simple (c executables) tests in >>> parallel. >>> >> >> Sounds fair enough. >> >>> I was wondering if somebody has a WIP branch that has some level of >>> implementation for this? >> >> I'm not familiar with a WiP or PoC on this (yet). If anyone has >> experimented with it, I'd happy to hear about it. >> >>> Or If somebody is familiar with the code base, I'd appreciate some >>> direction on how to implement this. >>> >> >> Avocado already runs every single test in a fresh new process. This is, >> at least theoretically, a good start. Also, the test process is >> handled based on the standard Python multiprocessing module: >> >> https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L363 >> >> >> The first experimentation I'd do would be to attempt using the also >> Python standard multiprocessing.Pool: >> >> https://docs.python.org/2.7/library/multiprocessing.html#using-a-pool-of-workers >> > > In this case, there would be a separate python thread for each test > being run in parallel. > Each python thread would actually call the test executable using a > sub-process? >
Ideally, the Avocado test runner would remain a single process, that is, without one additional thread (or process) to manage each *test* process. > That can be OK for Desktops but won't scale well for using avocado in > memory > constrained Embedded devices. > I must admit I haven't attempted to run Avocado in resource constrained environments. Can you explain what is your bigger concern? Do you feel that Avocado (as a single process test *runner*) plus one process for each *test* is not suitable to those environments? - Cleber. > Please correct me if I am reading this incorrectly. > > Regards, > ZubairLK > >> >> This would most certainly lead to changes in how Avocado currently >> serially waits for the test status: >> >> https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L403 >> >> >> Which ultimately is added to the (Job wide) results: >> >> https://github.com/avocado-framework/avocado/blob/master/avocado/core/runner.py#L455 >> >> >> Since the results for many tests will now be acquired in unpredictable >> order, this will require changes to the ResultEvent based plugins (such >> as the UI). >> >>> Thanks >>> >>> Regards, >>> ZubairLK >>> >> >> I hope this is a good initial set of pointers. If you feel adventurous >> and wants to start hacking on this, you're more then welcome. >> >> BTW: we've had quite a number of features that started as >> experiments/ideas/not-really-perfect-pull-requests from the community >> that Avocado "core team" members embraced and pushed all the way to >> completeness. >> -- Cleber Rosa [ Sr Software Engineer - Virtualization Team - Red Hat ] [ Avocado Test Framework - avocado-framework.github.io ]
signature.asc
Description: OpenPGP digital signature
