Another one that might be worth taking a peek at is: https://github.com/ansible/pytest-mp
I've never dug in personally but it always looked promising. On Thu, Oct 24, 2019 at 4:22 PM RonnyPfannschmidt <opensou...@ronnypfannschmidt.de> wrote: > > Hi Freddy, > > > unfortunately pytest-concurrent is fundamentally broken for managing fixtures > and other details, > as things are my suggestion is to avoid it. > > > -- Ronny > > Am 24.10.19 um 15:22 schrieb Rietdijk: > > Hi, > > I think having a simple `pytest-multiprocessing` or `pytest-concurrent` would > be extremely useful and cover a lot of people's use cases and would strongly > recommend to have that as a separate plugin from something bigger that does > feature hooks and/or remote execution.. It seems though as such a package > already exists, https://github.com/reverbc/pytest-concurrent, which offers > --concmode and --concworkers instead. > > Frederik > > On Thu, Oct 24, 2019 at 2:46 PM Bruno Oliveira <nicodde...@gmail.com> wrote: >> >> Hi everyone, >> >> For some time now execnet has been in maintenance only mode, and even so >> very few people are willing to maintain it; lately just myself and I’m not a >> good choice given that I don’t know the codebase at all, plus I have tons on >> my plate already. This poses a problem because often we have bug reports or >> feature requests in pytest-xdist which would require fixing or improving >> execnet, and are currently left without solution. >> >> Ronny and I have on occasion discussed the possibility of changing execnet‘s >> backend , with the current contenders being multiprocessing for local >> distribution and mitogen for remote distribution. >> >> I would like to write down some thoughts and gather opinions from the >> mailing list members. >> >> multiprocessing >> >> Aside from not needing to maintain execnet anymore, being able to use >> multiprocessing would bring several benefits: >> >> pytest -s would work just fine, because multiprocessing does not use >> stdout/stderr for communication. This is a often requested feature but which >> we can’t (with current expertise) implement on execnet. >> >> It would automatically support anything that can be picked to be transmitted >> across the wire. Currently execnet does not support custom user types and >> some builtins (frozen sets for example). >> >> Without execnet we would lose the ability of running the tests in different >> interpreter versions, but I believe this has been supplanted by tox in the >> ecosystem. >> >> mitogen >> >> Using mitogen for remote execution would provide automatic bootstrap of the >> Python environment, which is not currently supported by xdist‘s ssh remote >> execution, greatly limiting its usefulness in real-time scenarios. >> >> This is Ronny’s idea and he can add more here if wanted. >> >> pytest-xdist2? >> >> All the changes mentioned above would break backward compatibility hard, >> because details of the execnet implementation are currently exposed to users >> in the command-line: >> >> pytest -d --tx popen//python >> >> And in several hooks: >> >> def pytest_xdist_newgateway(gateway): >> """ called on new raw gateway creation. """ >> >> (gateway is an execnet object) >> >> To incorporate the new backends, we would need to severely break both >> command-line and existing hooks. Given the level of breakage this could >> cause, just bumping the major version in this case doesn’t seem enough, it >> would be a disservice to users given that probably nobody pins pytest-xdist >> to <=2, and it would break the world with hard to understand error messages >> once a first major release was released. >> >> Because of this, we have been discussing creating a new package, >> pytest-xdist2 (other suggestions are welcome), without any backward >> compatibility guarantees with pytest-xdist. >> >> pytest-xdist2 would, at first, only support local test distribution using >> multiprocessing (-n flag), no new hooks, and no remote execution. I’ve made >> a proof of concept here: >> >> https://github.com/pytest-dev/pytest-xdist/pull/479 >> >> So it definitely seems possible. One immediate benefit is that the proof of >> concept above supports pytest -s already, and can transfer anything that’s >> pickled over the wire. >> >> So I would like to know what people think of the above ideas, if they are >> good/bad, and/or suggest alternatives that we are not seeing right now. >> >> Ronny, please feel free to add to this email and correct anything I got >> wrong. >> >> Cheers, >> Bruno >> >> _______________________________________________ >> pytest-dev mailing list >> pytest-dev@python.org >> https://mail.python.org/mailman/listinfo/pytest-dev > > > _______________________________________________ > pytest-dev mailing list > pytest-dev@python.org > https://mail.python.org/mailman/listinfo/pytest-dev > > _______________________________________________ > pytest-dev mailing list > pytest-dev@python.org > https://mail.python.org/mailman/listinfo/pytest-dev _______________________________________________ pytest-dev mailing list pytest-dev@python.org https://mail.python.org/mailman/listinfo/pytest-dev
