Hi Tobias, Thanks for your reply.
On 2025-09-28 11:54, Tobias Waldekranz wrote:
I'll leave some thoughts that went through my mind as I was doing the dm work. This is from the perspective of someone with almost no experience of neither pytest in general, nor of labgrid. It would be great if much of the logic from the workflow file could be moved to a regular Make target or something, so that it is easier to answer questions like "Will the CI job run with testfs for this config now?"
I agree that there should be a simple way to "make test" and that this should closely resemble what we run in CI. With all the skip and xfail logic we have, it can happen quite easily to have pytest succeed when run manually and then fail on CI. What I did to replicate the github CI was using "act" with -r (to keep containers running) and podman. Then after the run I opened a shell in the container to tinker with the CI failure. This approach is not something we can present in the docs to explain to contributors how to get the CI green. Agreed that we should move the build, kconfig, test invocation logic from the github workflow to a script. Maybe a Makefile, maybe just shell, and invoke that from the workflow yaml. Then we document how contributors can invoke that script in the barebox ci container to replicate what happens on github. I didn't get to do it this time unfortunately. Maybe next time I touch the pytests if the list here agrees on this direction. Regards, Jonas -- Pengutronix e.K. | Jonas Rebmann | Steuerwalder Str. 21 | http://www.pengutronix.de/ | 31137 Hildesheim, Germany | Phone: +49-5121-206917-0 | Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-9 |
