On Mon, Jun 22, 2020 at 12:23:35PM -0600, Simon Glass wrote: > Hi Heinrich, > > On Mon, 22 Jun 2020 at 10:40, Heinrich Schuchardt <xypron.g...@gmx.de> wrote: > > > > On 22.06.20 18:17, Simon Glass wrote: > > > Hi Heinrich, > > > > > > On Mon, 22 Jun 2020 at 10:07, Heinrich Schuchardt <xypron.g...@gmx.de> > > > wrote: > > >> > > >> Call pytest3 with argument -ra to display reason why Python tests are > > >> skipped. > > >> > > >> Signed-off-by: Heinrich Schuchardt <xypron.g...@gmx.de> > > >> --- > > >> .gitlab-ci.yml | 2 +- > > >> 1 file changed, 1 insertion(+), 1 deletion(-) > > >> > > >> diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml > > >> index f2e491c117..f53098ea5f 100644 > > >> --- a/.gitlab-ci.yml > > >> +++ b/.gitlab-ci.yml > > >> @@ -46,7 +46,7 @@ stages: > > >> # "${var:+"-k $var"}" expands to "" if $var is empty, "-k $var" if > > >> not > > >> - export PATH=/opt/qemu/bin:/tmp/uboot-test-hooks/bin:${PATH}; > > >> export PYTHONPATH=/tmp/uboot-test-hooks/py/travis-ci; > > >> - ./test/py/test.py --bd ${TEST_PY_BD} ${TEST_PY_ID} > > >> + ./test/py/test.py -ra --bd ${TEST_PY_BD} ${TEST_PY_ID} > > >> ${TEST_PY_TEST_SPEC:+"-k ${TEST_PY_TEST_SPEC}"} > > >> --build-dir "$UBOOT_TRAVIS_BUILD_DIR" > > > > > > Do you have a link showing the current output with this patch? > > > > Hello Simon, > > > > here is an example output: > > > > https://gitlab.denx.de/u-boot/custodians/u-boot-efi/-/jobs/112385 > > That's what I was afraid of. The skip output is more than the normal > output, and if we don't intend to fix it, I'd rather not have > unactionable warnings in the output. > > Having said that, we need to enable SPI flash, FPGA and MMC > environment tests by the look of it.
So, looking at the output. 40 lines of test output, 32 lines of explanation of why skips. I think it's still readable. It also shows where perhaps we could group / re-org things a little as it already does smart things like: SKIPPED [131] /builds/u-boot/custodians/u-boot-efi/test/py/conftest.py:468: board "qemu_arm64" not supported Looking at the spi flash tests (and also the mmc read/write tests) we should have a generic check in the tests for them be configured and bail if not, as that would reduce the skip summary lines and collect them all in one spot. > > Some of the skips are ok. But some we should really fix like: > > > > SKIPPED [1] > > /builds/u-boot/custodians/u-boot-efi/test/py/tests/test_efi_loader.py:100: > > No static network configuration is defined > > Also, I suspect that the warnings will be huge when running on real hardware. Well, that's beside the point with this patch. It's very specifically only doing it on GitLab (and if/when we add this, v2 should cover travis/Azure) as it's CI and logs are cheap but recreation can be annoying. But it's also not true at least here. I see a few more lines than the above example on my am335x_evm config, but I also have a few more skips. -- Tom
signature.asc
Description: PGP signature