Hi Tom, On 1 June 2018 at 09:36, Tom Rini <[email protected]> wrote: > On Fri, Jun 01, 2018 at 08:27:03AM -0600, Simon Glass wrote: > >> Hi Tom, >> >> On 26 May 2018 at 11:06, Tom Rini <[email protected]> wrote: >> > The most portable way to get access to coverage is to invoke it as >> > 'python-coverage'. >> > >> > Cc: Simon Glass <[email protected]> >> > Signed-off-by: Tom Rini <[email protected]> >> > --- >> > So, here's the set of problems I have. First, running 'pip install >> > coverage' doesn't get me a 'coverage' binary, so I've just never made >> > good use of 'make tests' and so never have these run. It also means >> > they aren't strictly in travis-ci, so they're never run automatically. >> > With this patch, make tests executes for me, and now fails noting that a >> > lot of things don't have coverage. I'm honestly not sure what the best >> > path forward here is. >> > --- >> > tools/binman/README | 3 +-- >> > tools/binman/binman.py | 4 ++-- >> > 2 files changed, 3 insertions(+), 4 deletions(-) >> > >> >> Reviewed-by: Simon Glass <[email protected]> >> >> For me this patch works fine. The -T option works correctly. >> >> Typing 'python-coverage html' produces this report: >> >> https://pste.eu/p/NI7q.html >> >> Note that your patch needs to update this line too: >> >> print "Type 'coverage html' to get a report in htmlcov/index.html" >> >> Can you send me the output you see? > > So, I need to amend this a bit. On my Docker-based ubuntu 16.04 setup, > I see as non-fatal (and not coverage related, but binman test): > <unittest.result.TestResult run=79 errors=0 failures=0> > <unittest.result.TestResult run=11 errors=0 failures=4> > Traceback (most recent call last): > File "/home/trini/work/u-boot/u-boot/tools/patman/test.py", line 218, in > testGood > self.assertEqual(result.ok, True) > AssertionError: False != True > > Traceback (most recent call last): > File "/home/trini/work/u-boot/u-boot/tools/patman/test.py", line 252, in > testIndent > self.assertEqual(len(result.problems), 1) > AssertionError: 2 != 1 > > Traceback (most recent call last): > File "/home/trini/work/u-boot/u-boot/tools/patman/test.py", line 230, in > testNoSignoff > self.assertEqual(len(result.problems), 1) > AssertionError: 2 != 1 > > Traceback (most recent call last): > File "/home/trini/work/u-boot/u-boot/tools/patman/test.py", line 241, in > testSpaces > self.assertEqual(len(result.problems), 3) > AssertionError: 4 != 3 >
This is new to the SPDX change in checkpatch. I'll send a pull request with a fix for this soon. > Checking: https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/7.3.0/ > <unittest.result.TestResult run=35 errors=0 failures=1> > Traceback (most recent call last): > File "/home/trini/work/u-boot/u-boot/tools/buildman/test.py", line 426, in > testToolchainDownload > self.toolchains.LocateArchUrl('arm')) > AssertionError: > 'https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/4.9.0/x86_64-gcc-4.9.0-nolibc_arm-unknown-linux-gnueabi.tar.xz' > != > 'https://www.kernel.org/pub/tools/crosstool/files/bin/x86_64/7.3.0/x86_64-gcc-7.3.0-nolibc_arm-linux-gnueabi.tar.xz' This is due to the version number changes. It needs a fix to the test. > > <unittest.result.TestResult run=12 errors=0 failures=0> > <unittest.result.TestResult run=79 errors=0 failures=0> > > And then the coverage part works fine. With the changes in this RFC > patch so that my desktop itself could run coverage stuff I also see: > Missing tests for u_boot_with_ucode_ptr.py, u_boot_spl_bss_pad.py, entry.py, > u_boot_spl.py, u_boot_spl_with_ucode_ptr.py, u_boot_spl_dtb.py, > u_boot_img.py, intel_vbt.py, u_boot_dtb.py, intel_vga.py, > intel_descriptor.py, u_boot_dtb_with_ucode.py, intel_cmc.py, x86_start16.py, > u_boot_nodtb.py, u_boot.py, intel_fsp.py, blob.py, intel_mrc.py, > u_boot_spl_nodtb.py, u_boot_ucode.py, x86_start16_spl.py, intel_me.py > ValueError: Test coverage failure > Tests FAILED > Makefile:1710: recipe for target 'tests' failed > make: *** [tests] Error 1 This looks like it is not getting any coverage info at all. Perhaps there is a complete test failure? Can you try: binman -t and see what tests fail? > > So I guess there's something wrong with trying to use coverage this way, > rather than via pip. Maybe, but it works OK for me. Given that it works fine on travis (right?) I think we should apply your patch, with the additional change I mentioned in my previous email. Regards, Simon _______________________________________________ U-Boot mailing list [email protected] https://lists.denx.de/listinfo/u-boot
