Luis,
On 1 July 2015 at 08:21, Luis Araujo <luis.ara...@collabora.co.uk> wrote: > On 06/30/2015 09:39 PM, Milosz Wasilewski wrote: >> >> On 30 June 2015 at 10:05, Milosz Wasilewski >> <milosz.wasilew...@linaro.org> wrote: >>> >>> On 25 June 2015 at 16:32, Luis Araujo <luis.ara...@collabora.co.uk> >>> wrote: >>>>> >>>>> I checked and it doesn't work (doesn't detect missing results). Here >>>>> is example job: >>>>> https://validation.linaro.org/scheduler/job/382325 (I'm not sure it's >>>>> publicly available) >>>> >>>> >>>> I cannot access it (even logged in with my launchpad account). >>>> >>>> Can you send me a link publicly available with this same problem?, I >>>> really >>>> would like to check this out. >>>> >>> I'll try to reproduce it with some other device. >>> >> Luis, >> >> Not exactly the same case, but I'm able to show that the missing >> results are not detected. Here is the job: >> https://validation.linaro.org/scheduler/job/410147 >> >> Here is my output: >> Generating lqa report for job(s): 410147 >> >> Report for job(s) (Tue Jun 30 14:33:11 2015): >> 410147 >> 1 test job(s) ran: 1 complete (0 fully successful, 1 wih failures), 0 >> incomplete >> >> * --- Failed Jobs --- * >> >> (F) Jobs with failed tests: >> >> 410147: >> https://ci.linaro.org/jenkins/job/linux-linaro-stable-lsk-v3.18/hwpack=beaglebone,label=linaro-kernel-ci/13/ >> >> ==================================================================================================================== >> 319 passed, 17 failed, 27 skipped, 0 unknown >> FAILED | rcutorture:rcutorture-start >> FAILED | rcutorture:lava-test-shell-run >> FAILED | pwrmgmt:cputopology_01 >> FAILED | usb-test-basic:list-all-usb-devices >> FAILED | usb-test-basic:examine-all-usb-devices >> FAILED | usb-test-basic:print-supported-protocols >> FAILED | usb-test-basic:print-supported-speeds >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> FAILED | lava:test_kernel_exception_1 >> Job: https://validation.linaro.org/scheduler/job/410147 >> Bundle: >> https://validation.linaro.org/dashboard/permalink/bundle/f022ec4a6ce822e154ff600d731025ad978a961b/ >> >> In this case the LTP entries are there in the bundle. However one is >> missing the parameters. Anyway, I'll try to reproduce the original >> problem in some other test (maybe an artificial one, so you). In case >> of the original problem the entry for one LTP shell is missing from >> the bundle. >> >> milosz > > > Right, the method won't report missing results for this job since all the > testdef's > specified in the yaml are available in the bundle: > > https://validation.linaro.org/scheduler/job/410147/definition > https://validation.linaro.org/dashboard/streams/anonymous/mwasilew/bundles/f022ec4a6ce822e154ff600d731025ad978a961b/?search=&length=25#table > > The _find_missing_tests method is actually used only to check for those > testdef from > the lava_test_shell command that for some reason were not properly reported > in the > bundle stream. Now, if these tests have missing results, certainly, the > method won't check > for those, but this could be a later improvement for sure. > As I wrote this isn't exactly the same issue as I reported previously. If you take one of LTP result entries out from the bundle you will get what I wrote about before. So the situation is you ask for 2 LTP test shells with different parameters and result bundle only contains one result for LTP. The other one is entirely missing. If you want to do the missing detection a bit better, take a look here: https://git.linaro.org/qa/qa-reports.git/blob/refs/heads/refactoring:/utils/testminer.py#l172 Eventually it should be fixed on LAVA side, but for the moment this is what we get. milosz _______________________________________________ linaro-validation mailing list linaro-validation@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-validation
