On Thu, Sep 26, 2019 at 3:55 PM oVirt Jenkins <[email protected]> wrote:
> Change 103548,1 (vdsm) is probably the reason behind recent system test > failures in the "ovirt-4.3" change queue and needs to be fixed. > > This change had been removed from the testing queue. Artifacts build from > this > change will not be released until it is fixed. > This is a bug in the change queue, because For further details about the change see: > https://gerrit.ovirt.org/#/c/103548/1 this is a travis configuration change, and it cannot break system tests :-) Do we have stats of such failures in the past? For failed test results see: > http://jenkins.ovirt.org/job/ovirt-4.3_change-queue-tester/2188/ Maybe we can add a simple file based condition to tell if a patch need to run system tests? For example in vdsm we have this structure: lib/ static/ tests/ automation/ .travis.ymal ... Changes in tests/ and automation/ cannot break vdsm during runtime, so there is no need to run system tests for patches modifying these directories. Here are changes stats per directory since 4.2: $ git shortlog -sn --since v4.20.0 ./tests ./automation | awk '{sum+=$1} END {print sum}' 4579 $ git shortlog -sn --since v4.20.0 ./lib ./static | awk '{sum+=$1} END {print sum}' 4413 So 50% of the changes cannot break the system and do not require system tests. SLOC Directory SLOC-by-Language (Sorted) 65205 lib python=64977,sh=228 62415 tests python=62317,sh=98 3432 vdsm_hooks python=3432 803 contrib python=740,sh=63 468 automation sh=454,python=14 313 static python=210,sh=103 289 init sh=162,python=127 112 build-aux sh=72,python=40 64 top_dir sh=64 30 doc python=30 30 helpers python=30 1 vdsm_log sh=1 0 docker (none) 0 m4 (none) We can see that the tests are about 50% of the code. This is pretty good for legacy project, but if we look in a more modern project like ovirt-imageio: common: SLOC Directory SLOC-by-Language (Sorted) 3641 test python=3641 2365 ovirt_imageio_common python=2201,ansic=164 103 top_dir python=103 daemon: SLOC Directory SLOC-by-Language (Sorted) 1588 test python=1588 792 ovirt_imageio_daemon python=792 18 top_dir python=18 0 data (none) We see that test code ratio is close to 2/1. So adding a filter per project that can tell if a patch needs system tests very important, and can speed our workflow by fact of 2-3. Adding infra-support to file a bug, we need to work on this. Nir
_______________________________________________ Devel mailing list -- [email protected] To unsubscribe send an email to [email protected] Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/[email protected]/message/IMHBHVRPNCSFFBWWQBH6CDD5UOTLHOQ2/
