On 13/04/2012 09:41, ext [email protected] wrote: >> Random question of the day: do you happen to have stats about how >> often those insignificant tests actually fail? That should help to >> figure out which ones are actually working, and therefore should >> not be marked as insignificant. > > I'm glad you asked. I spent some time yesterday and today doing that > analysis for the 40-odd insignificant tests that had no Jira task > associated. > > I used the data from the last eleven CI runs for each module (times > eight configs for which autotests are executed). That's not hugely > statistically significant, but it's not that easy to pull large > numbers of build logs out of the CI system manually through the web > interface. (Any CI folks who can tell me how to extract/search all > the stored logs?)
I have access to those files, so just let me know what you want. I can for example hand you the 20 last runs from the following branch/runs: mstormo@gunpowder:/var/www/testresults/ci$ ls | grep -v -e CodeCover -e buildsystem -e on_demand -e containers -e 4.8 -e RC1 -e api_changes -e qml2 -e build_performance | xargs du -sh 122M Qt3D_master_Integration 895M Qt5_master_Integration 11M QtActiveQt_master_Integration 2.4G QtBase_master_Integration 38M QtConnectivity_master_Integration 1017M QtDeclarative_master_Integration 15M QtDocGallery_master_Integration 89M QtDoc_master_Integration 44M QtFeedback_master_Integration 12M QtGraphicalEffects_master_Integration 5.5M QtImageFormats_master_Integration 13M QtJSBackend_master_Integration 29M QtJsonDb_hbtree_Integration 1.3M QtJsonDb_kvs_Integration 317M QtJsonDb_master_Integration 502M QtLocation_master_Integration 212M QtMultimedia_master_Integration 4.1M QtPhonon_master_Integration 268M QtPim_master_Integration 47M QtQA_master_Integration 28M QtQuick1_master_Integration 62M QtScript_master_Integration 107M QtSensors_master_Integration 8.1M QtSvg_master_Integration 160M QtSystems_master_Integration 81M QtTools_master_Integration 7.3M QtTranslations_master_Integration 612K QtWebKit_Examples_and_Demos_master_Integration 16M QtXmlPatterns_master_Integration Just tell me what you need. -- .marius _______________________________________________ Development mailing list [email protected] http://lists.qt-project.org/mailman/listinfo/development
