On Wednesday 2015-11-04 16:48 +0100, Michael Henretty wrote: > On Wed, Nov 4, 2015 at 4:45 PM, Fabrice Desré <[email protected]> wrote: > > Can we *right now* identify the worst offenders by looking at the tests > > results/re-runs? You know that sheriffs will very quickly hide and > > ignore tests that are really flaky. > > Yes, that's an important point. The problem is that you have to actually > look at the logs of an individual chunk to see which tests failed. If a > certain Gij test passes at least 1 out of it's 5 given runs, it will not > surface to Treeherder, which means we can't start it. Looking through each > chunk log file (of which we have 40 per run) is doable, but more time > consuming and error prone.
Can you write a script to gather the data? Seems like you should be able to get to it through various JSON files exposed for treeherder. (The code in https://hg.mozilla.org/users/dbaron_mozilla.com/buildbot-json-tools/ might not work anymore (although it might), but might provide some useful hints on how to find the right logs. Treeherder source should provide better hints, but is larger.) -David -- 𝄞 L. David Baron http://dbaron.org/ 𝄂 𝄢 Mozilla https://www.mozilla.org/ 𝄂 Before I built a wall I'd ask to know What I was walling in or walling out, And to whom I was like to give offense. - Robert Frost, Mending Wall (1914)
signature.asc
Description: Digital signature
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
