On 10/06/2015 08:20 AM, Etienne Segonzac wrote:
>
> On Tue, Oct 6, 2015 at 4:48 PM, Fabrice Desré <[email protected]
> <mailto:[email protected]>> wrote:
>
> Raptor automatically reports performance regressions. If one is due to
> gecko (like when someone broke nuwa recently) it needs to be treated the
> same way we would do with gaia. I see absolutely no difference there.
>
> Well this is clearly not happening.
> You just asked Russ to bisect gecko to pinpoint a potential performance
> regression found by manually testing the video app.
> What exactly was done automatically in this scenario?
Nothing it seems, and there can be several causes:
- raptor was not ready yet when the regression happened.
- it's not a single but a bunch a small regressions that individually
were buried in the noise but overall ended up being large. Unfortunately
that will make bisecting hard, so we may have to live with it.
> Note that we don't run Talos performance tests on try either - you get
> emails after the fact if your commit is in the range of a regression.
> Maybe raptor should also send this kind of emails?
>
> As always there's no disagreement about what we need. But there's a big
> disconnect with what we have.
Not that much. We only need automatic emailing from raptor. Eli, how
hard does that look?
Fabrice (happy to not be in the Paris office, I feel my head would
probably be on a fork right now).
--
Fabrice Desré
b2g team
Mozilla Corporation
_______________________________________________
dev-fxos mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-fxos
