Raptor automatically reports performance regressions. If one is due to
gecko (like when someone broke nuwa recently) it needs to be treated the
same way we would do with gaia. I see absolutely no difference there.
Note that we don't run Talos performance tests on try either - you get
emails after the fact if your commit is in the range of a regression.
Maybe raptor should also send this kind of emails?
Fabrice
On 10/06/2015 02:35 AM, Dietrich Ayala wrote:
> I don't see any individual bashing in this thread. It's an
> organizational change, as you say.
>
>
> On Tue, Oct 6, 2015, 11:29 AM Etienne Segonzac <[email protected]
> <mailto:[email protected]>> wrote:
>
>
> On Tue, Oct 6, 2015 at 11:12 AM, Dietrich Ayala <[email protected]
> <mailto:[email protected]>> wrote:
>
> To be clear, you mean manually but not locally, right? I am
> talking about using automation prior to landing on the trees
> consumed by downstream, like is done with gecko+Firefox with
> inbound trees, etc. Things like try servers.
>
> Nobody should have to run them locally and our learning there is
> that locally run test results are not to be trusted.
>
>
> Interesting. We're in complete agreement about what we need (sorry
> about the snarky-ness of my earlier comment).
>
> But the harsh reality is that raptor tests are still too noisy to be
> displayed on *gaia*-try.
> Which means there's a long way to go before gecko try builds warn
> platform developers about OS performance regressions.
>
> It's a huge task, with many technical challenges (AFAIK we'd need
> on-device runs to get really good results), and we have 2 developers
> on it (against, AFAIK).
>
> This is why I'd like to see a bit less individual-developer bashing
> and a bit more organizational change to show that, as a project, we
> care about performance.
>
--
Fabrice Desré
b2g team
Mozilla Corporation
_______________________________________________
dev-fxos mailing list
[email protected]
https://lists.mozilla.org/listinfo/dev-fxos
