It would be really good to have a 2.2 + master Gecko vs. master Gaia + master Gecko comparison. We need to be able to filter out the noise introduced by platform changes (and platform need to filter out the noise between platform and Gaia changes too).
Still really liking Raptor though btw :) --Chris On Mon, Oct 12, 2015 at 7:55 PM, Eli Perelman <[email protected]> wrote: > Hello again fxos, > > Previously [1] I had laid out a comparison of our v2.2 performance > baseline with data available for master. This was done against the baseline > we had at the time, the 319MB Flame, but the effort is now to switch to the > 512MB Flame as the new baseline. > > We have revamped our v2.2 performance baseline for the 512MB Flame, which > also lets us use an updated v2.2 Gecko and Gaia, updated base build, longer > rests between application launches, and performing GCs before memory > capturing. The result is an overall more stable baseline to compare against. > > v2.2 -> master comparison: > https://gist.github.com/eliperelman/b81a27b7f7f3b3f75f49 > > Thanks to some amazing statistical insight from Stas, we are now also > introducing a different calculation to ground our baseline in: 95% Upper > Bound [2]. This is now the calculation used in Raptor v3.x instead of 95th > percentile. > > The dashboards [3] have been updated to display the new baseline for the > 512MB Flame and now use the 95% Upper Bound for charting metrics. If you > have any questions about these metrics and how they are derived, or would > like assistance in testing Raptor with your own apps, please feel free to > reach out. > > Thanks! > > Eli Perelman > > P.S. Huge thanks to :gandalf, :stas, and :rwood for making this new > baseline possible! > > [1] > https://groups.google.com/d/msg/mozilla.dev.fxos/HeGci2jormU/a3rQZcSvAwAJ > [2] I'll let Stas elaborate on what that means if he feels inclined. :) > [3] https://raptor.mozilla.org/ > > _______________________________________________ > dev-fxos mailing list > [email protected] > https://lists.mozilla.org/listinfo/dev-fxos > >
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
