If by try runs you mean automated performance testing when opening a PR, then no. Right now the best way to ensure performance is up-to-snuff with your patch is to run Raptor during development. With Raptor installed, use a performance profile on the device by using the same flags as `make raptor`, then testing an app is as easy as:
raptor test coldlaunch --app clock --runs 10 raptor test coldlaunch --app communications --entry-point dialer --runs 20 During development use a small number of runs for a tighter feedback loop. Before committing/landing, use many runs to ensure a better statistical guarantee. See the Raptor docs for details on getting started [1]. If you need any help getting up and running with Raptor, Rob Wood and myself would be happy to lend a hand, just ping us [2]. [1] https://developer.mozilla.org/en-US/docs/Mozilla/Firefox_OS/Automated_testing/Raptor#Getting_Started [2] We're in #fxos or #raptor Eli Perelman On Mon, Oct 5, 2015 at 5:37 PM, Naoki Hirata <[email protected]> wrote: > I think to add what dietrich is saying, I personally am not condeming > performance on your own local branch or doing test runs there... > > I think we're condeming anything that lands on master as an experiment, > without testing or without running through our jenkins setup etc for > testing performance. > > I don't want to speak too much for the performance team, I'm not sure what > their load is in handling requests though. Eli is there an easy way for > dev to setup try runs w/ performance testing? > > On Mon, Oct 5, 2015 at 1:53 PM, Dietrich Ayala <[email protected]> wrote: > >> +1 to the frog metaphor. >> >> History has shown it's *incredibly* hard to claw back from performance >> regressions. And every moment spent doing so is done *at the cost* of >> exactly the type of work Chris described - work that actually moves the >> project *forward*. >> >> I strongly disagree with any acceptance of any performance regression for >> any reason except emergency security patches. Only a zero tolerance policy >> for perf regressions will result in performant software in such a large and >> complex project. >> >> If you have a tension between perf and features, then it's time to cut >> the slow features, or get some more time. >> >> The polish/bugs problems mentioned is fixed by landing fewer bugs (a >> culture of detailed automated tests and a project-wide love and acceptance >> of backouts), not by accepting perf regressions. >> >> Also, I recommend not using any subjective measure to compare app startup >> times across different platforms. We used tools to do this in the past. >> >> (My first patch ever, in 2006, regressed Firefox startup time and I spent >> a few days on the hook... until my feature could land with no startup hit. >> Can you tell it had an impact on me :D) >> >> On Mon, Oct 5, 2015 at 5:19 PM Fabrice Desré <[email protected]> wrote: >> >>> On 10/05/2015 01:46 AM, Christopher Lord wrote: >>> > I'll preface what I say with the hopefully obvious statement that we >>> > should always aim for everything to be better. That said, however, I'd >>> > take a 2mb memory regression and a half-second startup time regression >>> > if it meant the app was polished and performed well. >>> >>> Some apps regressed by way more than 2MB. And also, beware of the >>> boiling frog. >>> >>> > Have you guys used an Android phone recently? Their startup time for >>> > apps is generally atrocious compared to ours (even on high-end devices) >>> > - we shouldn't drop the ball, but it's not where we compare badly. >>> Given >>> > we aren't targeting 256mb devices anymore, I'd gladly have all our apps >>> > use double the memory they did in 2.2 if it meant we had a consistent >>> > 60Hz update, consistent transitions and snappy response. >>> >>> That's not what I see on a Nexus 4 running CM 12 and on a z3c running L. >>> They are both super fast and snappy when launching the default apps. >>> Still better than us on the same hardware. >>> >>> Fabrice >>> -- >>> Fabrice Desré >>> b2g team >>> Mozilla Corporation >>> _______________________________________________ >>> dev-fxos mailing list >>> [email protected] >>> https://lists.mozilla.org/listinfo/dev-fxos >>> >> >> _______________________________________________ >> dev-fxos mailing list >> [email protected] >> https://lists.mozilla.org/listinfo/dev-fxos >> >> >
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
