One "trick" that I've regularly done when concerned about the perf impact of a patch is to:
a) Wait till late at night pacific time, when very few checkins are landing (often there are multi-hour gaps). b) Land your change, and wait till the build bots all start building (about a minute later). c) Revert your change. You then get a PILE of perf stats with minimal effort. It is MUCH harder to try to emulate this effect on your own machine(s), as the perf bots are dedicated to the chore, and you have a control build to watch... and a before/after set of results to compare with. By doing this late at night you have the highest probability of getting the impact of ONLY your change, and not having a second contemporaneous landing add noise to your results. :-) If you don't use those computer cycles late at night... we were going to have to waste them! YMMV. Jim p.s., Be sure you are good at doing reverts before playing this game ;-). On Tue, Nov 3, 2009 at 2:04 PM, Anton Muhin <[email protected]> wrote: > > Thanks, James. > > So I ran SunSpider, V8 and page_cyclers. SunSpider and V8 are roughly the > same. > > page_cyclers trickier. First of all, I failed to run some tests on my > box (for both clean and patched builds)---all *Http tests just hang. > I hope it's missetup on my part, but hopefully not crucial. Of tests > ran for reference build second run (patched version) was slightly > faster, so numbers of patched version should be higher just due to > somewhat different state of my box. Still for most of the tests > patched version performed slightly faster, the only problematic test > was the first one (MozFile): 35326 ms total for clean vs. 38936 ms > total for patched. Given that's the only one (e.g. Intl1File ran > 115913 ms on clean vs. 94831 ms on patched), that might have been a > spike. > > If that sounds reasonable to you all, I'd ask to commit-queue+ a patch > and see how it'd perform on real buildbots. > > tia and yours, > anton. > > On Tue, Nov 3, 2009 at 11:31 PM, James Robinson <[email protected]> wrote: > > Essentially this. Some of the page cyclers have fairly interesting data > > sets. > > I think we are fairly lacking in good real-world performance tests to do > > comparisons like this, which is kind of unfortunate. I'd love to be able > to > > point a build with a speculative patch in it at the internet and then > have > > it report back how well it did but there are a lot of variables to > control. > > - James > > > > On Tue, Nov 3, 2009 at 10:38 AM, Anton Muhin <[email protected]> > wrote: > >> > >> Thanks a lot, Peter. > >> > >> Yes, I am aware of those. Just wanted to run something before > committing > >> :) > >> > >> I actually ran some DOM related tests. Adam (on IM) suggested to run > >> SunSpider, V8 and page load cyclers. I am going to do it and if > >> numbers are fine, then would try to commit my change. > >> > >> yours, > >> anton. > >> > >> On Tue, Nov 3, 2009 at 9:25 PM, Peter Kasting <[email protected]> > wrote: > >> > On Tue, Nov 3, 2009 at 7:30 AM, Anton Muhin <[email protected]> > wrote: > >> >> > >> >> So the question: how could I check that I won't (notably) regress > >> >> Chromium performance with this change? > >> > > >> > We have perf bots; you can check in and look at the effects there, and > >> > there > >> > might even be a way to get perf results from the tryservers. > >> > PK > > > > > > > > > > > > > --~--~---------~--~----~------------~-------~--~----~ Chromium Developers mailing list: [email protected] View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~----------~----~----~----~------~----~------~--~---
