very cool! this will be great for A/B comparing things like effects implementations and magic numbers.
On Fri, 2014-11-14 at 01:42 -0500, RJ Ryan wrote: > Have you ever compiled two versions of Mixxx to run side-by-side to > try and get a feel for whether your "smoothness" change is any good? > > > Are you annoyed at having to continuously re-compile or re-run to test > two different approaches you're considering? > > > Are you bothered by sloppiness in our performance stat tracking -- how > we include Mixxx startup and shutdown events (usually including > mega-outliers) in all of our stat counters and then make decisions > based on that? > > > You may be interested in my latest commit! > https://github.com/mixxxdj/mixxx/commit/25a5fef18eadece78e65390f808abdd66e359796 > > > > This adds a lightweight "experiment" mode that lets you segment your > measurements into a base bucket and an experiment bucket. > > > Create your usual timers / counters / stats trackers as usual and gate > your new code on the experiment flag: > > > #include "util/experiment.h" > > > void CoolWidget::paintEvent(...) { > ScopedTimer t("CoolWidget::paintEvent"); > > > ... beginning of paintEvent ... > > > // added this code > > if (Experiment::isExperiment()) { > // Hope this speeds things up... > QThread::msleep(1); > } else if (Experiment::isBase()) { > // Optional, you can leave blank / as the existing behavior. > } else { > // If you want you can really test 3 different pieces of logic > // but you will only get explicit stats tracking > // for BASE and EXPERIMENT modes. > } > > > ... rest of paintEvent ... > } > > > Compile and run Mixxx. > > > After Mixxx starts up and stabilizes: > Hit Ctrl+Shift+B (or Developer -> Base) to start recording BASE > samples. Toggle to stop recording. > Hit Ctrl+Shift+E (or Developer -> Experiment) to start recording > EXPERIMENT samples. Toggle to stop recording. > You can use this to capture exactly the time periods you would like to > include in your sample of events (recording is cumulative within a > single Mixxx session). > > > When you quit Mixxx, where the stats are normally printed you now get > a "BASE STATS" and "EXPERIMENT STATS" section. > > > Key points: > - No more startup and shutdown events included in your stats > tracking! > - Stats segmented into BASE and EXPERIMENT -- only for the times you > choose to record. > - Easily check anecdotal "feel" of a feature without re-compiling or > re-starting Mixxx! > > > Future work: > - Automatic comparison of stats across base and test (with hints about > whether you have enough samples and statistical significance?) > - Automatic polling modes that regularly swap between base / > experiment? > > > Go forth and measure! > RJ > ------------------------------------------------------------------------------ > Comprehensive Server Monitoring with Site24x7. > Monitor 10 servers for $9/Month. > Get alerted through email, SMS, voice calls or mobile push notifications. > Take corrective actions from your mobile device. > http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk > _______________________________________________ > Get Mixxx, the #1 Free MP3 DJ Mixing software Today > http://mixxx.org > > > Mixxx-devel mailing list > Mixxx-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/mixxx-devel ------------------------------------------------------------------------------ Comprehensive Server Monitoring with Site24x7. Monitor 10 servers for $9/Month. Get alerted through email, SMS, voice calls or mobile push notifications. Take corrective actions from your mobile device. http://pubads.g.doubleclick.net/gampad/clk?id=154624111&iu=/4140/ostg.clktrk _______________________________________________ Get Mixxx, the #1 Free MP3 DJ Mixing software Today http://mixxx.org Mixxx-devel mailing list Mixxx-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mixxx-devel
