On Tue, Mar 26, 2013 at 8:15 PM, Ryan Lane <[email protected]> wrote: > On Tue, Mar 26, 2013 at 3:58 PM, Asher Feldman <[email protected]>wrote: > >> There are all good points, and we certainly do need better tooling for >> individual developers. >> >> There are a lot of things a developer can do on just a laptop in terms of >> profiling code, that if done consistently, could go a long way, even >> without it looking anything like production. Things like understanding if >> algorithms or queries are O(n) or O(2^n), etc. and thinking about the >> potential size of the relevant production data set might be more useful at >> that stage than raw numbers. When it comes to gathering numbers in such an >> environment, it would be helpful if either the mediawiki profiler could >> gain an easy visualization interface appropriate for such environments, or >> if we standardized around something like xdebug. >> >> The beta cluster has some potential as a performance test bed if only it >> could gain a guarantee that the compute nodes it runs on aren't >> oversubscribed or that the beta virts were otherwise consistently >> resourced. By running a set of performance benchmarks against beta and >> production, we may be able to gain insight on how new features are likely >> to perform. >> >> > This is possible in newer versions of OpenStack, using scheduler hinting. > > That said, the instances would still be sharing a host with each other, > which can cause some inconsistencies. We'd likely not want to use beta > itself, but something that has limited access for performance testing > purposes only, as we wouldn't want other unrelated testing load to skew > results. Additionally, we'd want to make sure to avoid things like > /data/project or /home (both of which beta is using), even once we've moved > to a more stable shared storage solution, as it could very heavily skew > results based on load from other projects. > > We need to upgrade to the Folsom release or greater for a few other > features anyway, and enabling scheduler hinting is pretty simple. I'd say > let's consider adding something like this to the Labs infrastructure > roadmap once the upgrade happens and we've tested out the hinting feature.
I am concerned in this discussion with insufficient testbed load generation and avoidance of confounding variables in the performance analysis... -- -george william herbert [email protected] _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
