Thank you Lila. Pine On Oct 9, 2014 11:50 AM, "Lila Tretikov" <[email protected]> wrote:
> We are planning to do a review of alignment/divergence with the plan mid > year. That would be the right time for this discussion as we will have a > good overview/gap analysis at that time. > > On Thu, Oct 9, 2014 at 11:37 AM, Pine W <[email protected]> wrote: > > > Thanks for the detailed comments, Erik. > > > > As someone who spent several volunteer hours reviewing the current Annual > > Plan, I would appreciate getting an understanding of how the change of > > emphasis to quarterly reviews affects budgets, hiring plans, and > > fundraising goals. Is that something that you can address or should I ask > > Garfield? > > > > Thanks, > > Pine > > On Oct 9, 2014 4:44 AM, "Erik Moeller" <[email protected]> wrote: > > > > > On Thu, Oct 9, 2014 at 10:26 AM, Pine W <[email protected]> wrote: > > > > > > > I'm sure a Board member, Lila, or Erik will correct me if I am > > mistaken, > > > > but my understanding is that there is internal agreement at Board > level > > > > that the Product side of the org needs some systemic changes, that > Lila > > > was > > > > chosen with the goal of making those changes, and that some changes > are > > > > already happening. > > > > > > There's agreement at all levels that we want to continue down the path > > > set by Sue back in 2012 [1] for WMF to truly understand itself as a > > > technology and grantmaking organization. That path led to where we are > > > today: > > > > > > 1) As part of the ED transition, Sue recommended (and the Board > > > accepted the recommendation) to seek an ED with a strong > > > technology/product background, and we hired Lila Tretikov as Sue's > > > successor who matches those requirements. > > > > > > 2) In November 2012, I recommended that we prepare for building out > > > new functions for UX and Analytics, and prepare for dedicated > > > leadership for Engineering and Product. Sue accepted this > > > recommendation. I hired Directors for UX and Analytics in 2013, > > > followed by Community Engagement in 2014, and finally we hired a VP > > > Engineering last week to complete the process. > > > > > > 3) To better account for the need to learn quickly and adjust course > > > as appropriate, we introduced quarterly reviews in December 2012 [3] > > > and increasingly reduced the specificity of Annual Plan level > > > commitments while increasing the focus on metrics and accountability > > > in the reviews. > > > > > > 4) On the technology and product front, many improvements to process > > > and support infrastructure have been implemented in the last couple of > > > years, including but not limited to: > > > > > > - Development of MediaWiki Vagrant as a standardized dev environment, > > > to reduce failure cases due to developer environment inconsistencies > > > - Improvements to continuous integration infrastructure for PHP unit > > > tests and QUnit JavaScript unit tests, and increased focus (but not > > > nearly enough yet) on automated tests, especially for newly developed > > > features > > > - Introduction and continued improvement of BetaLabs as a staging > > > environment for all commits, increased use of automated end-to-end > > > browser tests and QA testing by humans to catch bugs and regressions > > > prior to production rollouts > > > - Introduction and use of various tools for measuring the impact of > > > features, including EventLogging as a standard instrumentation > > > framework for measuring feature usage, dashboards for visualizing > > > usage, WikiMetrics for analyzing editor cohort behavior, Editor > > > Engagement Vital Signs for understanding system-wide user behavior, > > > analysis of pageview data using Hadoop (just rolled out), etc. > > > - Highly specialized automated testing frameworks for specific > > > projects, e.g. Parsoid round-trip testing and visual diffing (!) to > > > detect dirty diffs or output problems > > > - Introduction of design research as a discipline in the UX team > > > (through hiring of Abbey Ripstra as User Research Lead) and > > > incorporation of user studies in a much more systematic way across > > > products > > > - Community liaisons dedicated to key products, responding to user > > > feedback and helping Product Managers understand more complex > > > community needs > > > - Continued shortening of release/deployment cycles; significant > > > improvements to deployment tooling, rewriting our legacy "scap" tools > > > to increase the ability to monitor and reason about deployments; > > > introduction of daily "SWAT" deploys to quickly release fixes, etc. > > > - Introduction of various infrastructure tools that help us better > > > analyze/profile issues, including logstash for log analysis, increased > > > use of graphite for performance metrics collection and various > > > front-ends for visualizing those metrics > > > - Shift towards loosely coupled services, addressing the difficulty of > > > maintaining and improving our highly monolithic codebase (examples > > > include Parsoid, Citoid, Mathoid, and the new Content API in > > > development) > > > - Introduction of Beta Features framework to stage features for early > > > adopters > > > > > > 5) The changes Lila has pushed for since we started include: > > > > > > - Greater focus on quarterly prioritization and a "rolling roadmap" > > > rather than a fiscal year view of the world > > > - Increased emphasis on understanding the needs of different user > > > personas at all cycles of software development, including through use > > > of qualitative and quantitative methods > > > - Reducing velocity of user-facing changes (esp. on desktop) to > > > increase focus on foundations (platform/process improvements) that > > > ultimately will enable us to move faster and more effectively > > > - Documenting product development methodology on-wiki and establishing > > > a clearer social contract (to reduce the reliance on RFCs/votes > > > regarding feature configurations) > > > - Surveying the needs of current users to more systematically balance > > > projects that serve future/new users vs. projects that serve the users > > > we have today > > > - Improved communication channels for community engagement to make it > > > easier to understand what major projects are currently in development > > > and how to provide feedback > > > > > > This already means, effectively, that the commitments in the Annual > > > Plan developed during Sue's time should be taken with a big block of > > > salt at this point in time -- we're slowing down the deployment (not > > > development) of big user-facing features like Flow and VE as much as > > > needed to ensure that we incorporate user feedback, data and > > > qualitative research into the product development process > > > appropriately and spend sufficient time on the technical foundations > > > for these projects. > > > > > > The quarterly prioritization alone has been, IMO, a huge improvement > > > that's already paying off. In the "Annual Plan" view of the world, > > > it's unlikely that we would have prioritized a project like HHVM the > > > way we did, because we were generally stuck on the priorities set for > > > the whole year. But it was very clear that this project would provide > > > huge benefits to our users, and I'm glad we were able to call it out > > > as _the_ top priority for Q1 and give the team the space to really > > > focus on getting it done (almost there now, starting to serve reader > > > traffic [4]). > > > > > > Our draft Q2 top priorities (not yet posted on-wiki, but discussed in > > > the metrics meeting last week) are consistent with the above, with the > > > main user-facing push being on mobile web/apps and editing > > > performance, while the other priorities are more > > > platform/process-related. Once again, we're continuing to work on VE / > > > Flow, but focusing more on fundamentals (performance, architecture, > > > testing, use case analysis, etc.) than accelerating deployments. > > > > > > My focus over the coming days is to flesh out the details for the Q2 > > > priorities, and then shift to putting more effort in documenting and > > > refining product development methodologies and processes on-wiki. On > > > the engineering side, there's plenty of process/infrastructure > > > improvement to do as well. From my point of view, continued > > > improvement to test coverage and CI/testing infrastructure, developer > > > tools, profiling/instrumentation, staged roll-out support and > > > strengthening of architectural leadership are the big pieces for > > > coming months, but I'll let Damon speak to his focus areas as he gets > > > the lay of the land. > > > > > > Erik > > > > > > [1] https://meta.wikimedia.org/wiki/User:Sue_Gardner/Narrowing_focus > > > [2] > > > > > > https://lists.wikimedia.org/pipermail/wikimedia-l/2012-November/122663.html > > > [3] > > > > > > https://lists.wikimedia.org/pipermail/wikimedia-l/2012-December/123088.html > > > [4] https://gerrit.wikimedia.org/r/#/c/165004/ > > > -- > > > Erik Möller > > > VP of Product & Strategy, Wikimedia Foundation > > > > > > _______________________________________________ > > > Wikimedia-l mailing list, guidelines at: > > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > > [email protected] > > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > > <mailto:[email protected]?subject=unsubscribe> > > _______________________________________________ > > Wikimedia-l mailing list, guidelines at: > > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > > [email protected] > > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > > <mailto:[email protected]?subject=unsubscribe> > > > _______________________________________________ > Wikimedia-l mailing list, guidelines at: > https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines > [email protected] > Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, > <mailto:[email protected]?subject=unsubscribe> _______________________________________________ Wikimedia-l mailing list, guidelines at: https://meta.wikimedia.org/wiki/Mailing_lists/Guidelines [email protected] Unsubscribe: https://lists.wikimedia.org/mailman/listinfo/wikimedia-l, <mailto:[email protected]?subject=unsubscribe>
