On Wed, Jul 22, 2015 at 9:58 AM, Dan Duvall <[email protected]> wrote: > I like this idea for development and manual testing, but I'm not sure it's > appropriate for automated testing. > Agreed. I was thinking more for manual testing.
> Generally in automated testing, each test case (unit test or integration > scenario) should be able to set up or guarantee its preconditions whenever > possible—and otherwise communicate its assumptions—which in the context of > MediaWiki includes user accounts, user settings, or article content that the > test case will manipulate or otherwise depend on. Putting this initial > content in MW-Vagrant, which is orthogonal to MW test suites, creates the > need for more coupling between test code and test environments and promotes > more non-deterministic test behavior, something we've been trying very hard > to reduce.[1][2] > > Also, for development content, we might want to figure out a better place > for it than in the MW-Vagrant repo itself. Article dumps are likely to build > up fast, and we risk bloating the project repo itself with big blobs. > > [1] > https://www.mediawiki.org/wiki/Quality_Assurance/Browser_testing/Environment_abstraction_layer > [2] > https://doc.wikimedia.org/rubygems/mediawiki-selenium/index.html#User_Factory > > > On Tue, Jul 21, 2015 at 10:25 AM, Jon Robson <[email protected]> wrote: >> >> On 20 Jul 2015 5:56 pm, "Greg Grossmeier" <[email protected]> wrote: >>> >>> Given the topic, let's keep the QA list in the loop on this so the >>> MW-Vagrant maintainers can participate/see. >> >> Great :) >>> >>> >>> Also, it looks like the original bug (reported in the MW-Vagrant >>> project) covers this specific request from Reading, no? Essentially, >>> >>> let's see how far we can get with a general "MW-Vagrant (WMF?) testing >>> data import" instead of a vertical specific "reading-web test data set". >>> If what the Reading team needs is way too much for this then we can >>> break it out, otherwise it seems like a needless distinction. >> >> >> >> It does yup. I've already tagged the bug with it. I'm hoping by tackling >> this we can come up with a common solution. The way I imagine this working >> in future is we have various vagrant roles for stock data e.g. >> reading-web-stock-data, editing-web-stock-data, sad-web-stock-data >> There would also be non team specific stock data that might be a sub role >> of this, for example, the reading web team commonly has to setup the >> wikidata role and manually create articles in the wikidata instance and >> local instance that are tied to each other - this takes a ridiculous amount >> of time and is one I'm keen to automate, given that we are leaning more >> heavily on wikidata descriptions and other data in there. >> >> Rob - I've setup https://www.mediawiki.org/wiki/Reading/QA/Sample_articles >> as a place we can start to collect and think about these pages. >> >>> >>> Greg >>> >>> PS: https://wikitech.wikimedia.org/wiki/Labs_labs_labs >>> >>> <quote name="Rob Moen" date="2015-07-20" time="17:11:07 -0700"> >>> > Historically developers have had to setup their own content in >>> > mediawiki >>> > and in mediawiki-vagrant. While this can be done with a simple import, >>> > getting everyone on the same page is apparently not as easy. This is >>> > generally problematic as we would like to test code locally and >>> > remotely >>> > with the same content for various reasons. >>> > >>> > Slightly more frustrating, there are pages titled "0.4425590476103759" >>> > on >>> > beta labs. While trying to sign off on a feature, there is usually a >>> > struggle when trying to find an article with suitable content. AFAIK >>> > this >>> > won't change beta labs but would provide a nice standard for our >>> > content on >>> > test wikis. >>> > >>> > We aim to better things by creating a vagrant role for importing a set >>> > of >>> > articles for testing purposes. For more information please see related >>> > phabricator tasks [1] and [2]. >>> > >>> > In hopes of making this a nice collection of articles that multiple >>> > teams >>> > would use, we would like to get input from our designers and devs on >>> > what >>> > types of articles should be in this import. What qualities should >>> > these >>> > articles contain? >>> > >>> > 1: https://phabricator.wikimedia.org/T104561 >>> > 2: https://phabricator.wikimedia.org/T62116 >>> > >>> > >>> > >>> > -- >>> > Rob Moen >>> > Wikimedia Foundation >>> > [email protected] >>> >>> > _______________________________________________ >>> > Mobile-l mailing list >>> > [email protected] >>> > https://lists.wikimedia.org/mailman/listinfo/mobile-l >>> >>> >>> -- >>> | Greg Grossmeier GPG: B2FA 27B1 F7EB D327 6B8E | >>> | identi.ca: @greg A18D 1138 8E47 FAC8 1C7D | >>> >>> _______________________________________________ >>> QA mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/qa >> >> >> _______________________________________________ >> Mobile-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/mobile-l >> > > > > -- > Dan Duvall > Automation Engineer > Wikimedia Foundation -- Jon Robson * http://jonrobson.me.uk * https://www.facebook.com/jonrobson * @rakugojon _______________________________________________ Mobile-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mobile-l
