Thank you for the update Timo, good to see so many improvements finally coming together.
DJ > On 4 apr. 2015, at 05:40, Krinkle <[email protected]> wrote: > > TL;DR: > * Our QUnit jobs now uses latest Chromium instead of PhantomJS. > * You can run the test suite from the command line locally now. > > Thanks to Tim Starling, Antoine "hashar" Musso, Kunal (legoktm), > Bryan Davis, S Page, and James Forrester; for their help in different areas. > > https://www.mediawiki.org/wiki/Manual:JavaScript_unit_testing > > > Are you sitting comfortably? > > The past few months can be summarised as a long journey through a forest of > technical debt. It was also amazing to see just how many layers of > infrastructure were able to block this task (and did). [1] > > From a local development point of view it couldn't be simpler. In the > Gruntfile: Remove grunt-contrib-qunit, add grunt-karma with > karma-chrome-launcher. Done? Not quite. > > For standalone front-end projects it actually was this simple. OOjs and > VisualEditor have been enjoying this new stack since July 2014. This helped > refine the stack and the underlying technologies over the past months. More > about what this new stack provides in a minute. > > == Journey == > > === Export from Special:JavaScriptTest === > > For MediaWiki core and extensions, one must install MediaWiki before running > QUnit. [2][3] That in itself isn't too complicated (set up DB and run > install; we have standard macros for that). But, the test doesn't just > communicate with MediaWiki. The test suite is actually served *by* MediaWiki. > Karma enforces the principle that unit tests run in pure javascript (blank > page with sockets, load source files, load QUnit, run test suite). > > Back in MediaWiki 1.19 our test suite abided by these best practices. The > test suite was a static HTML file that loaded relevant scripts directly. [4] > This file was later migrated to Special:JavaScriptTest, which opened the door > for undeclared dependencies. > > 1. Source code is registered to ResourceLoader. > 2. Extensions register tests via hooks in PHP. > 3. OutputPage and Skin provide config vars and HTML that tests could depend > on. > > Point 3 was easily resolved by adding the appropriate mw.config or DOM > fixture to the few tests that were missing it. > > Though Point 2 and 1 weren't going anywhere. We needed to, once again, access > our tests suite as a pure JavaScript payload, with no HTML or script tags > loading relevant resources. > > Hence, the introduction of Special:JavaScriptTest/qunit/export in > https://gerrit.wikimedia.org/r/178551. > > === Migrate all the things === > > Objective: Karma [5] on Chromium. Easy to run locally for developers. > > To be re-usable, the logic must be generic and use composer/npm. In 2013, > most jobs ran on Wikimedia production servers. In order to use npm we had to > migrate to Wikimedia Labs first. > > Back then, production had only just begun using Ubuntu Trusty. > > The relevant modules require Node.js v0.10. Ubuntu Precise has Node.js v0.8. > Precise is also stuck with Chromium 37 (EOL). [7] To get auto-updating latest > stable Chromium and Node v0.10, we needed to migrate our infrastructure to > Ubuntu Trusty first. > > Much of the process to install MediaWiki on a Jenkins slave also wasn't > puppetised. > > === SQLite === > > This is its own story. See https://phabricator.wikimedia.org/T89180 for > details. > > The short of it is, we also migrated everything to MySQL. Which we wanted to > do anyway. > > == New stack == > > 1. Chromium > > PhantomJS is a great application for many purposes. It was good for us while > it lasted. But for JavaScript unit testing, PhantomJS just isn't meant to be. > It's too distant from a "real" browser. When we introduced PhantomJS, it was > a big step forward toward testing cross-browser. It "uses WebKit" which meant > we were kind of covering Chrome and Safari (in 2012). > > Safari has had several major releases since PhantomJS v1.9 came out. Chrome > had even more releases (and since dropped WebKit in favour of Blink). And, in > truth, PhantomJS wasn't that much like Safari and mainstream WebKit to begin > with. [8] > > Chromium is an actual browser. A browser we actually support. A browser that > represents real users of our software. It's only one browser for now, but > it's a start. > > 2. Karma > > With Karma as a solid foundation, it's truly simple to add more browsers in > the future. OOjs has already added Firefox alongside Chromium in Jenkins. > Karma uses Web Driver, a standardised protocol to operate browsers (locally > and remotely alike). It has an official plugin hooking it up to SauceLabs, > which opens the door to any other browser and platform we want. It's as > simple as adding 1 line to the config. [5][6] > > We haven't done this yet as there are scalability, performance and security > considerations. Having said that, OOjs is currently using an experimental > pipeline to test in six different browsers from Jenkins via SauceLabs. > Including IE 6 and IE 11 on Windows, and Safari 5 on Mac! [9] > > 3. Code coverage > > One of the advantages of having a pure stack is the ability to run > pre-processors on the code using Karma built-ins. [10] The first one I'm > looking at is code coverage. > > 4. Local development and Jenkins equally > > It runs the full QUnit test suite in a real browser (multiple even) by simply > running "npm test". [11] > > Aside from Node.js, it requires no pre-installed software and is as easy as > "npm install" to set up. > > > — Krinkle > > [1] https://phabricator.wikimedia.org/T74063 [epic] Adopt Karma with Chromium > (tracking). > > [2] https://phabricator.wikimedia.org/T89433 Make QUnit tests run without > installing MediaWiki. > [3] https://phabricator.wikimedia.org/T89432 Make PHPUnit tests run without > installing MediaWiki. > [4] https://github.com/wikimedia/mediawiki/blob/REL1_19/tests/qunit/index.html > > [5] https://karma-runner.github.io/ > [6] https://karma-runner.github.io/0.12/config/browsers.html > > [7] http://packages.ubuntu.com/precise/chromium-browser > [8] http://codepen.io/Krinkle/blog/phantomjs-anno-2014 > > [9] > https://github.com/wikimedia/oojs/blob/v1.1.6/Gruntfile.js#L76-L107 > https://integration.wikimedia.org/ci/job/npm/1680/console > > [11] > https://karma-runner.github.io/0.12/config/preprocessors.html > https://github.com/karma-runner/karma-coverage > > [12] https://www.mediawiki.org/wiki/Manual:JavaScript_unit_testing > > > _______________________________________________ > Wikitech-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
signature.asc
Description: Message signed with OpenPGP using GPGMail
_______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
