Hello, I would like to introduce Quibble, a small script intended to easily run all MediaWiki tests with just a few keystrokes. It will ultimately be used to power up all the CI jobs having to deal with MediaWiki.
During the Vienna hackathon (2017), we had a lot of discussions regarding Docker and how to reproduce what is being run by CI. I wrote a quick proof of concept that was able to run the MediaWiki PHPUnit tests as well as the QUnit ones. For the last three weeks I came back to the POC and largely polished it to a point in confident in switching to it. Given an environment providing a PHP flavor, NodeJS, chromium, composer/npm and a database (MySQL, MariaDB, sqlite), Quibble setups MediaWiki and then runs all tests commands you can imagine of. To install Quibble: git clone https://gerrit.wikimedia.org/r/p/integration/quibble cd quibble python3 -m pip install -e . Then eventually one can just: quibble Or try some other alternatives: ZUUL_BRANCH=REL1_30 quibble --db sqlite --packages-source composer Refer to the README.md for some more basic setup, specially to setup a cache of git repositories. There is a Dockerfile available at the root of the repository for some quick testing, and a reference Docker container intended for CI in integration/config: docker-registry.discovery.wmnet/releng/quibble-stretch:latest The default workflow is: * clone mediawiki/core and mediawiki/skins/Vector * clone mediawiki/vendor * spawn mysqld (only tested with MariaDB) * run install.php * composer require mediawiki/core devDependencies * npm install At this point the basics are set. Quibble then runs: * phpunit excluding tests in the 'Database' group * composer test, passing it solely files changed in HEAD * npm test * spawn a webserver using maintenance/dev/includes/router.php * qunit (grunt karma:main) * if no DISPLAY is available, spawn Xvfb and Chromedriver pointing to it * the Webdriver.io suite (grunt webdriver:test) * Finally the PHPUnit Database tests. A full run in Jenkins takes roughly 6 minutes with a warm cache. There is some support to install extensions and run their tests (phpunit.php --testsuite extensions). I have added an experimental job to mediawiki/core mediawiki/vendor and mediawiki/skins/Vector. I will make sure it passes on master and start switching to it. The job is mediawiki-core-quibble-vendor-mysql-php7-docker in the 'experimental' pipeline: https://integration.wikimedia.org/ci/job/mediawiki-core-quibble-vendor-mysql-php7-docker/ Please give it a try and send issues, support requests to Phabricator: https://phabricator.wikimedia.org/tag/quibble -- Antoine "hashar" Musso _______________________________________________ QA mailing list QA@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/qa
