I LOVE IT But really, I was failing to reproduce a test failure[1] locally and I gave quibble a try and it failed! After a bit of playing around I was able to do a full git bisect, pinpointing the problem.
My main feature request would be to be able to run each "stage" independently, e.g. --stage=qunit or --stage=phpunit or --stage=composer-test, etc. I'd think we'd need that anyways to keep our current parallelization of jobs. [1] https://phabricator.wikimedia.org/T191863 -- Legoktm On 04/05/2018 01:13 PM, Antoine Musso wrote: > Hello, > > I would like to introduce Quibble, a small script intended to easily run > all MediaWiki tests with just a few keystrokes. It will ultimately be > used to power up all the CI jobs having to deal with MediaWiki. > > During the Vienna hackathon (2017), we had a lot of discussions > regarding Docker and how to reproduce what is being run by CI. I wrote > a quick proof of concept that was able to run the MediaWiki PHPUnit > tests as well as the QUnit ones. For the last three weeks I came back to > the POC and largely polished it to a point in confident in switching to it. > > Given an environment providing a PHP flavor, NodeJS, chromium, > composer/npm and a database (MySQL, MariaDB, sqlite), Quibble setups > MediaWiki and then runs all tests commands you can imagine of. > > To install Quibble: > > git clone https://gerrit.wikimedia.org/r/p/integration/quibble > cd quibble > python3 -m pip install -e . > > Then eventually one can just: > > quibble > > Or try some other alternatives: > > ZUUL_BRANCH=REL1_30 quibble --db sqlite --packages-source composer > > > Refer to the README.md for some more basic setup, specially to setup a > cache of git repositories. There is a Dockerfile available at the root > of the repository for some quick testing, and a reference Docker > container intended for CI in integration/config: > > docker-registry.discovery.wmnet/releng/quibble-stretch:latest > > > The default workflow is: > > * clone mediawiki/core and mediawiki/skins/Vector > * clone mediawiki/vendor > * spawn mysqld (only tested with MariaDB) > * run install.php > * composer require mediawiki/core devDependencies > * npm install > > At this point the basics are set. Quibble then runs: > > * phpunit excluding tests in the 'Database' group > * composer test, passing it solely files changed in HEAD > * npm test > > * spawn a webserver using maintenance/dev/includes/router.php > * qunit (grunt karma:main) > * if no DISPLAY is available, spawn Xvfb and Chromedriver pointing to it > * the Webdriver.io suite (grunt webdriver:test) > > * Finally the PHPUnit Database tests. > > A full run in Jenkins takes roughly 6 minutes with a warm cache. > > > There is some support to install extensions and run their tests > (phpunit.php --testsuite extensions). > > I have added an experimental job to mediawiki/core mediawiki/vendor and > mediawiki/skins/Vector. I will make sure it passes on master and start > switching to it. > > The job is mediawiki-core-quibble-vendor-mysql-php7-docker in the > 'experimental' pipeline: > https://integration.wikimedia.org/ci/job/mediawiki-core-quibble-vendor-mysql-php7-docker/ > > Please give it a try and send issues, support requests to Phabricator: > https://phabricator.wikimedia.org/tag/quibble >
signature.asc
Description: OpenPGP digital signature
_______________________________________________ QA mailing list QA@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/qa
