hashar added a comment.
Since this is causing outages, I guess it is really time to tackle the issue. For the context: We have wmf-quibble-* jobs which clone several extensions and run all their tests. That is also known as the extension-gate. Scribunto has a set of test to make sure it works fine with some LUA interpreters. That is really nice but takes a few minutes iirc. When one send a patch for another extension, we are 100% sure they are not going to affect how LUA runs. The wmf-quibble jobs do: - trigger for extensions having in zuul/layout.yaml the template `extension-gate` - have a list of all extensions listed in zuul/parameter_functions.py in `gated_extensions` variable - a test in integration/config ensure that both lists match #Quibble <https://phabricator.wikimedia.org/tag/quibble/> roughly runs the MediaWiki core PHPUnit suite `e``xtensions` defined as: name=tests/phpunit/suite.xml <testsuite name="extensions"> <directory>structure</directory> <file>suites/ExtensionsTestSuite.php</file> <file>suites/ExtensionsParserTestSuite.php</file> <file>suites/LessTestSuite.php</file> </testsuite> It does a first run without tests flagged with `@group Database`: tests/phpunit/phpunit.php --testsuite extensions --exclude-group Broken,ParserFuzz,Stub,Database Then a second run with those Database tests: tests/phpunit/phpunit.php --testsuite extensions --group Database --exclude-group Broken,ParserFuzz,Stub The group exclusions are hardcoded in Quibble. The extension test suite would add tests from any extensions /tests/phpunit directories as well as those added via the UnitTestsList hook. ---- The behavior I would like is a way to exclude a subset of Scribunto test unless the patch got send for that extension. What I thought is to flag those specific tests with a new group such as `@group system-integration`. Then Quibble would exclude that group from both of the commands above and add a third run which would be: tests/phpunit/phpunit.php --group system-integration --exclude-group Broken,ParserFuzz,Stub -- /path/to/extension/that/triggered/the/patch Which I think would address it? TASK DETAIL https://phabricator.wikimedia.org/T125050 EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: hashar Cc: Pablo-WMDE, Jakob_WMDE, Tarrow, gerritbot, Addshore, TerraCodes, Liuxinyu970226, EBernhardson, Smalyshev, greg, Legoktm, Stashbot, Lydia_Pintscher, Aleksey_WMDE, Ladsgroup, WMDE-leszek, Anomie, hashar, aude, thcipriani, Paladox, hoo, Aklapper, JanZerebecki, StudiesWorld, Redabr4, alaa_wmde, Zanziii, Sadisticturd, joker88john, CucyNoiD, Nandana, NebulousIris, Gaboe420, A.S.Kochergin, Versusxo, Majesticalreaper22, Giuliamocci, Adrian1985, Cpaulf30, Lahi, Gq86, Baloch007, Xinbenlv, Darkminds3113, Bsandipan, Lordiis, GoranSMilovanovic, Adik2382, Th3d3v1ls, Ramalepe, Liugev6, QZanden, LawExplorer, WSH1906, Lewizho99, Maathavan, SundanceRaphael, _jensen, rosalieper, Jonas, Wikidata-bugs, Dinoguy1000, jayvdb, MrStradivarius, Jackmcbarn, Mbch331
_______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
