[Wikidata-bugs] [Maniphest] [Updated] T160989: Revisit Jenkins jobs being triggered for Wikibase

[hashar]

hashar added a comment.

I wanted to check the overlaps between three different tests jobs, generic, client and repo:

• mwext-Wikibase-repo-tests-sqlite-hhvm-jessie
• mwext-Wikibase-client-tests-mysql-hhvm-jessie
• mwext-testextension-hhvm-composer-jessie

None of them seems to rely on mediawiki/vendor.

Config vary

The jobs invoke the Wikibase script build/jenkins/mw-apply-wb-settings.sh which to vary $wg settings:

client

$wgEnableWikibaseRepo = false;                                                                                                
$wgEnableWikibaseClient = true;                                                                                               
$wgWikimediaJenkinsCI = true;                                                                                                 
$wmgUseWikibaseRepo = false;                                                                                                  
$wmgUseWikibaseClient = true;

repo
$wgEnableWikibaseRepo = true;                                                                                                        
$wgEnableWikibaseClient = true;                                                                                                      
$wgWikimediaJenkinsCI = true;                                                                                                        
$wmgUseWikibaseRepo = true;                                                                                                          
$wmgUseWikibaseClient = true;

Extensions dependencies:


generic
BetaFeatures Capiunto CirrusSearch Cite cldr Echo Elastica GeoData MwEmbedSupport PdfHandler Scribunto SiteMatrix TimedMediaHandler VisualEditor
client
Scribunto,Capiunto,cldr,Echo
repo
CirrusSearch,Elastica,GeoData,cldr


PHPUnit is invoked with --group filters:


client
--group Wikibase,WikibaseClient
repo
--group Wikibase,WikibaseAPI,Purtle




Tests list analysis

I have picked the JUnit result files from each three jobs, parsed them to render a text only tree of tests then used vimdiff to compare the three files. The scripts are P5173 and P5174.

client vs repo

They both run Wikibase group albeit with different LocalSettings.

Some Wikibase\Client\Tests are also run in repo.

In both jobs, some tests ends up with 0 assertions. Maybe due to different tags.

client does not run any Wikibase\Repo\Tests (good).

generic vs client

It is almost a full overlap, but some tests in client run extra tests and assertions for some reason. Probably because they have a slightly different set of MediaWiki extensions or some tests are skipped based on the database backend.

Utilities:

F7112173: Makefile

F7112174: parsejunit.py

Tests trees for diff:

Tests are indented by two spaces, each line show the testsuite name, the number of tests and number of assertions:

F7112167: client-mysql.txt

F7112168: client-sqlite.txt

F7112170: repo-sqlite.txt

F7112171: generic-hhvm.txt

TASK DETAIL

https://phabricator.wikimedia.org/T160989

EMAIL PREFERENCES

https://phabricator.wikimedia.org/settings/panel/emailpreferences/

To: hashar
Cc: Krinkle, Paladox, Ladsgroup, greg, Tobi_WMDE_SW, daniel, Lydia_Pintscher, Addshore, thcipriani, hashar, Aklapper, QZanden, Salgo60, D3r1ck01, Izno, Wikidata-bugs, aude, Mbch331, Jay8g

_______________________________________________
Wikidata-bugs mailing list
Wikidata-bugs@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs