| WMDE-leszek created this task. WMDE-leszek added projects: Wikidata, Lexicographical data, Continuous-Integration-Infrastructure. Herald added a subscriber: Aklapper. |
Background: discovered when adding a node selenium test to WikibaseLexeme extension.
In order to run node selenium tests of the extension, we've added mediawiki-core-qunit-selenium-jessie job to extension config template (see https://gerrit.wikimedia.org/r/#/c/419479/). I've assumed that this job runs all node selenium tests of MW core, and all installed extensions. This seemed like a reasonable thing to do as a part of tests of the extension.
Our extension installs CirrusSearch as a part of the test environment, and apparently said job does not compute together with the way CirrusSearch structures its selenium tests, see the test failure: https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/16892/console.
Apparently CirrusSearch has its regular selenium tests in tests/integration, and runs them using the npm job. tests/selenium only contains smoke tests, which, according to README (https://phabricator.wikimedia.org/diffusion/ECIR/browse/master/tests/selenium/README.md), are run daily targeting beta enwiki. In other words, CirrusSearch does not even use mediawiki-core-qunit-selenium-jessie in their test pipeline. And no wonder those smoke tests fail when run targeting local wiki as part of our CI build.
So it could be that tests/selenium is not the place the extension should their "normal" selenium tests. We could have made a mistake in WikibaseLexeme, but it seemed other extensions, e.g. AdvancedSearch, or Popups have made it too.
What would be a reasonable and right thing to do here? We don't want to force CirrusSearch to change anything, clearly this structure makes sense for them. We'd prefer to use a tests/selenium as a directory to store our tests though, and also skipping CirrusSearch from the test environment is also not really an option.
Should we move our selenium tests to some other dir, not use mediawiki-core-qunit-selenium-jessie, and have them run using npm test, using something similar CirrusSearch does: https://phabricator.wikimedia.org/diffusion/ECIR/browse/master/Gruntfile.js
Cc: Addshore, hashar, WMDE-leszek, Aklapper, Lahi, Gq86, Cinemantique, GoranSMilovanovic, QZanden, LawExplorer, thcipriani, Wikidata-bugs, aude, Darkdadaah, Mbch331, Jay8g, greg
_______________________________________________ Wikidata-bugs mailing list Wikidata-bugs@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
