EBernhardson has uploaded a new change for review. (
https://gerrit.wikimedia.org/r/392470 )
Change subject: Get all nodejs tests passing from empty database
......................................................................
Get all nodejs tests passing from empty database
* Use the new cirrus-article-dump api to wait for edits to make it into
elastic. Failures from an empty database seem almost entirely tied to
tests running before the articles have made it into cirrus.
* Convert one-off batch calls in hooks.js to use a single function
so we don't duplicate checking the batch has made it into elastic
* While at it reduce some promise spaghetti by converting things
over to bluebird coroutines. If we require nodejs >= 7.6 we could
use async/await directly, but coroutines allow us to support node 6
which is default on many distributions.
* Swap config over to headless while we are here.
* Put the @suggest hook, that builds the completion suggester, at the
end of the hooks file. Cucumberjs seems to run these hooks in the
order they are defined, so this ensures all the other tags from
prefix_search_api.feature have run already
* Merge suggest_api.feature with prefix_search_api.feature, as they
both use the @suggest tag.
Change-Id: Ie2f3142d8af9036a6a6e473a2a7d2fd557abeaca
---
M tests/integration/config/wdio.conf.js
M tests/integration/features/prefix_search_api.feature
M tests/integration/features/step_definitions/page_step_helpers.js
M tests/integration/features/step_definitions/page_steps.js
D tests/integration/features/suggest_api.feature
M tests/integration/features/support/hooks.js
M tests/integration/features/support/world.js
7 files changed, 363 insertions(+), 370 deletions(-)
git pull ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/CirrusSearch
refs/changes/70/392470/1
diff --git a/tests/integration/config/wdio.conf.js
b/tests/integration/config/wdio.conf.js
index 179e302..2cd2fb5 100644
--- a/tests/integration/config/wdio.conf.js
+++ b/tests/integration/config/wdio.conf.js
@@ -122,7 +122,7 @@
browserName: 'chrome',
// Since Chrome v57
https://bugs.chromium.org/p/chromedriver/issues/detail?id=1625
chromeOptions: {
- args: [ '--enable-automation' ]
+ args: [ '--enable-automation', '--headless' ]
}
} ],
//
diff --git a/tests/integration/features/prefix_search_api.feature
b/tests/integration/features/prefix_search_api.feature
index e84e830..e524c1b 100644
--- a/tests/integration/features/prefix_search_api.feature
+++ b/tests/integration/features/prefix_search_api.feature
@@ -45,7 +45,6 @@
Scenario: Searching for a bare namespace finds everything in the namespace
Given a page named Template talk:Foo exists
- And within 20 seconds api searching for Template talk:Foo yields
Template talk:Foo as the first result
When I get api suggestions for template talk:
Then Template talk:Foo is in the api suggestions
@@ -155,3 +154,85 @@
# And there are 1000 redirects to IHaveTonsOfRedirects of the form
TonsOfRedirects%s
# When I type TonsOfRedirects into the search box
# Then suggestions should appear
+
+ Scenario: Search suggestions
+ When I ask suggestion API for main
+ Then the API should produce list containing Main Page
+
+ Scenario: Created pages suggestions
+ When I ask suggestion API for x-m
+ Then the API should produce list containing X-Men
+
+ Scenario: Nothing to suggest
+ When I ask suggestion API for jabberwocky
+ Then the API should produce empty list
+
+ Scenario: Ordering
+ When I ask suggestion API for x-m
+ Then the API should produce list starting with X-Men
+
+ Scenario: Fuzzy
+ When I ask suggestion API for xmen
+ Then the API should produce list starting with X-Men
+
+ Scenario: Empty tokens
+ When I ask suggestion API for はー
+ Then the API should produce list starting with はーい
+ And I ask suggestion API for はい
+ Then the API should produce list starting with はーい
+
+ Scenario Outline: Search redirects shows the best redirect
+ When I ask suggestion API for <term>
+ Then the API should produce list containing <suggested>
+ Examples:
+ | term | suggested |
+ | eise | Eisenhardt, Max |
+ | max | Max Eisenhardt |
+ | magnetu | Magneto |
+
+ Scenario Outline: Search prefers exact match over fuzzy match and ascii
folded
+ When I ask suggestion API for <term>
+ Then the API should produce list starting with <suggested>
+ Examples:
+ | term | suggested |
+ | max | Max Eisenhardt |
+ | mai | Main Page |
+ | eis | Eisenhardt, Max |
+ | ele | Elektra |
+ | éle | Électricité |
+
+ Scenario Outline: Search prefers exact db match over partial prefix match
+ When I ask suggestion API at most 2 items for <term>
+ Then the API should produce list starting with <first>
+ And the API should produce list containing <other>
+ Examples:
+ | term | first | other |
+ | Ic | Iceman | Ice |
+ | Ice | Ice | Iceman |
+
+ Scenario: Ordering & limit
+ When I ask suggestion API at most 1 item for x-m
+ Then the API should produce list starting with X-Men
+ And the API should produce list of length 1
+
+ Scenario Outline: Search fallback to prefix search if namespace is provided
+ When I ask suggestion API for <term>
+ Then the API should produce list starting with <suggested>
+ Examples:
+ | term | suggested |
+ | Special: | Special:ActiveUsers |
+ | Special:Act | Special:ActiveUsers |
+
+ Scenario Outline: Search prefers main namespace over crossns redirects
+ When I ask suggestion API for <term>
+ Then the API should produce list starting with <suggested>
+ Examples:
+ | term | suggested |
+ | V | Venom |
+ | V: | V:N |
+ | Z | Zam Wilson |
+ | Z: | Z:Navigation |
+
+ Scenario: Default sort can be used as search input
+ When I ask suggestion API for Wilson
+ Then the API should produce list starting with Sam Wilson
diff --git a/tests/integration/features/step_definitions/page_step_helpers.js
b/tests/integration/features/step_definitions/page_step_helpers.js
index fbcff9d..c7453a2 100644
--- a/tests/integration/features/step_definitions/page_step_helpers.js
+++ b/tests/integration/features/step_definitions/page_step_helpers.js
@@ -26,90 +26,106 @@
}
deletePage( title ) {
- return this.apiPromise.then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.delete( title, "CirrusSearch
integration test delete" )
- .catch( ( err ) => {
- // still return true if page
doesn't exist
- return expect( err.message
).to.include( "doesn't exist" );
- } );
- } );
+ return Promise.coroutine( function* () {
+ let client = yield this.apiPromise;
+ try {
+ yield client.delete( title, "CirrusSearch
integration test delete" )
+ yield this.waitForOperation( 'delete', title );
+ } catch ( err ) {
+ // still return true if page doesn't exist
+ expect( err.message ).to.include( "doesn't
exist" );
+ };
} );
}
editPage( title, text, append = false ) {
- return this.apiPromise.then( ( api ) => {
+ return Promise.coroutine( function* () {
+ let client = yield this.apiPromise;
+
if ( text[0] === '@' ) {
text = fs.readFileSync( path.join( __dirname,
'articles', text.substr( 1 ) ) ).toString();
}
- return this.getWikitext( title ).then( ( fetchedText )
=> {
- if ( append ) {
- text = fetchedText + text;
- }
- if ( text.trim() !== fetchedText.trim() ) {
- return api.loginGetEditToken().then( ()
=> api.edit( title, text ) );
- }
- }, ( error ) => {
- throw error;
- } );
- } );
+ let fetchedText = yield this.getWikitext( title );
+ if ( append ) {
+ text = fetchedText + text;
+ }
+ if ( text.trim() !== fetchedText.trim() ) {
+ yield client.edit( title, text );
+ yield this.waitForOperation( 'edit', title );
+ }
+ } ).call( this );
}
getWikitext( title ) {
- return this.apiPromise.then( ( api ) => {
- return api.request( {
+ return Promise.coroutine( function* () {
+ let client = yield this.apiPromise;
+ let response = yield client.request( {
action: "query",
format: "json",
formatversion: 2,
prop: "revisions",
rvprop: "content",
titles: title
- } ).then( ( response ) => {
- if ( response.query.pages[0].missing ) {
- return "";
- }
- return
response.query.pages[0].revisions[0].content;
- } );
- } );
+ } )
+ if ( response.query.pages[0].missing ) {
+ return "";
+ }
+ return response.query.pages[0].revisions[0].content;
+ } ).call( this );
}
suggestionSearch( query, limit = 'max' ) {
- return this.apiPromise.then( ( api ) => {
- return api.request( {
- action: 'opensearch',
- search: query,
- cirrusUseCompletionSuggester: 'yes',
- limit: limit
- } );
- } ).then(
- ( response ) => this.world.setApiResponse( response ),
- ( error ) => this.world.setApiError( error ) );
+ return Promise.coroutine( function* () {
+ let client = yield this.apiPromise;
+
+ try {
+ let response = yield client.request( {
+ action: 'opensearch',
+ search: query,
+ cirrusUseCompletionSuggester: 'yes',
+ limit: limit
+ } );
+ this.world.setApiResponse( response );
+ } catch ( err ) {
+ this.world.setApiError( error )
+ }
+ } ).call( this );
}
suggestionsWithProfile( query, profile ) {
- return this.apiPromise.then( ( api ) => {
- return api.request( {
- action: 'opensearch',
- search: query,
- profile: profile
- } );
- } ).then(
- ( response ) => this.world.setApiResponse( response ),
- ( error ) => this.world.setApiError( error ) );
+ return Promise.coroutine( function* () {
+ let client = yield this.apiPromise;
+
+ try {
+ let response = yield client.request( {
+ action: 'opensearch',
+ search: query,
+ profile: profile
+ } );
+ this.world.setApiResponse( response );
+ } catch ( err ) {
+ this.world.setApiError( err );
+ }
+ } ).call( this );
}
searchFor( query, options = {} ) {
- return this.apiPromise.then( ( api ) => {
- return api.request( Object.assign( options, {
- action: "query",
- list: "search",
- srsearch: query,
- srprop:
"snippet|titlesnippet|redirectsnippet|sectionsnippet|categorysnippet|isfilematch",
- formatversion: 2
- } ) );
- } ).then(
- ( response ) => this.world.setApiResponse( response ),
- ( error ) => this.world.setApiError( error ) );
+ return Promise.coroutine( function* () {
+ let client = yield this.apiPromise;
+
+ try {
+ let response = yield client.request(
Object.assign( options, {
+ action: "query",
+ list: "search",
+ srsearch: query,
+ srprop:
"snippet|titlesnippet|redirectsnippet|sectionsnippet|categorysnippet|isfilematch",
+ formatversion: 2
+ } ) );
+ this.world.setApiResponse( response );
+ } catch ( err ) {
+ this.world.setApiError( err );
+ }
+ } ).call( this );
}
waitForOperation( operation, title ) {
diff --git a/tests/integration/features/step_definitions/page_steps.js
b/tests/integration/features/step_definitions/page_steps.js
index 177fe24..d166ef6 100644
--- a/tests/integration/features/step_definitions/page_steps.js
+++ b/tests/integration/features/step_definitions/page_steps.js
@@ -13,12 +13,13 @@
SpecialVersion = require('../support/pages/special_version'),
ArticlePage = require('../support/pages/article_page'),
expect = require( 'chai' ).expect,
- querystring = require( 'querystring' );
+ querystring = require( 'querystring' ),
+ Promise = require( 'bluebird' );
// Attach extra information to assertion errors about what api call triggered
the problem
function withApi( world, fn ) {
try {
- return fn();
+ return fn.call( world );
} catch ( e ) {
let request = world.apiResponse ? world.apiResponse.__request :
world.apiError.request,
qs = Object.assign( {}, request.qs, request.form ),
@@ -34,7 +35,7 @@
defineSupportCode( function( {Given, When, Then} ) {
When( /^I go to (.*)$/, function ( title ) {
- this.visit( ArticlePage.title( title ) );
+ return this.visit( ArticlePage.title( title ) );
} );
When( /^I ask suggestion API for (.*)$/, function ( query ) {
@@ -50,31 +51,31 @@
} );
Then( /^the API should produce list containing (.*)/, function( term ) {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiResponse[ 1 ] ).to.include( term );
} );
} );
Then( /^the API should produce empty list/, function() {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiResponse[ 1 ] ).to.have.length( 0 );
} );
} );
Then( /^the API should produce list starting with (.*)/, function( term
) {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiResponse[ 1 ][ 0 ] ).to.equal( term );
} );
} );
Then( /^the API should produce list of length (\d+)/, function( length
) {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiResponse[ 1 ] ).to.have.length(
parseInt( length, 10 ) );
} );
} );
When( /^the api returns error code (.*)$/, function ( code ) {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiError ).to.include( {
code: code
} );
@@ -87,7 +88,7 @@
} );
Then( /^(.+) is the (.+) api suggestion$/, function ( title, position )
{
- withApi( this, () => {
+ return withApi( this, () => {
let pos = ['first', 'second', 'third', 'fourth',
'fifth', 'sixth', 'seventh', 'eigth', 'ninth', 'tenth'].indexOf( position );
if ( title === "none" ) {
if ( this.apiError && pos === 1 ) {
@@ -104,7 +105,7 @@
} );
Then( /^(.+) is( not)? in the api suggestions$/, function ( title,
should_not ) {
- withApi( this, () => {
+ return withApi( this, () => {
if ( should_not ) {
expect( this.apiResponse[1] ).to.not.include(
title );
} else {
@@ -114,7 +115,7 @@
} );
Then( /^the api should offer to search for pages containing (.+)$/,
function( term ) {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiResponse[0] ).to.equal( term );
} );
} );
@@ -151,12 +152,12 @@
}
}
Then( /^(.+) is( in)? the ((?:[^ ])+(?: or (?:[^ ])+)*) api search
result$/, function ( title, in_ok, indexes ) {
- withApi( this, () => {
- checkApiSearchResultStep.call( this, title, in_ok,
indexes );
+ return withApi( this, () => {
+ return checkApiSearchResultStep.call( this, title,
in_ok, indexes );
} );
} );
- function apiSearchStep( enableRewrites, qiprofile, offset, lang,
namespaces, search ) {
+ When( /^I api search( with rewrites enabled)?(?: with query independent
profile ([^ ]+))?(?: with offset (\d+))?(?: in the (.*) language)?(?: in
namespaces? (\d+(?: \d+)*))? for (.*)$/, function ( enableRewrites, qiprofile,
offset, lang, namespaces, search ) {
let options = {
srnamespace: (namespaces || "0").split(' '),
srenablerewrites: enableRewrites ? 1 : 0,
@@ -181,49 +182,47 @@
search = search.replace(/%\{\\i([\dA-Fa-f]{4,6})\}%/, ( match,
codepoint ) => JSON.parse( `"\\u${codepoint}"` ) );
return this.stepHelpers.searchFor( search, options );
- }
- When(/^I api search( with rewrites enabled)?(?: with query independent
profile ([^ ]+))?(?: with offset (\d+))?(?: in the (.*) language)?(?: in
namespaces? (\d+(?: \d+)*))? for (.*)$/, apiSearchStep );
+ } );
Then( /^within (\d+) seconds api searching for (.+) yields (.+) as the
(.+) result$/, function( seconds, query, title, indexes ) {
- let timeout = Date.now() + ( 1000 * seconds );
- let runSteps = ( resolve, reject ) => {
- apiSearchStep.call( this, undefined, undefined,
undefined, undefined, 0, query ).then( () => {
- checkApiSearchResultStep.call( this, title,
false, indexes );
- } ).then( resolve, ( error ) => {
- if ( Date.now() > timeout ) {
- console.log( 'within rejected due to
timeout' );
- reject( error );
+ return withApi( this, Promise.coroutine( function* () {
+ let timeout = Date.now() + ( 1000 * seconds );
+ while ( true ) {
+ yield apiSearchStep.call( this, undefined,
undefined, undefined, undefined, 0, query );
+ try {
+ checkApiSearchResultStep.call( this,
title, false, indexes )
+ return;
+ } catch ( err ) {
+ if ( Date.now() > timeout ) {
+ console.log( 'within rejected
due to timeout' );
+ throw err;
+ }
+ console.log( 're-running within' );
}
- console.log( 're-running within' );
- // Use process.nextTick to keep from exploding
the stack.
- process.nextTick( () => runSteps( resolve,
reject ) );
- } );
- };
- withApi( this, () => {
- return new Promise( runSteps );
- } );
+ }
+ } ) );
} );
Then( /there are no errors reported by the api/, function () {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiError ).to.be.undefined; // jshint
ignore:line
} );
} );
Then( /there is an api search result/, function () {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiResponse.query.search
).to.not.have.lengthOf( 0 );
} );
} );
Then( /there are no api search results/, function () {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiResponse.query.search
).to.have.lengthOf( 0 );
} );
} );
Then( /^(.+) is( not)? in the api search results$/, function( title,
not ) {
- withApi( this, () => {
+ return withApi( this, () => {
let titles = this.apiResponse.query.search.map( res =>
res.title );
if ( not ) {
expect( titles ).to.not.include( title );
@@ -234,7 +233,7 @@
} );
Then( /^this error is reported by api: (.+)$/, function (
expected_error ) {
- withApi( this, () => {
+ return withApi( this, () => {
expect( this.apiError.info ).to.equal(
expected_error.trim() );
} );
} );
diff --git a/tests/integration/features/suggest_api.feature
b/tests/integration/features/suggest_api.feature
deleted file mode 100644
index 0d70c7f..0000000
--- a/tests/integration/features/suggest_api.feature
+++ /dev/null
@@ -1,95 +0,0 @@
-#
-# This file is subject to the license terms in the COPYING file found in the
-# CirrusSearch top-level directory and at
-# https://phabricator.wikimedia.org/diffusion/ECIR/browse/master/COPYING. No
part of
-# CirrusSearch, including this file, may be copied, modified, propagated, or
-# distributed except according to the terms contained in the COPYING file.
-#
-# Copyright 2012-2014 by the Mediawiki developers. See the CREDITS file in the
-# CirrusSearch top-level directory and at
-# https://phabricator.wikimedia.org/diffusion/ECIR/browse/master/CREDITS
-#
-@api @suggest
-Feature: Suggestion API test
-
- Scenario: Search suggestions
- When I ask suggestion API for main
- Then the API should produce list containing Main Page
-
- Scenario: Created pages suggestions
- When I ask suggestion API for x-m
- Then the API should produce list containing X-Men
-
- Scenario: Nothing to suggest
- When I ask suggestion API for jabberwocky
- Then the API should produce empty list
-
- Scenario: Ordering
- When I ask suggestion API for x-m
- Then the API should produce list starting with X-Men
-
- Scenario: Fuzzy
- When I ask suggestion API for xmen
- Then the API should produce list starting with X-Men
-
- Scenario: Empty tokens
- When I ask suggestion API for はー
- Then the API should produce list starting with はーい
- And I ask suggestion API for はい
- Then the API should produce list starting with はーい
-
- Scenario Outline: Search redirects shows the best redirect
- When I ask suggestion API for <term>
- Then the API should produce list containing <suggested>
- Examples:
- | term | suggested |
- | eise | Eisenhardt, Max |
- | max | Max Eisenhardt |
- | magnetu | Magneto |
-
- Scenario Outline: Search prefers exact match over fuzzy match and ascii
folded
- When I ask suggestion API for <term>
- Then the API should produce list starting with <suggested>
- Examples:
- | term | suggested |
- | max | Max Eisenhardt |
- | mai | Main Page |
- | eis | Eisenhardt, Max |
- | ele | Elektra |
- | éle | Électricité |
-
- Scenario Outline: Search prefers exact db match over partial prefix match
- When I ask suggestion API at most 2 items for <term>
- Then the API should produce list starting with <first>
- And the API should produce list containing <other>
- Examples:
- | term | first | other |
- | Ic | Iceman | Ice |
- | Ice | Ice | Iceman |
-
- Scenario: Ordering & limit
- When I ask suggestion API at most 1 item for x-m
- Then the API should produce list starting with X-Men
- And the API should produce list of length 1
-
- Scenario Outline: Search fallback to prefix search if namespace is provided
- When I ask suggestion API for <term>
- Then the API should produce list starting with <suggested>
- Examples:
- | term | suggested |
- | Special: | Special:ActiveUsers |
- | Special:Act | Special:ActiveUsers |
-
- Scenario Outline: Search prefers main namespace over crossns redirects
- When I ask suggestion API for <term>
- Then the API should produce list starting with <suggested>
- Examples:
- | term | suggested |
- | V | Venom |
- | V: | V:N |
- | Z | Zam Wilson |
- | Z: | Z:Navigation |
-
- Scenario: Default sort can be used as search input
- When I ask suggestion API for Wilson
- Then the API should produce list starting with Sam Wilson
diff --git a/tests/integration/features/support/hooks.js
b/tests/integration/features/support/hooks.js
index e1e5082..f727418 100644
--- a/tests/integration/features/support/hooks.js
+++ b/tests/integration/features/support/hooks.js
@@ -1,92 +1,153 @@
/*jshint esversion: 6, node:true */
var {defineSupportCode} = require( 'cucumber' );
+var Promise = require( 'bluebird' );
defineSupportCode( function( { After, Before } ) {
let BeforeOnce = function ( options, fn ) {
- Before( options, function () {
- return this.tags.check( options.tags ).then( ( status )
=> {
- if ( status === 'new' ) {
- return fn.call ( this ).then( () =>
this.tags.complete( options.tags ) );
- }
- } );
- } );
+ Before( options, Promise.coroutine( function* () {
+ status = yield this.tags.check( options.tags );
+ if ( status === 'new' ) {
+ yield fn.call( this );
+ yield this.tags.complete( options.tags );
+ }
+ } ) );
};
- BeforeOnce( { tags: "@clean" }, function () {
- return this.stepHelpers.deletePage( "DeleteMeRedirect" );
- } );
+ let waitForBatch = Promise.coroutine( function* ( wiki, batchJobs ) {
+ let stepHelpers;
+ if ( batchJobs === undefined ) {
+ stepHelpers = this.stepHelpers;
+ batchJobs = wiki;
+ } else {
+ stepHelpers = this.stepHelpers.onWiki( wiki );
+ }
- Before( { tags: "@api" }, function () {
- return true;
- } );
-
- BeforeOnce( { tags: "@prefix" }, function () {
- console.log( 'starting prefix hook' );
- let batchJobs = {
- edit: {
- "L'Oréal": "L'Oréal",
- "Jean-Yves Le Drian": "Jean-Yves Le Drian"
+ if ( Array.isArray( batchJobs ) ) {
+ for ( let job of batchJobs ) {
+ yield stepHelpers.waitForOperation( job[0],
job[1] );
}
- };
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } );
- } );
- } );
-
- BeforeOnce( { tags: "@redirect" }, function () {
- let batchJobs = {
- edit: {
- "SEO Redirecttest": "#REDIRECT [[Search Engine
Optimization Redirecttest]]",
- "Redirecttest Yikes": "#REDIRECT [[Redirecttest
Yay]]",
- "User_talk:SEO Redirecttest": "#REDIRECT
[[User_talk:Search Engine Optimization Redirecttest]]",
- "Seo Redirecttest": "Seo Redirecttest",
- "Search Engine Optimization Redirecttest":
"Search Engine Optimization Redirecttest",
- "Redirecttest Yay": "Redirecttest Yay",
- "User_talk:Search Engine Optimization
Redirecttest": "User_talk:Search Engine Optimization Redirecttest",
- "PrefixRedirectRanking 1":
"PrefixRedirectRanking 1",
- "LinksToPrefixRedirectRanking 1":
"[[PrefixRedirectRanking 1]]",
- "TargetOfPrefixRedirectRanking 2":
"TargetOfPrefixRedirectRanking 2",
- "PrefixRedirectRanking 2": "#REDIRECT
[[TargetOfPrefixRedirectRanking 2]]"
+ } else {
+ for ( let operation in batchJobs ) {
+ let operationJobs = batchJobs[operation];
+ if ( Array.isArray( operationJobs ) ) {
+ for ( title of operationJobs ) {
+ yield
stepHelpers.waitForOperation( operation, title );
+ }
+ } else {
+ for ( title in operationJobs ) {
+ yield
stepHelpers.waitForOperation( operation, title );
+ }
+ }
}
- };
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } );
- } );
+ }
} );
- BeforeOnce( { tags: "@accent_squashing" }, function () {
- let batchJobs = {
- edit: {
- "Áccent Sorting": "Áccent Sorting",
- "Accent Sorting": "Accent Sorting"
- }
- };
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } );
- } );
+ let runBatchFn = ( wiki, batchJobs ) => Promise.coroutine( function* ()
{
+ let client;
+ if ( batchJobs === undefined ) {
+ batchJobs = wiki;
+ client = yield this.onWiki();
+ } else {
+ client = yield this.onWiki( wiki );
+ }
+
+ yield client.batch( batchJobs, 'CirrusSearch integration test
edit' );
+ yield waitForBatch.call( this, batchJobs );
} );
- BeforeOnce( { tags: "@accented_namespace" }, function () {
- let batchJobs = {
- edit: {
- "Mó:Test": "some text"
- }
- };
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } );
- } );
- } );
+ BeforeOnce( { tags: "@clean" }, runBatchFn( {
+ delete: [ 'DeleteMeRedirect' ]
+ } ) );
- BeforeOnce( { tags: "@suggest" }, function () {
+ BeforeOnce( { tags: "@prefix" }, runBatchFn( {
+ edit: {
+ "L'Oréal": "L'Oréal",
+ "Jean-Yves Le Drian": "Jean-Yves Le Drian"
+ }
+ } ) );
+
+ BeforeOnce( { tags: "@redirect", timeout: 60000 }, runBatchFn( {
+ edit: {
+ "SEO Redirecttest": "#REDIRECT [[Search Engine
Optimization Redirecttest]]",
+ "Redirecttest Yikes": "#REDIRECT [[Redirecttest Yay]]",
+ "User_talk:SEO Redirecttest": "#REDIRECT
[[User_talk:Search Engine Optimization Redirecttest]]",
+ "Seo Redirecttest": "Seo Redirecttest",
+ "Search Engine Optimization Redirecttest": "Search
Engine Optimization Redirecttest",
+ "Redirecttest Yay": "Redirecttest Yay",
+ "User_talk:Search Engine Optimization Redirecttest":
"User_talk:Search Engine Optimization Redirecttest",
+ "PrefixRedirectRanking 1": "PrefixRedirectRanking 1",
+ "LinksToPrefixRedirectRanking 1":
"[[PrefixRedirectRanking 1]]",
+ "TargetOfPrefixRedirectRanking 2":
"TargetOfPrefixRedirectRanking 2",
+ "PrefixRedirectRanking 2": "#REDIRECT
[[TargetOfPrefixRedirectRanking 2]]"
+ }
+ } ) );
+
+ BeforeOnce( { tags: "@accent_squashing" }, runBatchFn( {
+ edit: {
+ "Áccent Sorting": "Áccent Sorting",
+ "Accent Sorting": "Accent Sorting"
+ }
+ } ) );
+
+ BeforeOnce( { tags: "@accented_namespace" }, runBatchFn( {
+ edit: {
+ "Mó:Test": "some text"
+ }
+ } ) );
+
+ BeforeOnce( { tags: "@setup_main or @filters or @prefix or @bad_syntax
or @wildcard or @exact_quotes or @phrase_prefix", timeout: 60000 }, runBatchFn(
{
+ edit: {
+ "Template:Template Test": "pickles
[[Category:TemplateTagged]]",
+ "Catapult/adsf": "catapult subpage [[Catapult]]",
+ "Links To Catapult": "[[Catapult]]",
+ "Catapult": "♙ asdf [[Category:Weaponry]]",
+ "Amazing Catapult": "test [[Catapult]]
[[Category:Weaponry]]",
+ "Category:Weaponry": "Weaponry refers to any items
designed or used to attack and kill or destroy other people and property.",
+ "Two Words": "ffnonesenseword catapult
{{Template_Test}} anotherword [[Category:TwoWords]] [[Category:Categorywith
Twowords]] [[Category:Categorywith \" Quote]]",
+ "AlphaBeta": "[[Category:Alpha]] [[Category:Beta]]",
+ "IHaveATwoWordCategory": "[[Category:CategoryWith
ASpace]]",
+ "Functional programming": "Functional programming is
referential transparency.",
+ "वाङ्मय": "वाङ्मय",
+ "वाङ्\u200dमय": "वाङ्\u200dमय",
+ "वाङ्\u200cमय": "वाङ्\u200cमय",
+ "ChangeMe": "foo",
+ "Wikitext": "{{#tag:somebug}}",
+ "Page with non ascii letters": "ἄνθρωπος, широкий"
+ }
+ } ) );
+
+ BeforeOnce( { tags: "@setup_main or @prefix or @bad_syntax" },
runBatchFn( {
+ // TODO: File upload
+ // And a file named File:Savepage-greyed.png exists with
contents Savepage-greyed.png and description Screenshot, for test purposes,
associated with https://bugzilla.wikimedia.org/show_bug.cgi?id=52908 .
+ edit: {
+ "Rdir": "#REDIRECT [[Two Words]]",
+ "IHaveAVideo": "[[File:How to Edit Article in Arabic
Wikipedia.ogg|thumb|267x267px]]",
+ "IHaveASound": "[[File:Serenade for Strings -mvt-1-
Elgar.ogg]]"
+ }
+ } ) );
+
+ BeforeOnce( { tags: "@setup_main or @prefix or @go or @bad_syntax" },
runBatchFn( {
+ edit: {
+ "África": "for testing"
+ }
+ } ) );
+
+ BeforeOnce( { tags: "@boost_template" }, runBatchFn( {
+ edit: {
+ "Template:BoostTemplateHigh": "BoostTemplateTest",
+ "Template:BoostTemplateLow": "BoostTemplateTest",
+ "NoTemplates BoostTemplateTest": "nothing important",
+ "HighTemplate": "{{BoostTemplateHigh}}",
+ "LowTemplate": "{{BoostTemplateLow}}",
+ }
+ } ) );
+
+ // This needs to be the *last* hook added. That gives us some hope that
everything
+ // else is inside elasticsearch by the time cirrus-suggest-index runs
and builds
+ // the completion suggester
+ BeforeOnce( { tags: "@suggest", timeout: 60000 }, Promise.coroutine(
function* () {
+ let client = yield this.onWiki();
let batchJobs = {
edit: {
"X-Men": "The X-Men are a fictional team of
superheroes",
@@ -94,7 +155,7 @@
"X-Force": "X-Force is a fictional team of of
[[X-Men]]",
"Magneto": "Magneto is a fictional character
appearing in American comic books",
"Max Eisenhardt": "#REDIRECT [[Magneto]]",
- "Eisenhardt: Max": "#REDIRECT [[Magneto]]",
+ "Eisenhardt, Max": "#REDIRECT [[Magneto]]",
"Magnetu": "#REDIRECT [[Magneto]]",
"Ice": "It's cold.",
"Iceman": "Iceman (Robert \"Bobby\" Drake) is a
fictional superhero appearing in American comic books published by Marvel
Comics and is...",
@@ -114,89 +175,11 @@
"はーい": "makes sure we do not fail to index
empty tokens (T156234)"
}
};
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } ).then( () => {
- return api.request( {
- action: 'cirrus-suggest-index'
- } );
- } );
+ yield client.batch( batchJobs );
+ yield waitForBatch.call( this, batchJobs );
+ yield client.request( {
+ action: 'cirrus-suggest-index'
} );
- } );
+ } ) );
- BeforeOnce( { tags: "@setup_main or @filters or @prefix or @bad_syntax
or @wildcard or @exact_quotes or @phrase_prefix" }, function () {
- let batchJobs = {
- edit: {
- "Template:Template Test": "pickles
[[Category:TemplateTagged]]",
- "Catapult/adsf": "catapult subpage
[[Catapult]]",
- "Links To Catapult": "[[Catapult]]",
- "Catapult": "♙ asdf [[Category:Weaponry]]",
- "Amazing Catapult": "test [[Catapult]]
[[Category:Weaponry]]",
- "Category:Weaponry": "Weaponry refers to any
items designed or used to attack and kill or destroy other people and
property.",
- "Two Words": "ffnonesenseword catapult
{{Template_Test}} anotherword [[Category:TwoWords]] [[Category:Categorywith
Twowords]] [[Category:Categorywith \" Quote]]",
- "AlphaBeta": "[[Category:Alpha]]
[[Category:Beta]]",
- "IHaveATwoWordCategory":
"[[Category:CategoryWith ASpace]]",
- "Functional programming": "Functional
programming is referential transparency.",
- "वाङ्मय": "वाङ्मय",
- "वाङ्\u200dमय": "वाङ्\u200dमय",
- "वाङ्\u200cमय": "वाङ्\u200cमय",
- "ChangeMe": "foo",
- "Wikitext": "{{#tag:somebug}}",
- "Page with non ascii letters": "ἄνθρωπος,
широкий"
- }
- };
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } );
- } );
- } );
-
- BeforeOnce( { tags: "@setup_main or @prefix or @bad_syntax" }, function
() {
- // TODO: File upload
- // And a file named File:Savepage-greyed.png exists with
contents Savepage-greyed.png and description Screenshot, for test purposes,
associated with https://bugzilla.wikimedia.org/show_bug.cgi?id=52908 .
- let batchJobs = {
- edit: {
- "Rdir": "#REDIRECT [[Two Words]]",
- "IHaveAVideo": "[[File:How to Edit Article in
Arabic Wikipedia.ogg|thumb|267x267px]]",
- "IHaveASound": "[[File:Serenade for Strings
-mvt-1- Elgar.ogg]]"
- }
- };
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } );
- } );
- } );
-
- BeforeOnce( { tags: "@setup_main or @prefix or @go or @bad_syntax" },
function () {
- let batchJobs = {
- edit: {
- "África": "for testing"
- }
- };
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } );
- } );
- } );
-
- BeforeOnce( { tags: "@boost_template" }, function () {
- let batchJobs = {
- edit: {
- "Template:BoostTemplateHigh":
"BoostTemplateTest",
- "Template:BoostTemplateLow":
"BoostTemplateTest",
- "NoTemplates BoostTemplateTest": "nothing
important",
- "HighTemplate": "{{BoostTemplateHigh}}",
- "LowTemplate": "{{BoostTemplateLow}}",
- }
- };
- return this.onWiki().then( ( api ) => {
- return api.loginGetEditToken().then( () => {
- return api.batch(batchJobs, 'CirrusSearch
integration test edit');
- } );
- } );
- } );
} );
diff --git a/tests/integration/features/support/world.js
b/tests/integration/features/support/world.js
index b405433..7e8c9c2 100644
--- a/tests/integration/features/support/world.js
+++ b/tests/integration/features/support/world.js
@@ -14,7 +14,8 @@
net = require( 'net' ),
Bot = require( 'mwbot' ),
StepHelpers = require( '../step_definitions/page_step_helpers' ),
- Page = require( './pages/page' );
+ Page = require( './pages/page' ),
+ Promise = require( 'bluebird' );
// Client for the Server implemented in lib/tracker.js. The server
// tracks what tags have already been initialized so we don't have
@@ -47,15 +48,16 @@
}
check( tag ) {
- if ( this.tags[tag] ) {
- return Promise.resolve( 'complete' );
- }
- return this.request( {
- check: tag
- } ).then( ( response ) => {
+ return Promise.coroutine( function* () {
+ if ( this.tags[tag] ) {
+ return 'complete';
+ }
+ let response = yield this.request( {
+ check: tag
+ } );
this.tags[tag] = true;
return response.status;
- } );
+ } ).call( this );
}
complete( tag ) {
@@ -66,6 +68,9 @@
}
let tagClient = new TagClient( browser.options.trackerPath );
+// world gets re-created all the time. Try and save some time logging
+// in by sharing api clients
+let apiClients = {};
function World( { attach, parameters } ) {
// default properties
@@ -77,8 +82,8 @@
// (I have a feeling this is prone to race conditions).
// By suggestion of this stack overflow question.
//
https://stackoverflow.com/questions/26372724/pass-variables-between-step-definitions-in-cucumber-groovy
- this.apiResponse = "";
- this.apiError = "";
+ this.apiResponse = undefined;
+ this.apiError = undefined;
this.setApiResponse = function( value ) {
this.apiResponse = value;
@@ -97,6 +102,10 @@
// Per-wiki api clients
this.onWiki = function( wiki = this.config.wikis.default ) {
+ if ( apiClients[wiki] ) {
+ return apiClients[wiki];
+ }
+
let w = this.config.wikis[ wiki ];
let client = new Bot();
client.setOptions({
@@ -106,14 +115,6 @@
concurrency: 1,
apiUrl: w.apiUrl
});
- let origLoginGetEditToken = client.loginGetEditToken;
- client.loginGetEditToken = function () {
- return origLoginGetEditToken.call( client, {
- username: w.username,
- password: w.password,
- apiUrl: w.apiUrl
- } );
- };
// Add a generic method to get access to the request that
triggered a response, so we
// can add generic error reporting that includes the requested
api url
@@ -125,8 +126,16 @@
} );
};
- // TODO: Why a promise? I guess it's just easier to chain...
- return Promise.resolve( client );
+ apiClients[wiki] = client.loginGetEditToken( {
+ username: w.username,
+ password: w.password,
+ apiUrl: w.apiUrl
+ } ).then( () => client );
+
+ // Catch anything trying to re-login and break everything
+ client.loginGetEditToken = undefined;
+
+ return apiClients[wiki];
};
// Binding step helpers to this World.
--
To view, visit https://gerrit.wikimedia.org/r/392470
To unsubscribe, visit https://gerrit.wikimedia.org/r/settings
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ie2f3142d8af9036a6a6e473a2a7d2fd557abeaca
Gerrit-PatchSet: 1
Gerrit-Project: mediawiki/extensions/CirrusSearch
Gerrit-Branch: master
Gerrit-Owner: EBernhardson <[email protected]>
_______________________________________________
MediaWiki-commits mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/mediawiki-commits
