The results of the A/B test have now been published: https://meta.wikimedia.org/wiki/Research:Reducing_Zero_Result_Searches_Through_Confidence_Changes
In summary, there was a statistically significant difference, with the original configuration being better than the new configuration we tried. However, practically speaking, that change was so small as to be basically irrelevant, so we intend to stick with the original configuration. Thanks, Dan On 7 August 2015 at 13:32, Pine W <[email protected]> wrote: > Can we call you "Discovery Dan"? ;) > > Thanks for working on improving search results. > > Pine > > > On Fri, Aug 7, 2015 at 1:19 PM, Dan Garry <[email protected]> wrote: > > > Hello! > > > > As part of our goal to reduce the zero results rate > > < > > > https://www.mediawiki.org/wiki/Wikimedia_Engineering/2015-16_Q1_Goals#Search > > >, > > the Discovery Department is currently running an A/B test to try > different > > parameters for the search suggester. We're hoping that our new parameters > > will give users more suggestions without decreasing their quality. > > > > The reason we've chosen to tweak the suggestions is because of our recent > > work <https://phabricator.wikimedia.org/T105202> to automatically run > > queries for the user if they get zero results but have a suggestion. The > > purpose of this A/B test is to determine whether this has significant > > impact towards achieving our goal or not. > > > > This is the first A/B test that the Discovery Department has run, so > we're > > still ironing out the process. We hope to run many more A/B tests in the > > future. > > > > For further information on this, please review the associated Phabricator > > task <https://phabricator.wikimedia.org/T108103>. > > > > If you have any questions, I'd be happy to answer them. > > > > Thanks, > > Dan > > > > -- > > Dan Garry > > Lead Product Manager, Discovery > > Wikimedia Foundation > > _______________________________________________ > > Wikitech-l mailing list > > [email protected] > > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > _______________________________________________ > Wikitech-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikitech-l -- Dan Garry Lead Product Manager, Discovery Wikimedia Foundation _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
