Yep; they'll be documented on Meta in the same way all research projects are.

On 20 August 2015 at 13:06, Tomasz Finc <> wrote:
> thanks for the quick summary Oliver. Let's get these tests archived no
> the wiki so that at the end of the quarter we aren't having to crawl
> through mailing list archives
> On Thu, Aug 20, 2015 at 9:07 AM, Oliver Keyes <> wrote:
>> Dear all,
>> Last week we ran an A/B test over changes to how our search system
>> provides results (and we look forward to sharing the results of that
>> with you shortly). Today, we're launching a second A/B test.
>> This test looks at the "phrase slop" setting within our search
>> infrastructure, a thoroughly disgusting term that simply refers to the
>> "distance" in words between a search query and a match. For example,
>> the search term "Ben Folds Five" has a "phrase slop" of 0 to the match
>> "Ben Folds Five". The search term "Ben Five" has a "phrase slop" of 1
>> - there is 1 word (Folds) in distance between the search term and the
>> match.
>> What we are going to do is experiment with altering the phrase slop
>> settings for 3% of users. 1% will have a slop of 0 (the current
>> default), 1% a slop of 1, and 1% a slop of 2. The hope is that by
>> tweaking this setting we can measurably reduce the number of search
>> queries that return 0 results by broadening the conditions under which
>> something is considered a match.
>> This search will run for a week, and kick off at 4pm PST. Once it's
>> completed we'll share the results publicly, as is the norm for our A/B
>> tests. Huge thanks go particularly to Trey Jones and David Causse for
>> their feedback on what settings we could alter, Mikhail Popov for his
>> work on experimental design, and Erik Bernhardson for both his
>> feedback and turning around necessary changes to the search
>> infrastructure on such short notice. For context, we began planning
>> this test last Thursday morning - a one-week turnaround on design and
>> implementation is phenomenal.
>> Thanks,
>> --
>> Oliver Keyes
>> Count Logula
>> Wikimedia Foundation
>> _______________________________________________
>> Wikimedia-search mailing list
> _______________________________________________
> Wikimedia-search mailing list

Oliver Keyes
Count Logula
Wikimedia Foundation

Wikimedia-search mailing list

Reply via email to