Yep; they'll be documented on Meta in the same way all research projects are.

On 20 August 2015 at 13:06, Tomasz Finc <tf...@wikimedia.org> wrote:
> thanks for the quick summary Oliver. Let's get these tests archived no
> the wiki so that at the end of the quarter we aren't having to crawl
> through mailing list archives
>
> On Thu, Aug 20, 2015 at 9:07 AM, Oliver Keyes <oke...@wikimedia.org> wrote:
>> Dear all,
>>
>> Last week we ran an A/B test over changes to how our search system
>> provides results (and we look forward to sharing the results of that
>> with you shortly). Today, we're launching a second A/B test.
>>
>> This test looks at the "phrase slop" setting within our search
>> infrastructure, a thoroughly disgusting term that simply refers to the
>> "distance" in words between a search query and a match. For example,
>> the search term "Ben Folds Five" has a "phrase slop" of 0 to the match
>> "Ben Folds Five". The search term "Ben Five" has a "phrase slop" of 1
>> - there is 1 word (Folds) in distance between the search term and the
>> match.
>>
>> What we are going to do is experiment with altering the phrase slop
>> settings for 3% of users. 1% will have a slop of 0 (the current
>> default), 1% a slop of 1, and 1% a slop of 2. The hope is that by
>> tweaking this setting we can measurably reduce the number of search
>> queries that return 0 results by broadening the conditions under which
>> something is considered a match.
>>
>> This search will run for a week, and kick off at 4pm PST. Once it's
>> completed we'll share the results publicly, as is the norm for our A/B
>> tests. Huge thanks go particularly to Trey Jones and David Causse for
>> their feedback on what settings we could alter, Mikhail Popov for his
>> work on experimental design, and Erik Bernhardson for both his
>> feedback and turning around necessary changes to the search
>> infrastructure on such short notice. For context, we began planning
>> this test last Thursday morning - a one-week turnaround on design and
>> implementation is phenomenal.
>>
>> Thanks,
>>
>> --
>> Oliver Keyes
>> Count Logula
>> Wikimedia Foundation
>>
>> _______________________________________________
>> Wikimedia-search mailing list
>> Wikimedia-search@lists.wikimedia.org
>> https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
>
> _______________________________________________
> Wikimedia-search mailing list
> Wikimedia-search@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikimedia-search



-- 
Oliver Keyes
Count Logula
Wikimedia Foundation

_______________________________________________
Wikimedia-search mailing list
Wikimedia-search@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimedia-search

Reply via email to