thanks for the quick summary Oliver. Let's get these tests archived no the wiki so that at the end of the quarter we aren't having to crawl through mailing list archives
On Thu, Aug 20, 2015 at 9:07 AM, Oliver Keyes <oke...@wikimedia.org> wrote: > Dear all, > > Last week we ran an A/B test over changes to how our search system > provides results (and we look forward to sharing the results of that > with you shortly). Today, we're launching a second A/B test. > > This test looks at the "phrase slop" setting within our search > infrastructure, a thoroughly disgusting term that simply refers to the > "distance" in words between a search query and a match. For example, > the search term "Ben Folds Five" has a "phrase slop" of 0 to the match > "Ben Folds Five". The search term "Ben Five" has a "phrase slop" of 1 > - there is 1 word (Folds) in distance between the search term and the > match. > > What we are going to do is experiment with altering the phrase slop > settings for 3% of users. 1% will have a slop of 0 (the current > default), 1% a slop of 1, and 1% a slop of 2. The hope is that by > tweaking this setting we can measurably reduce the number of search > queries that return 0 results by broadening the conditions under which > something is considered a match. > > This search will run for a week, and kick off at 4pm PST. Once it's > completed we'll share the results publicly, as is the norm for our A/B > tests. Huge thanks go particularly to Trey Jones and David Causse for > their feedback on what settings we could alter, Mikhail Popov for his > work on experimental design, and Erik Bernhardson for both his > feedback and turning around necessary changes to the search > infrastructure on such short notice. For context, we began planning > this test last Thursday morning - a one-week turnaround on design and > implementation is phenomenal. > > Thanks, > > -- > Oliver Keyes > Count Logula > Wikimedia Foundation > > _______________________________________________ > Wikimedia-search mailing list > Wikimedia-search@lists.wikimedia.org > https://lists.wikimedia.org/mailman/listinfo/wikimedia-search _______________________________________________ Wikimedia-search mailing list Wikimedia-search@lists.wikimedia.org https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
