Yep; they'll be documented on Meta in the same way all research projects are.
On 20 August 2015 at 13:06, Tomasz Finc <tf...@wikimedia.org> wrote: > thanks for the quick summary Oliver. Let's get these tests archived no > the wiki so that at the end of the quarter we aren't having to crawl > through mailing list archives > > On Thu, Aug 20, 2015 at 9:07 AM, Oliver Keyes <oke...@wikimedia.org> wrote: >> Dear all, >> >> Last week we ran an A/B test over changes to how our search system >> provides results (and we look forward to sharing the results of that >> with you shortly). Today, we're launching a second A/B test. >> >> This test looks at the "phrase slop" setting within our search >> infrastructure, a thoroughly disgusting term that simply refers to the >> "distance" in words between a search query and a match. For example, >> the search term "Ben Folds Five" has a "phrase slop" of 0 to the match >> "Ben Folds Five". The search term "Ben Five" has a "phrase slop" of 1 >> - there is 1 word (Folds) in distance between the search term and the >> match. >> >> What we are going to do is experiment with altering the phrase slop >> settings for 3% of users. 1% will have a slop of 0 (the current >> default), 1% a slop of 1, and 1% a slop of 2. The hope is that by >> tweaking this setting we can measurably reduce the number of search >> queries that return 0 results by broadening the conditions under which >> something is considered a match. >> >> This search will run for a week, and kick off at 4pm PST. Once it's >> completed we'll share the results publicly, as is the norm for our A/B >> tests. Huge thanks go particularly to Trey Jones and David Causse for >> their feedback on what settings we could alter, Mikhail Popov for his >> work on experimental design, and Erik Bernhardson for both his >> feedback and turning around necessary changes to the search >> infrastructure on such short notice. For context, we began planning >> this test last Thursday morning - a one-week turnaround on design and >> implementation is phenomenal. >> >> Thanks, >> >> -- >> Oliver Keyes >> Count Logula >> Wikimedia Foundation >> >> _______________________________________________ >> Wikimedia-search mailing list >> Wikimediafirstname.lastname@example.org >> https://lists.wikimedia.org/mailman/listinfo/wikimedia-search > > _______________________________________________ > Wikimedia-search mailing list > Wikimediaemail@example.com > https://lists.wikimedia.org/mailman/listinfo/wikimedia-search -- Oliver Keyes Count Logula Wikimedia Foundation _______________________________________________ Wikimedia-search mailing list Wikimediafirstname.lastname@example.org https://lists.wikimedia.org/mailman/listinfo/wikimedia-search