Yep; they'll be documented on Meta in the same way all research projects are.
On 20 August 2015 at 13:06, Tomasz Finc tf...@wikimedia.org wrote:
thanks for the quick summary Oliver. Let's get these tests archived no
the wiki so that at the end of the quarter we aren't having to crawl
through mailing list archives
On Thu, Aug 20, 2015 at 9:07 AM, Oliver Keyes oke...@wikimedia.org wrote:
Dear all,
Last week we ran an A/B test over changes to how our search system
provides results (and we look forward to sharing the results of that
with you shortly). Today, we're launching a second A/B test.
This test looks at the phrase slop setting within our search
infrastructure, a thoroughly disgusting term that simply refers to the
distance in words between a search query and a match. For example,
the search term Ben Folds Five has a phrase slop of 0 to the match
Ben Folds Five. The search term Ben Five has a phrase slop of 1
- there is 1 word (Folds) in distance between the search term and the
match.
What we are going to do is experiment with altering the phrase slop
settings for 3% of users. 1% will have a slop of 0 (the current
default), 1% a slop of 1, and 1% a slop of 2. The hope is that by
tweaking this setting we can measurably reduce the number of search
queries that return 0 results by broadening the conditions under which
something is considered a match.
This search will run for a week, and kick off at 4pm PST. Once it's
completed we'll share the results publicly, as is the norm for our A/B
tests. Huge thanks go particularly to Trey Jones and David Causse for
their feedback on what settings we could alter, Mikhail Popov for his
work on experimental design, and Erik Bernhardson for both his
feedback and turning around necessary changes to the search
infrastructure on such short notice. For context, we began planning
this test last Thursday morning - a one-week turnaround on design and
implementation is phenomenal.
Thanks,
--
Oliver Keyes
Count Logula
Wikimedia Foundation
___
Wikimedia-search mailing list
Wikimedia-search@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
___
Wikimedia-search mailing list
Wikimedia-search@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
--
Oliver Keyes
Count Logula
Wikimedia Foundation
___
Wikimedia-search mailing list
Wikimedia-search@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikimedia-search
