Great explanation. Thanks, Dan! On Fri, Feb 19, 2016 at 5:19 PM, Dan Garry <[email protected]> wrote:
> On 19 February 2016 at 17:13, Jon Katz <[email protected]> wrote: >> >> Thanks, Erik! This is very helpful. What do you mean by 'back testing'? >> >> > > For search, there's a few different approaches for quantitative testing > that are less difficult than A/B testing in terms of development overhead, > data analysis and coordination. One of those is to replay real user queries > against the index, but run the query with slightly different parameters > from original. This is super cheap compared to an A/B test, but the > downside is that it can only answer really deterministic (for lack of a > better word) things, like how the parameters affect the zero-results rate > or result ordering; since there's no user interaction with the replayed > queries, you don't know what the clickthrough would've been, so it's hard > to measure how satisfied the user would've been. > > Hopefully that helps explain it. > > Thanks, > Dan > > -- > Dan Garry > Lead Product Manager, Discovery > Wikimedia Foundation >
_______________________________________________ Mobile-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mobile-l
