David or Erik can correct me if need be, but I think Common Terms is going to improve results quality more than ZRR.
Also, we don't have anything that quite lives up to the label of "system" for re-running queries. We have "methods"—hacks, scripts, and other non-systematic ways of achieving that goal. [Insert plug for Relevance Lab here.] Trey Jones Software Engineer, Discovery Wikimedia Foundation On Mon, Oct 5, 2015 at 12:18 PM, Mikhail Popov <[email protected]> wrote: > 1. Is the common terms query work supposed to address zero results rate > or…? Don't we have a system now for running old queries through the engine > and seeing whether the change affected the ZRR? If the test is not > dependent on the user's actions, we don't need to do an A/B test that > involves users. > > 2. We finally have a couple of ways of assessing user engagement with our > search results (more on that in an announcement later today). If this work > is meant to address that, then we need to run an A/B test, but include the > TestSatisfaction2 schema (or a variant of it that also tracks the grouping). > > On Mon, Oct 5, 2015 at 8:59 AM, Erik Bernhardson < > [email protected]> wrote: > >> David's work on the common terms query is merged and will be deploying >> this week. It's hidden behind a feature flag so no user visible change >> will occur. >> >> I thought we had decided on an A/B test to check the usefulness of this >> approach but I can't find anything in phab. So the question is, how do we >> want to AB test this? It will be easiest to run this test through our >> backend AB testing infrastructure, i think we just need to choose a >> sampling rate. >> >> _______________________________________________ >> discovery mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/discovery >> >> > > > -- > *Mikhail Popov* // Data Analyst, Discovery > <https://www.mediawiki.org/wiki/Wikimedia_Discovery> > https://wikimediafoundation.org/ > > *Imagine a world in which every single human being can freely share in the > **sum of all knowledge. That's our commitment.* Donate > <https://donate.wikimedia.org/>. > > _______________________________________________ > discovery mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/discovery > >
_______________________________________________ discovery mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/discovery
