I've very excited about this. Where can we see updated search metrics that don't require manual reports? I want to make sure that everyone can see these.
--tomasz On Fri, Dec 5, 2014 at 1:19 PM, Dan Garry <[email protected]> wrote: > Hi everyone, > > tl;dr: The Mobile Apps Team is going to trial seamlessly surfacing full text > search results instead of prefixsearch results in cases where prefixsearch > does not give satisfactory results. > > The Mobile Apps Team has received a lot of feedback that our search feature > isn't the best. Our latest metrics confirm this; around 19% of queries give > the user no results. Our working hypothesis is that this is because we use > prefixsearch on article titles, which is very insensitive to typos and > free-form text. > > To help with this, we implemented full-text search. The user has two options > for searching, prefixsearch and full text. You can see this example to see > what these options look like. > > However, the way we present the two options to users is suboptimal. There's > no clear mental model for when one should be used compared to the other. The > design team recommended that we simply present whichever result set is > better for any given query. But how do we decide which result set is better? > To validate this course of action, we audited which of the two options, > prefixsearch or full text search, was better for a set of queries. The > results are here. > > The takehomes of our audit: > > In cases where there are very few prefixsearch results (less than around 5), > the full text results are just as good or better than the prefixsearch > results. Often, this is because the "did you mean" functionality of the full > text API helps the user out. > In cases where there are a good number of prefixsearch results, the > prefixsearch results tend to be better than the fulltext results. > > Here's what we're going to try: > > Remove the buttons from the UI. > By default, use prefixsearch for searches. > If there are fewer than 5 prefixsearch results, show fulltext search results > instead. > > Metrics for success: > > Higher search clickthrough (users finding what they need more) > Fewer number of queries give 0 results (users served more results) > Fewer number of queries per search session (users finding what they need > faster) > > The advantage of this experiment is that it's safe to fail: there is no > actual UX change, so if we decide our solution isn't good enough, then we > can rollout the fallback of surfacing the buttons without users thinking > we're just endlessly tweaking the UI. > > Please do get in touch if you have any questions! > > Thanks, > Dan > > -- > Dan Garry > Associate Product Manager, Mobile Apps > Wikimedia Foundation > > _______________________________________________ > Mobile-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/mobile-l > _______________________________________________ Mobile-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mobile-l
