Re: [Wikimedia-search] On frequency of A/B tests and peeking at the data early

2015-09-01 Thread Tomasz Finc
On Mon, Aug 31, 2015 at 11:05 AM, Mikhail Popov wrote: > Evan Miller, How Not To Run An A/B Test If you guys ever want me to get Evan to come talk to us just let know. We both worked at A9 together. He's a fun guy. --tomasz ___ Wikimedia-search maili

Re: [Wikimedia-search] On frequency of A/B tests and peeking at the data early

2015-09-01 Thread Trey Jones
Well, peeking is okay as long as you don't act on it: “Peeking” at the data is OK as long as you can restrain yourself from > stopping an experiment before it has run its course. I know this goes > against something in human nature, so perhaps the best advice is: no > peeking! It does take up ti

Re: [Wikimedia-search] On frequency of A/B tests and peeking at the data early

2015-09-01 Thread Kevin Smith
We will, of course, continue to sanity-check the data within a day or so after a new test starts to run, to make sure that we are logging the information that we will need to perform analyses, that our bucket sizes appear to working as designed, etc. Kevin Smith Agile Coach, Wikimedia Foundation

[Wikimedia-search] FYI: Upcoming UX sub-team meetings

2015-09-01 Thread Kevin Smith
Just as an FYI, next Thursday the Discovery's UX sub-team will start having weekly meetings, to groom the backlog and plan work for the week. For now, these will include Moiz and Dan, with Tomasz and Wes optional. As additional UX folks are hired, we'll add them, and we will also consider bringing

[Wikimedia-search] Fwd: Discovery Department A/B testing an alternative to prefix search next week

2015-09-01 Thread Dan Garry
Cross-posting from wikitech-l. Please reply there. -- Forwarded message -- From: Dan Garry Date: 1 September 2015 at 20:43 Subject: Discovery Department A/B testing an alternative to prefix search next week To: Wikimedia developers Hi everyone, *tl;dr: Discovery Department to