Jon, Are there any technical reasons not to explore the % rollout model for collections instead of beta? -J
On Tue, Jan 27, 2015 at 11:20 AM, Maryana Pinchuk <[email protected]> wrote: > Yeah, I think there are a lot of problems with the opt-in beta model. I > much prefer releasing new features to a small % of users, logging > events/usage, and if we suspect something has the potential to be > disruptive/offputting, letting them know the feature they're seeing is beta > and letting them turn it off. > > That said, beta is still useful for sandboxing new features and in-person > user testing, so I don't think we should kill it altogether. I just think > we need to supplement it with a graduated release model – which we're > already doing with stuff like WikiGrok :) > > On Tue, Jan 27, 2015 at 11:10 AM, Sam Smith <[email protected]> > wrote: > >> Have we thought about automatically opting people into beta mode e.g. >>> a sample of our users in a certain geographic region / certain zero >>> enabled area/ all users in a certain bucket based on their user id ? >> >> >> I like this idea. In fact, I'm for it, provided that we make it clear to >> the user that they've been entered into an experiment and they're seeing >> non-standard UI. >> >> How many users could beta actually handle? >> >> >> Not sure. But, interestingly, we can find out by bucketing users and >> slowly assigning them the beta variant. >> >> Is this technically possible? >> >> >> Yes. If we're generating and storing tokens on the client, which we do >> for anonymous users in other experiments, then we can enter anonymous users >> into the experiment at the cost of a little control over how tokens are >> stored. >> >> If someone was bucketed into beta would they be able to opt out into >>> stable again under any of the above situations? >> >> >> See my first inline response. We must make it clear to the user that >> they're seeing a variant of an experiment… and we must make it simple to >> opt out of the experiment. >> >> Also, all instrumentation for beta features will need to be augmented >> with a is_beta_opt_in flag. >> >> –Sam >> >> On Tue, Jan 27, 2015 at 10:18 AM, Jon Robson <[email protected]> >> wrote: >> >>> One of the frustrations I have heard so far is that the audience there >>> is too small to get meaningful data around various experiments. >>> Currently people have to opt in by going to >>> https://en.m.wikipedia.org/wiki/Special:MobileOptions which is hidden >>> away in the mobile interface. They can do this whilst anonymous or >>> logged in. >>> >>> Have we thought about automatically opting people into beta mode e.g. >>> a sample of our users in a certain geographic region / certain zero >>> enabled area/ all users in a certain bucket based on their user id ? >>> >>> How many users could beta actually handle? >>> Is this technically possible? >>> If someone was bucketed into beta would they be able to opt out into >>> stable again under any of the above situations? >>> >>> Jon >>> >>> _______________________________________________ >>> Mobile-l mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/mobile-l >>> >> >> >> _______________________________________________ >> Mobile-l mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/mobile-l >> >> > > > -- > Maryana Pinchuk > Product Manager, Wikimedia Foundation > wikimediafoundation.org > > _______________________________________________ > Mobile-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/mobile-l > >
_______________________________________________ Mobile-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mobile-l
