Forwarding email from Ravi - while he is waiting for permission to post to list.
BR Wilfred --- FxOS Product Management Mozilla Corp., UK > Begin forwarded message: > > From: Ravikumar Dandu <[email protected]> > Date: 3 November 2015 at 11:49:37 GMT > To: [email protected] > Cc: Wilfred Mathanaraj <[email protected]>, Tamara Hills > <[email protected]>, Francisco Jordano <[email protected]>, David Bialer > <[email protected]>, Dominik Strohmeier <[email protected]> > Subject: A/B testing on FxOS > > Hi, > > This email is for collecting ideas/thoughts for A/B testing on FxOS. > > Device metrics is collecting information on app usage and user engagement > through opt-in. Moving forward, we want to improve our processes for design > and features based on real usage feedback of FxOS. As part of this we want to > take action based on feedback from users (foxfooders/end users) - within our > privacy policies, and legal limits - to more clearly guide our development > process. > > To move this to the next level, A/B testing will play a crucial part. A/B > testing will enable testing multiple variants of apps/ux against each other, > so we can determine whether a particular change (a new feature or a change to > a feature) is improving the engagement/experience of user > > This is not a simple process and involves various teams to be in sync in > order to have the right results to guide us. By measuring the impact that > changes have on metrics such as App Usage, time spent on panels etc, we can > ensure that changes/features produces positive results. We also need to > identify processes for targeting user segments, and getting the right/random > mix for the different versions. Also, need to plan to adapt our OS to the > results we obtain. > > Product team is starting to gather information and requirements on how we can > implement A/B testing and guide our design and development work. We would > like to gather your thoughts and feedback on: > Results of A/B testing that you would like to see for the particular > development you are doing > Ideas for A/B testing framework (eg: Addon to deliver the proposed change to > an app for certain users) > Tools that you've previously used or are using (eg: Optimizely for webapps) > > This will help us help focus our resources in the appropriate place to get > biggest bang for our development buck. > > We've started an initial draft document on FxOS A/B Testing Product > Requirements at > https://docs.google.com/document/d/1BZZ42iWojJa4WJ2AUHMav7aFZ0v_fIZ6ynIYWTUIcYc/edit > > <https://docs.google.com/document/d/1BZZ42iWojJa4WJ2AUHMav7aFZ0v_fIZ6ynIYWTUIcYc/edit> > It is editable, so please add/comment on it. > > Please also feel free to reach out to me and Wilfred to discuss > ideas/feedback. We'll organize meetings with individual teams to gather more > input. > > Regards > Ravi & Wilfred > > <https://docs.google.com/document/d/1BZZ42iWojJa4WJ2AUHMav7aFZ0v_fIZ6ynIYWTUIcYc/edit>
_______________________________________________ dev-fxos mailing list [email protected] https://lists.mozilla.org/listinfo/dev-fxos
