So you want to track multiple metrics for a given A/B test? That would be an Absurdity https://github.com/xing/absurdity! Oh, the Vanity http://vanity.labnotes.org/ of some people. ;-D
Best, Rob PS: I know the Vanity docs don't really describe using more than one metric, but it has worked for me in the past and it does appear from http://vanity.labnotes.org/api/Vanity/Experiment/AbTest.html#metrics-instance_method that it should continue to work. On Nov 2, 2012, at 1:35 PM, Ian Young <[email protected]> wrote: > Hey all, > > I'm looking into A/B testing tools for Rails. There are a number of good > options available, but to my surprise, none of the tools I've looked at > appear to support measuring multiple metrics for a single test. I know I'm > not the only one to think of this, because > http://bjk5.com/post/28269263789/lessons-learned-a-b-testing-with-gae-bingo > describes and screenshots exactly what I had in mind. Unfortunately, that's a > Python tool he's talking about. > > It's easy enough to optimize your landing page based on a single metric > (conversions), but like Khan Academy, I want to test more nuanced questions - > are our users more engaged with the content? Do they participate more? Can we > increase click-through without harming long-term retention? It seems somewhat > constraining to test a new feature but only get an answer for a single > measurement. > > So, anyone know of a tool I've overlooked that might do this? Am I really the > only one who wants this feature? Anyone have a persuasive argument in favor > of sticking to a single metric? > > Ian > > -- > SD Ruby mailing list > [email protected] > http://groups.google.com/group/sdruby -- SD Ruby mailing list [email protected] http://groups.google.com/group/sdruby
