Is this intended to be an open piece of infrastructure that anyone can edit? https://www.mediawiki.org/wiki/Task_recommendations
Of course, you can say anything closed is something that just hasn't been made open yet, but that's exactly why I raise the issue. On 3 September 2014 15:18, Oliver Keyes <[email protected]> wrote: > Sure: point me to something I mentioned that's a closed product and isn't > a prerequisite for an open one? :p > > On Wednesday, 3 September 2014, Edward Saperia <[email protected]> > wrote: > >> In a movement like this with a lot of very active, very leveraged >> community activity, it seems to me that we should *always* be trying to >> make things that are infrastructure instead of closed products. >> >> cc Halfak - one of the few talks I managed to attend at Wikimania was his >> talk on "Research as Infrastructure", which I thought made the case very >> well. >> >> *Edward Saperia* >> Conference Director Wikimania London <http://www.wikimanialondon.org> >> email • facebook <http://www.facebook.com/edsaperia> • twitter >> <http://www.twitter.com/edsaperia> • 07796955572 >> 133-135 Bethnal Green Road, E2 7DG >> >> >> On 3 September 2014 00:17, Jonathan Morgan <[email protected]> wrote: >> >>> I agree with you, Ed. Although I don't think that it's realistic to >>> expect a product teamlike EE/Growth to create these open research tools. >>> Their primary output is always going to be the shiny products, not the >>> slightly-less-shiny infrastructure. Now *Analytics, *on the other >>> hand.. (*coughs* and looks pointedly at Ironholds...). >>> >>> Also, the next round of IEGs opened yesterday >>> <https://meta.wikimedia.org/wiki/Grants:IEG>. There's probably a >>> fundable project in what you describe, given a team with the right skill >>> sets. I'd be happy to provide feedback on a proposal. >>> >>> Cheers, >>> Jonathan >>> >>> >>> >>> >>> On Mon, Sep 1, 2014 at 8:16 AM, Edward Saperia <[email protected]> >>> wrote: >>> >>>> Sure, I understand how research is done. >>>> >>>> However, you could feasibly create components that allow for a certain >>>> types of experiments and open up the analysis side to the community. I >>>> think this could be a lot more successful than you'd expect - the community >>>> has many smart people, and together we could decide and promote best >>>> practice across projects/experiments. They'd also be able to drive >>>> suggestions for what new components to implement to expand the experiment >>>> space, and more generally grow interest in the work of the EE team. >>>> >>>> I understand that what you're doing now is quick and dirty and just >>>> trying to get something up and working, but I hope that longer term you >>>> have in mind the capability of the community to help you in this kind of >>>> endeavour. We're all keen to grow participation, and giving us tools to >>>> experiment ourselves will ultimately be more effective than anything you >>>> can do centrally. >>>> >>>> *Edward Saperia* >>>> Conference Director Wikimania London <http://www.wikimanialondon.org> >>>> email • facebook <http://www.facebook.com/edsaperia> • twitter >>>> <http://www.twitter.com/edsaperia> • 07796955572 >>>> 133-135 Bethnal Green Road, E2 7DG >>>> >>>> >>>> On 26 August 2014 20:03, Oliver Keyes <[email protected]> wrote: >>>> >>>>> Neither; the tools we have for running experiments are largely >>>>> hand-build on an ad hoc basis. For data collection we have tools like >>>>> eventlogging, although they require developer energy to integrate with >>>>> [potential area of experimentation]. But for actually analysing the >>>>> results >>>>> it looks very different. >>>>> >>>>> Let's use a couple of concrete examples: suppose we wanted to look at >>>>> whether there was a statistically significant variation in whether or not >>>>> people edited if we included a contributor tagline, versus didn't. We'd >>>>> need to take the same set of pages, ideally, and run a controlled study >>>>> around an A/B test. >>>>> >>>>> So first we'd display one version of the site for 50% of the >>>>> population and another for the other 50% (realistically we'd probably use >>>>> smaller sets and give the vast majority of editors the default experience, >>>>> but it's a hypothetical, so let's run with it). That would require >>>>> developer energy. Then we'd set up some kind of logging to pipe back edit >>>>> attempts and view attempts by [control sample/not control sample]. Also >>>>> developer energy, although much less. *Then*, crucially, we'd have to >>>>> actually do the analysis, which is not something that can be robustly >>>>> generalised. >>>>> >>>>> In this example we'd be looking for significance, so we'd be looking >>>>> at using some kind of statistical hypothesis test. Those vary depending on >>>>> what probability distributions the underlying population follows. So we >>>>> need to work out what probability distribution is most appropriate, and >>>>> then apply the test most appropriate to that distribution. And that's not >>>>> something that can be automated through software. As a result, we get the >>>>> data and then work out how to test for significance. >>>>> >>>>> The alternate hypothesis would be something observational; you make >>>>> the change and then compare the behaviour of people while the change is >>>>> live to their behaviour before and after. This cuts out most of the >>>>> developer cost but doesn't do anything for the research support or the >>>>> ad-hoc code and tools that need to come with it. >>>>> >>>>> >>>>> On 26 August 2014 10:52, Edward Saperia <[email protected]> >>>>> wrote: >>>>> >>>>>> You mean, you don't have them yourselves, or you can't expose them? >>>>>> >>>>>> *Edward Saperia* >>>>>> Conference Director Wikimania London <http://www.wikimanialondon.org> >>>>>> email • facebook <http://www.facebook.com/edsaperia> • twitter >>>>>> <http://www.twitter.com/edsaperia> • 07796955572 >>>>>> 133-135 Bethnal Green Road, E2 7DG >>>>>> >>>>>> >>>>>> On 26 August 2014 15:46, Oliver Keyes <[email protected]> wrote: >>>>>> >>>>>>> Except we don't have those tools. There are a lot of domains in the >>>>>>> ecosystem where this kind of experimentation and targeting on a >>>>>>> per-wiki or >>>>>>> per-project basis, but we have a big gap around functionality and >>>>>>> expertise >>>>>>> to let us scientifically test the efficacy of their various >>>>>>> implementations. >>>>>>> >>>>>>> >>>>>>> On 26 August 2014 10:34, Edward Saperia <[email protected]> >>>>>>> wrote: >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>> There's no point in polling existing community members about >>>>>>>>> functionality they will not see. >>>>>>>>> >>>>>>>> >>>>>>>> While I am a great supporter of your team's work, I'd just like to >>>>>>>> comment on the above; >>>>>>>> >>>>>>>> Wiser community members are aware that they are part of a powerful >>>>>>>> ecosystem, and that taming this ecosystem is a far more leveraged >>>>>>>> pursuit >>>>>>>> than doing the work yourself. Creating additional endpoints for >>>>>>>> onboarding >>>>>>>> processes that you're exposing to new users should be something that >>>>>>>> all >>>>>>>> projects are excited to take part in, so hopefully you'd want to poll >>>>>>>> the >>>>>>>> community for the valuable "Yes, and..." responses you'll get. >>>>>>>> >>>>>>>> If you find you don't get responses like this, perhaps you might >>>>>>>> want to consider re-framing your new functionality as open >>>>>>>> infrastructure >>>>>>>> that the rest of the community is invited to build on, for example >>>>>>>> maybe >>>>>>>> wikiprojects themselves could specify the suggestions that are shown >>>>>>>> to new >>>>>>>> editors who edit in their subject areas? >>>>>>>> >>>>>>>> Given appropriate tools to track effectiveness, this could create a >>>>>>>> huge, open environment for experimentation that could find interesting >>>>>>>> solutions faster than any engineering department ever could on their >>>>>>>> own. >>>>>>>> >>>>>>>> *Edward Saperia* >>>>>>>> Conference Director Wikimania London >>>>>>>> <http://www.wikimanialondon.org/> >>>>>>>> email • facebook <http://www.facebook.com/edsaperia> • twitter >>>>>>>> <http://www.twitter.com/edsaperia> • 07796955572 >>>>>>>> 133-135 Bethnal Green Road, E2 7DG >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> EE mailing list >>>>>>>> [email protected] >>>>>>>> https://lists.wikimedia.org/mailman/listinfo/ee >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Oliver Keyes >>>>>>> Research Analyst >>>>>>> Wikimedia Foundation >>>>>>> >>>>>>> _______________________________________________ >>>>>>> EE mailing list >>>>>>> [email protected] >>>>>>> https://lists.wikimedia.org/mailman/listinfo/ee >>>>>>> >>>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> EE mailing list >>>>>> [email protected] >>>>>> https://lists.wikimedia.org/mailman/listinfo/ee >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Oliver Keyes >>>>> Research Analyst >>>>> Wikimedia Foundation >>>>> >>>>> _______________________________________________ >>>>> EE mailing list >>>>> [email protected] >>>>> https://lists.wikimedia.org/mailman/listinfo/ee >>>>> >>>>> >>>> >>>> _______________________________________________ >>>> EE mailing list >>>> [email protected] >>>> https://lists.wikimedia.org/mailman/listinfo/ee >>>> >>>> >>> >>> >>> -- >>> Jonathan T. Morgan >>> Learning Strategist >>> Wikimedia Foundation >>> User:Jmorgan (WMF) <https://meta.wikimedia.org/wiki/User:Jmorgan_(WMF)> >>> [email protected] >>> >>> >>> _______________________________________________ >>> EE mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/ee >>> >>> >> > > -- > Sent from a portable device of Lovecraftian complexity. > > _______________________________________________ > EE mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/ee > >
_______________________________________________ EE mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/ee
