Hi all -- I'm going to try to address as many of the issues mentioned in this thread and the Phabricator ticket[1] as I can. I'm going to preface this by explaining why we're doing this.
First of all, this is only a test in order to increase our understanding of how our readers interact with our content. It's limited to a relatively small but representative sample of our readers and is not permanent. The results of this test will inform our mobile strategy. I'd also like to offer that in our community strategic consultation[3], mobile and apps was the single most commented upon topic, including many comments that we should build an app. Specific Issues 1. We're moving people from an open platform to a closed platform: I think this is an oversimplification of the situation -- as has been noted before, the android app is 100% open source and while the data is not, in my opinion, comprehensive, it's inarguable that a large percentage of mobile traffic on the internet is from apps. It's not possible to fulfill our mission[4] if Wikipedia and sister project content is not available in widely used channels. 2. The campaign was not publicized before launch: We notified the Finnish community on their Village pump before the campaign began[5] and the campaign is detailed on the central notice page[6]. We felt this was appropriate considering the scope of the test. 3. Banners/Interstitials don't work/suck/etc: There's a difference between a forced install and letting users know that an app exists and our designers have worked hard to make the banners effective without being excessively intrusive. You can see the designs on the Phab ticket above. I don't generally place a great deal of faith in blog posts or other company's data -- the google study showing the ineffectiveness of interstitials has already been challenged by other similarly reputable sources [7,8]. For this and other reasons, I believe that we need to gather our own data. 4. We don't understand what success looks like: We are planning a meeting with our Research team[9] to assess the statistical validity of our results, but the basic question is if users read more content using the app than the mobile web. This information will help guide us on future product decisions and will be shared with the community. -Toby [1] https://phabricator.wikimedia.org/T103896 [2] https://meta.wikimedia.org/wiki/Research:Projects [3] http://blog.wikimedia.org/2015/08/27/strategy-potential-mobile-multimedia-translation/ <https://upload.wikimedia.org/wikipedia/commons/c/c7/2015_Strategy_Consultation_Report.pdf> [4] https://wikimediafoundation.org/wiki/Mission_statement [5] https://fi.wikipedia.org/wiki/Wikipedia:Kahvihuone_%28uutiset%29#Running_banner_to_promote_Wikipedia_app_downloads [6] https://meta.wikimedia.org/wiki/CentralNotice/Calendar [7] http://googlewebmastercentral.blogspot.com/2015/07/google-case-study-on-app-download-interstitials.html [8] https://www.linkedin.com/pulse/all-app-promos-bad-googles-omar-restom [9] https://phabricator.wikimedia.org/T105561 On Wed, Sep 2, 2015 at 8:21 AM, Tilman Bayer <[email protected]> wrote: > On Tue, Sep 1, 2015 at 8:10 PM, Oliver Keyes <[email protected]> wrote: > > > > What was the publicising of the campaign prior to its launch? > > > > It should be pretty apparent to people with experience within the > > movement that this would be both entirely novel and pretty > > controversial. > > As mentioned on the Phabricator ticked, this is by no means the first > banner campaign inviting installation of an app. > > In June/July last year, there was a global campaign announcing the > launch of the new Android app (like now, shown on mobile web for > Android devices only): > > https://en.m.wikipedia.org/w/index.php?title=Special:Random&banner=Wpapp2014Androidmobile_1&uselang=en&force=1 > > https://en.m.wikipedia.org/w/index.php?title=Special:Random&banner=Wpapp2014Androidmobile_2&uselang=en&force=1 > (also ran in a few other languages besides English) > I don't recall it being controversial back then. > > And in 2013, the late Commons app was promoted in a similar campaign > on desktop and mobile: > > > https://en.wikipedia.org/w/index.php?title=Special:Random&banner=CommonsAppnonmobilewp&uselang=en&force=1 > (on desktop Wikipedia) > > https://commons.wikimedia.org/w/index.php?title=Special:Random&banner=CommonsAppnonmobilecommons&uselang=en&force=1 > (on Commons) > > https://en.m.wikipedia.org/w/index.php?title=Special:Random&banner=AndroidCommonsApp&uselang=en&force=1 > (mobile Wikipedia on Android devices) > > > > I'd expect some amount of transparency around it (a > > phabricator ticket is not, in and of itself, transparency). > > For those not familiar with the existing processes around banners, WMF > staff and community members who use this indeed highly prominent space > have been coordinating for years on this page: > https://meta.wikimedia.org/wiki/CentralNotice/Calendar > Quite a lot of people who care about banner use are watching it for > controversial or problematic uses > ( > https://meta.wikimedia.org/w/index.php?title=CentralNotice/Calendar&action=info#mw-pageinfo-watchers > ), discussion happens on the talk page there or is escalated to other > venues. > I see that the current banners were indeed listed there last week > before the launch. > > > To > > contrast, with search when we make /experimental/ modifications to the > > user experience of a tiny sample (through A/B testing) we not only > > list those changes in phabricator but also send explicit mailing list > > announcements - and those effect a smaller chunk of our user base on a > > platform. > Perhaps you could post some advice at > https://meta.wikimedia.org/wiki/Talk:CentralNotice about how people > running banners could learn from the WMF Discovery team in that > respect? > > > On Tue, Sep 1, 2015 at 8:30 AM, Ori Livneh <[email protected]> wrote: > > We appear to be running a banner campaign on the mobile web site, driving > > people to download the mobile app: > > > > https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_2 > > https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_1 > > > The links don't work for me (maybe because I'm not in Finland right > now); you can append "force=1" to make them show regardless of > targeting: > > https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_2&force=1 > https://en.m.wikipedia.org/wiki/?banner=Aug2015_app_banner_1&force=1 > > > > -- > Tilman Bayer > Senior Analyst > Wikimedia Foundation > IRC (Freenode): HaeB > > _______________________________________________ > Wikitech-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/wikitech-l > _______________________________________________ Wikitech-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikitech-l
