> On Sep 2, 2015, at 3:53 PM, Toby Negrin <[email protected]> wrote:
> 1. We're moving people from an open platform to a closed platform: I think
> this is an oversimplification of the situation -- as has been noted before,
> the android app is 100% open source and while the data is not, in my
> opinion, comprehensive, it's inarguable that a large percentage of mobile
> traffic on the internet is from apps. It's not possible to fulfill our
> mission[4] if Wikipedia and sister project content is not available in
> widely used channels.
I'm not sure this makes a lot of sense. The widest, most-open content
channel that the projects have is through the web interfaces: all phones, all
devices, all computers can access the same content in the same manner. That is
to say: 100% of our readers have the ability to use the web versions (either
desktop or mobile web) where as only a subset can use the Android app, which is
a different subset that can use iOS. (They also end up having fragmented
experiences, which is sub-optimal.)
So it seems to me that the apps are not required to fulfill the
mission. They feel like distractions, and - quite possibly - negatives to the
mission (in that we can't convert Readers into Editors through the app).
(Which, by the way, this whole "focus entirely on readers" shift seems
counter-intuitive to me. Having a billion readers doesn't mean anything if
there aren't any editors anymore. It's a complete failure at that point.)
> 2. The campaign was not publicized before launch: We notified the Finnish
> community on their Village pump before the campaign began[5] and the
> campaign is detailed on the central notice page[6]. We felt this was
> appropriate considering the scope of the test.
Restricting the conversation to two very small, almost impenetrable
discussion areas seems unwise. It seems obvious to me that this idea and
action would cause friction with the community. I don't think there's any
bad-faith going on here, but this definitely feels like an oversight.
> 3. Banners/Interstitials don't work/suck/etc: There's a difference between
> a forced install and letting users know that an app exists and our
> designers have worked hard to make the banners effective without being
> excessively intrusive. You can see the designs on the Phab ticket above. I
> don't generally place a great deal of faith in blog posts or other
> company's data -- the google study showing the ineffectiveness of
> interstitials has already been challenged by other similarly reputable
> sources [7,8]. For this and other reasons, I believe that we need to gather
> our own data.
Is "our own data" more important than the goodwill of our users or
developers? I think that's a big part of why people might be upset about this:
it's a step away from what had classically been the principles underlying the
movement's activities.
Even that said, though: this is the first anyone is saying "yes, we
did some research about interstitials". It seems to me that the Google study
was something that could have been discussed ahead of time. I also don't
understand why we can't do the whole Open Source thing and make use of other
people's research, unless this indicates a further shift into "not invented
here" territory.
> 4. We don't understand what success looks like: We are planning a meeting
> with our Research team[9] to assess the statistical validity of our
> results, but the basic question is if users read more content using the app
> than the mobile web. This information will help guide us on future product
> decisions and will be shared with the community.
An experiment without a box isn't an experiment.
"We would like to determine if people read more through the
apps than through the web interface" is a _great_ question (but also one that
could probably be answered just by looking at squid logs). I don't know that
it needs an advertising campaign to create app users to do it (though I could
be wrong, and often am, and would love to hear how if so). It further seems
that advertising the mobile apps would create a biases in the research (if only
that "newish" app users are likely to use it more often earlier in their
"We would like to determine if people download the app more
often if they've been given an interstitial" is also an interesting question
but it's got a secondary question that no one seems to care about: "How many
readers have we put off from returning by showing them this interstitial?" I
know that I often immediately shut windows and tabs when I'm told "download our
app!"
If this were brought to the wider community in a different manner,
there may have been a completely different response:
"We do not believe that people are aware that there are
official Wikipedia apps. We would like to run an experiment to see how likely
people are to switch to the app experience if they know it exists. Of those
that switch, we would like to find out how many of them increase their usage of
the content, and, ideally, we'd like to know which features of the app are the
most popular and useful. Additionally, we'd like to know the drop-off counts.
We want to do it in a controlled environment where we understand the patterns
as they exist fairly well. We'll run this experiment for X days, and we know
that there will be biases on W, Y, and Z."
I could easily get behind that set of questions.
I don't really expect a response to any of this, by the way.
---
Brandon Harris :: [email protected] :: made of steel wool and whiskey
_______________________________________________
Wikitech-l mailing list
[email protected]
https://lists.wikimedia.org/mailman/listinfo/wikitech-l
