It wasn't previously on iOS, so we've just been tracking Android? :/ On 28 May 2015 at 12:34, Adam Baso <[email protected]> wrote: > +Corey Floyd, who recently added code for this stuff on iOS (slated for > forthcoming 4.1.4 iOS Wikipedia app release). > > On Wed, May 27, 2015 at 12:33 PM, Adam Baso <[email protected]> wrote: >> >> Thought I'd step in here. People who know the mechanics of the relevant >> logging are as follows: >> >> Android: Dmitry Brant >> Desktop: Bahodir (Baha) Mansurov >> Mobile Web: Sam Smith & Baha >> >> CC'ing them. >> >> As I understand, Dan Garry's been talking with Baha already on the desktop >> piece. >> >> It looks like on Chrome/42 UAs the clickthrough rate for a given suggest >> search form interaction is about 40%. [1] >> >> A couple of patches are pending (JS for emitting event on new EL schema) >> that will make it possible to figure out how often form submission within a >> form interaction (ENTER/RETURN, tapping the magnifying class) occurs as >> well. >> >> Total form interactions (keys on userSessionToken...maybe a better name >> could be used) minus clickthroughs (click-result) minus form submission >> (submit-form in the pending patches on the new EL schema) would be a rough >> proxy of abandonment, I think. >> >> I had heard sendBeacon capable UAs were likely to have greater success >> emitting the click-result (i.e., when user clicks on a suggestion from the >> form on the search panel on desktop) event via mw.track, so it may make >> sense to confine queries for such analysis on desktop to known sendBeacon >> browsers [2] to increase the odds of high fidelity data just in case there >> are outlier browsers that manage to somehow emit click-result events through >> means other than sendBeacon (it seems there may be some of these, assuming >> non-forged UAs). >> >> -Adam >> >> [1] >> >> > SELECT count(*) FROM Search_11670541 WHERE timestamp >= '20150526' AND >> > timestamp < '20150527' AND event_action = 'click-result' AND wiki = >> > 'enwiki' >> > and userAgent LIKE '%Chrome/42%'; >> +----------+ >> | count(*) | >> +----------+ >> | 112 | >> +----------+ >> 1 row in set (2.96 sec) >> >> > SELECT count(DISTINCT event_userSessionToken) FROM Search_11670541 WHERE >> > timestamp >= '20150526' AND timestamp < '20150527' AND event_action = >> > 'click-result' AND wiki = 'enwiki' and userAgent LIKE '%Chrome/42%'; >> +----------------------------------------+ >> | count(DISTINCT event_userSessionToken) | >> +----------------------------------------+ >> | 112 | >> +----------------------------------------+ >> 1 row in set (38.06 sec) >> >> >> > SELECT count(DISTINCT event_userSessionToken) FROM Search_11670541 WHERE >> > timestamp > '20150526' AND timestamp < '20150527' AND wiki = 'enwiki' and >> > userAgent LIKE '%Chrome/42%'; >> +----------------------------------------+ >> | count(DISTINCT event_userSessionToken) | >> +----------------------------------------+ >> | 286 | >> +----------------------------------------+ >> 1 row in set (7.26 sec) >> >> >> [2] https://developer.mozilla.org/it/docs/Web/API/Navigator/sendBeacon >> >> On Tue, May 26, 2015 at 4:37 PM, Oliver Keyes <[email protected]> >> wrote: >>> >>> Thanks Tomasz; great feedback! In order: >>> >>> * yeah, top percentiles were a heavily-requested thing so I built it >>> in from the get-go. Similarly, mean/median so we have some ability to >>> avoid distorting the results when the distribution changes. >>> * The 3 days data thing is a known - >>> https://phabricator.wikimedia.org/T100056 - and is next on my to-do >>> list for bugfixes :). >>> * Glad you like the interface! It's actually functional on mobile, too >>> :D. >>> * Sample rate is crucial, yep. I'm reaching out to the authors of the >>> relevant EL schemas to find out how each was handled. >>> * Sessions < results opened makes sense in the event that users didn't >>> find what they wanted and went back to try again, but I'm not sure how >>> "session" is calculated; this is again something we lack transparency >>> around :(. Dan? You're the apps wizard. >>> >>> In supporting this: probably nothing at the moment although Nik/Kevin >>> chipping in on the relevant phabricator ticket >>> (https://phabricator.wikimedia.org/T99762 ) to validate how much of a >>> PITA the idea of a unified schema and the associated implementations >>> are, would be good. >>> >>> I'm sort of shocked to hear "we're supposed to be presenting this data >>> at the next metrics meeting": in the future if there are instances >>> where data is going to be up for public scrutiny, would it be possible >>> to explicitly associate time for that? My goal is to get us to the >>> point where our data is reliable all, or at least, most of the time, >>> and for a fragment of one person's time over two weeks, I think >>> progress on that is pretty fantastic. But prepping data for that kind >>> of event does change the priorities and what tasks should be worked >>> on. >>> >>> If we want to present data, generally speaking, let's discuss what we >>> can show off. If we want to present the dashboards I'll put my all >>> into making the data at least something where we know the >>> deficiencies, if not something where we consider the deficiencies >>> tolerable. >>> >>> On 26 May 2015 at 19:24, Tomasz Finc <[email protected]> wrote: >>> > Thanks Oliver >>> > >>> > Early observations >>> > >>> > * Really happy to see top percentiles in load graphs >>> > * Mobile Web has only three days data >>> > * Interface is simple and easy to use >>> > * We need to know the sample rate >>> > * Apps have fewer sessions than results page opened >>> > >>> > Speaking over IRC it's clear that we don't have confidence in this >>> > data. We need to fix this and fix it quickly so that we can accurately >>> > plan our work. We're supposed to be presenting this data at the next >>> > metrics meeting and we're not a point where I feel comfortable sharing >>> > our data let alone next steps. >>> > >>> > Oliver & Dan, what can the team do to support you guys on this? I want >>> > you guys to own this and know that were here to support you. >>> > >>> > Should I be adding new feature requests and bugs to >>> > https://phabricator.wikimedia.org/tag/search-data-analytics/ ? >>> > >>> > --tomasz >>> > >>> > On Tue, May 26, 2015 at 11:04 AM, James Douglas >>> > <[email protected]> wrote: >>> >> This is a very exciting preview of things to come. >>> >> >>> >> Where are the data coming from? Am I just confused, or does "6 search >>> >> sessions per day" seem low? >>> >> >>> >> On Fri, May 22, 2015 at 2:35 PM, Oliver Keyes <[email protected]> >>> >> wrote: >>> >>> >>> >>> http://searchdata.wmflabs.org/ - boop! This was my Friday. Previously >>> >>> we were playing around with them and testing what we needed with a >>> >>> static snapshot; these dashboards will now update once a day with new >>> >>> information. >>> >>> >>> >>> It has turned up some bugs ("is the mobile schema just not running?") >>> >>> and there are more metrics to add. But for the time being, is >>> >>> progress >>> >>> :) >>> >>> >>> >>> -- >>> >>> Oliver Keyes >>> >>> Research Analyst >>> >>> Wikimedia Foundation >>> >>> >>> >>> _______________________________________________ >>> >>> Wikimedia-search-private mailing list >>> >>> [email protected] >>> >>> https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private >>> >> >>> >> >>> >> >>> >> _______________________________________________ >>> >> Wikimedia-search-private mailing list >>> >> [email protected] >>> >> https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private >>> >> >>> > >>> > _______________________________________________ >>> > Wikimedia-search-private mailing list >>> > [email protected] >>> > https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private >>> >>> >>> >>> -- >>> Oliver Keyes >>> Research Analyst >>> Wikimedia Foundation >>> >>> _______________________________________________ >>> Analytics mailing list >>> [email protected] >>> https://lists.wikimedia.org/mailman/listinfo/analytics >> >> > > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics >
-- Oliver Keyes Research Analyst Wikimedia Foundation _______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
