+Corey Floyd, who recently added code for this stuff on iOS (slated for forthcoming 4.1.4 iOS Wikipedia app release).
On Wed, May 27, 2015 at 12:33 PM, Adam Baso <[email protected]> wrote: > Thought I'd step in here. People who know the mechanics of the relevant > logging are as follows: > > Android: Dmitry Brant > Desktop: Bahodir (Baha) Mansurov > Mobile Web: Sam Smith & Baha > > CC'ing them. > > As I understand, Dan Garry's been talking with Baha already on the desktop > piece. > > It looks like on Chrome/42 UAs the clickthrough rate for a given suggest > search form interaction is about 40%. [1] > > A couple of patches are pending (JS for emitting event on new EL schema) > that will make it possible to figure out how often form submission within a > form interaction (ENTER/RETURN, tapping the magnifying class) occurs as > well. > > Total form interactions (keys on userSessionToken...maybe a better name > could be used) minus clickthroughs (click-result) minus form submission > (submit-form in the pending patches on the new EL schema) would be a rough > proxy of abandonment, I think. > > I had heard sendBeacon capable UAs were likely to have greater success > emitting the click-result (i.e., when user clicks on a suggestion from the > form on the search panel on desktop) event via mw.track, so it may make > sense to confine queries for such analysis on desktop to known sendBeacon > browsers [2] to increase the odds of high fidelity data just in case there > are outlier browsers that manage to somehow emit click-result events > through means other than sendBeacon (it seems there may be some of these, > assuming non-forged UAs). > > -Adam > > [1] > > > SELECT count(*) FROM Search_11670541 WHERE timestamp >= '20150526' AND > timestamp < '20150527' AND event_action = 'click-result' AND wiki = > 'enwiki' and userAgent LIKE '%Chrome/42%'; > +----------+ > | count(*) | > +----------+ > | 112 | > +----------+ > 1 row in set (2.96 sec) > > > SELECT count(DISTINCT event_userSessionToken) FROM Search_11670541 WHERE > timestamp >= '20150526' AND timestamp < '20150527' AND event_action = > 'click-result' AND wiki = 'enwiki' and userAgent LIKE '%Chrome/42%'; > +----------------------------------------+ > | count(DISTINCT event_userSessionToken) | > +----------------------------------------+ > | 112 | > +----------------------------------------+ > 1 row in set (38.06 sec) > > > > SELECT count(DISTINCT event_userSessionToken) FROM Search_11670541 WHERE > timestamp > '20150526' AND timestamp < '20150527' AND wiki = 'enwiki' and > userAgent LIKE '%Chrome/42%'; > +----------------------------------------+ > | count(DISTINCT event_userSessionToken) | > +----------------------------------------+ > | 286 | > +----------------------------------------+ > 1 row in set (7.26 sec) > > > [2] https://developer.mozilla.org/it/docs/Web/API/Navigator/sendBeacon > > On Tue, May 26, 2015 at 4:37 PM, Oliver Keyes <[email protected]> > wrote: > >> Thanks Tomasz; great feedback! In order: >> >> * yeah, top percentiles were a heavily-requested thing so I built it >> in from the get-go. Similarly, mean/median so we have some ability to >> avoid distorting the results when the distribution changes. >> * The 3 days data thing is a known - >> https://phabricator.wikimedia.org/T100056 - and is next on my to-do >> list for bugfixes :). >> * Glad you like the interface! It's actually functional on mobile, too :D. >> * Sample rate is crucial, yep. I'm reaching out to the authors of the >> relevant EL schemas to find out how each was handled. >> * Sessions < results opened makes sense in the event that users didn't >> find what they wanted and went back to try again, but I'm not sure how >> "session" is calculated; this is again something we lack transparency >> around :(. Dan? You're the apps wizard. >> >> In supporting this: probably nothing at the moment although Nik/Kevin >> chipping in on the relevant phabricator ticket >> (https://phabricator.wikimedia.org/T99762 ) to validate how much of a >> PITA the idea of a unified schema and the associated implementations >> are, would be good. >> >> I'm sort of shocked to hear "we're supposed to be presenting this data >> at the next metrics meeting": in the future if there are instances >> where data is going to be up for public scrutiny, would it be possible >> to explicitly associate time for that? My goal is to get us to the >> point where our data is reliable all, or at least, most of the time, >> and for a fragment of one person's time over two weeks, I think >> progress on that is pretty fantastic. But prepping data for that kind >> of event does change the priorities and what tasks should be worked >> on. >> >> If we want to present data, generally speaking, let's discuss what we >> can show off. If we want to present the dashboards I'll put my all >> into making the data at least something where we know the >> deficiencies, if not something where we consider the deficiencies >> tolerable. >> >> On 26 May 2015 at 19:24, Tomasz Finc <[email protected]> wrote: >> > Thanks Oliver >> > >> > Early observations >> > >> > * Really happy to see top percentiles in load graphs >> > * Mobile Web has only three days data >> > * Interface is simple and easy to use >> > * We need to know the sample rate >> > * Apps have fewer sessions than results page opened >> > >> > Speaking over IRC it's clear that we don't have confidence in this >> > data. We need to fix this and fix it quickly so that we can accurately >> > plan our work. We're supposed to be presenting this data at the next >> > metrics meeting and we're not a point where I feel comfortable sharing >> > our data let alone next steps. >> > >> > Oliver & Dan, what can the team do to support you guys on this? I want >> > you guys to own this and know that were here to support you. >> > >> > Should I be adding new feature requests and bugs to >> > https://phabricator.wikimedia.org/tag/search-data-analytics/ ? >> > >> > --tomasz >> > >> > On Tue, May 26, 2015 at 11:04 AM, James Douglas <[email protected]> >> wrote: >> >> This is a very exciting preview of things to come. >> >> >> >> Where are the data coming from? Am I just confused, or does "6 search >> >> sessions per day" seem low? >> >> >> >> On Fri, May 22, 2015 at 2:35 PM, Oliver Keyes <[email protected]> >> wrote: >> >>> >> >>> http://searchdata.wmflabs.org/ - boop! This was my Friday. Previously >> >>> we were playing around with them and testing what we needed with a >> >>> static snapshot; these dashboards will now update once a day with new >> >>> information. >> >>> >> >>> It has turned up some bugs ("is the mobile schema just not running?") >> >>> and there are more metrics to add. But for the time being, is progress >> >>> :) >> >>> >> >>> -- >> >>> Oliver Keyes >> >>> Research Analyst >> >>> Wikimedia Foundation >> >>> >> >>> _______________________________________________ >> >>> Wikimedia-search-private mailing list >> >>> [email protected] >> >>> https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private >> >> >> >> >> >> >> >> _______________________________________________ >> >> Wikimedia-search-private mailing list >> >> [email protected] >> >> https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private >> >> >> > >> > _______________________________________________ >> > Wikimedia-search-private mailing list >> > [email protected] >> > https://lists.wikimedia.org/mailman/listinfo/wikimedia-search-private >> >> >> >> -- >> Oliver Keyes >> Research Analyst >> Wikimedia Foundation >> >> _______________________________________________ >> Analytics mailing list >> [email protected] >> https://lists.wikimedia.org/mailman/listinfo/analytics >> > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
