Awesome Oliver, thanks! Having a look. On Sat, Mar 21, 2015 at 4:32 PM, Oliver Keyes <[email protected]> wrote:
> Alrighty; existing implementation updated with > https://gerrit.wikimedia.org/r/#/c/198489/ which also exposes the > isAppPageview method and associates a UDF with it (Marcel, this means > you'll be able to do your stuff. Rockin'!) > > Let me know when y'all have the necessary varnish and app changes > proposed/committed/merged/etc so I can work on the updates to this > method and the documentation in parallel. > > On 20 March 2015 at 21:11, Oliver Keyes <[email protected]> wrote: > > Okay. So, distinct header, registering refresh=1 or [nothing] in the > > x-analytics field when it hits the varnish layer? Rocking! Let me know > > what happens/where it goes/when it goes/etc. > > > > The RESTbase services, yeah; they're going to make an impact. Not just > > in format either (I have literally no idea how the caching setup there > > works, if there is a caching setup, or which varnish cluster any such > > caching setup would go through and so if we're even picking up those > > requests at all). > > > > On 20 March 2015 at 19:11, Bernd Sitzmann <[email protected]> wrote: > >> Adam: Thanks for including us Android devs. > >> I think a distinct header is probably preferable. > >> > >> Oliver: > >> I think you already got this, so just to confirm: for Android you can > use > >> sections=0 for counting pageviews, and sections=all for counting saved > page > >> refreshes. > >> > >> As a heads-up we're experimenting with Node.js/RESTBase services. Not > sure > >> when and if those will be used in production. We're at an early stage. > Just > >> wanted to mention that this since it will change the way we request > pages > >> from the server significantly (actually it would be from different > servers, > >> too). That's probably another pro for using HTTP request headers. > >> > >> The downside is that we're not using this special request header yet. > >> > >> Bernd > >> > >> On Fri, Mar 20, 2015 at 3:17 PM, Oliver Keyes <[email protected]> > wrote: > >>> > >>> Sounds good for me, although I'd just go for the first option. There's > >>> nothing contained in the second option not found in the first that we > >>> need (that is: yes, it has device version and OS and all of that, > >>> which we need for other things, but then so does the user agent, so > >>> it's probably extraneous work to include all of that a /second/ time) > >>> > >>> On 20 March 2015 at 15:42, Adam Baso <[email protected]> wrote: > >>> > Either use of a distinct header (e.g., X-WMF-Refresh: 1) or use of a > >>> > distinct parameter (e.g., wprov=rfsi1 for "refresh from saved pages > on > >>> > iOS, > >>> > version 1") from the client would work. The use of a distinct > parameter > >>> > is > >>> > much easier. > >>> > > >>> > Then the VCL code could be updated to look for the field and enrich > the > >>> > X-Analytics header accordingly. > >>> > > >>> > Okay? > >>> > > >>> > > >>> > On Fri, Mar 20, 2015 at 11:37 AM, Oliver Keyes <[email protected] > > > >>> > wrote: > >>> >> > >>> >> Just a note that I'm started on a related patch now, using the (we > now > >>> >> know, not-futureproof) logic of: > >>> >> > >>> >> 1. if it's got sections=0 it's a pageview; > >>> >> 2. if it's got sections=all and is from iOS it's a pageview. > >>> >> > >>> >> Would appreciate some feedback on the idea of sending refresh=1 in > the > >>> >> x_analytics header, so we know what to expect there. > >>> >> > >>> >> On 20 March 2015 at 00:44, Oliver Keyes <[email protected]> > wrote: > >>> >> > In the long-term we'll just be relying on x_analytics, yes, > because > >>> >> > the app will be sending through the pageID and namespace in the > same > >>> >> > way as desktop and the mobile web do; so it'll be > >>> >> > pageid=50;ns=0;refresh-1 or pageid=50;ns=0, respectively. That's a > >>> >> > different patch. In the short-term I'm not seeing how building > out a > >>> >> > complex ecosystem for this is a valuable use of time given that we > >>> >> > know what we'll be switching to (and that it's standardised not > just > >>> >> > across apps, but also for desktop and the mobile web, to boot). > All > >>> >> > we > >>> >> > care about distinguishing that we won't get for free as part of > >>> >> > already-scheduled work is refreshes from pageviews. > >>> >> > > >>> >> > On 19 March 2015 at 23:59, Gergo Tisza <[email protected]> > wrote: > >>> >> >> On Thu, Mar 19, 2015 at 8:06 PM, Oliver Keyes < > [email protected]> > >>> >> >> wrote: > >>> >> >>> > >>> >> >>> As I see it, we basically have two possibilities here: > >>> >> >>> > >>> >> >>> 1. Make the URLs distinguishable; > >>> >> >>> 2. Add additional metadata in a non-URL place > >>> >> >>> > >>> >> >>> 1 is undesirable because it ruins caching, and we like caching. > So > >>> >> >>> we > >>> >> >>> look at 2, which realistically means the x_analytics field. Why > >>> >> >>> don't > >>> >> >>> we add a parameter there? refresh=1. And then, our app check > boils > >>> >> >>> down to (in pseudocode): > >>> >> >>> > >>> >> >>> if(other_checks & urlContains("sections=(0|all)" & > >>> >> >>> !xAnalyticsContains("refresh")){ > >>> >> >>> return true; > >>> >> >>> } > >>> >> >>> return false; > >>> >> >>> > >>> >> >>> Nice and simple and easy. It'll require some coordination with > >>> >> >>> Ottomata because it means modifying the UDF parameters, and > we're > >>> >> >>> using said UDF in production so it'll have to be synced to a > change > >>> >> >>> in > >>> >> >>> the relevant Oozie job, but it should be totally doable, and I > >>> >> >>> can't > >>> >> >>> see an easier way of doing it. > >>> >> >>> > >>> >> >>> Thoughts, people? > >>> >> >> > >>> >> >> > >>> >> >> Why use anything other than X-Analytics at all? Source of the > >>> >> >> pageview > >>> >> >> is > >>> >> >> exactly the kind of information it is meant for. Just set > >>> >> >> source=AndroidAppPageView / source=IosAppSectionView / > >>> >> >> source=AndroidAppSavedPageRefresh etc. (Or set up a mapping to > >>> >> >> three-letter > >>> >> >> acronyms if you want to be nice on the servers.) That puts > logging > >>> >> >> completely in the hands of the app developers, so there are no > >>> >> >> information > >>> >> >> flow problems and less organizational overhead; it also makes > rules > >>> >> >> more > >>> >> >> explicit (and thus harder to mess up / easier to spot errors). > >>> >> >> > >>> >> >> _______________________________________________ > >>> >> >> Analytics mailing list > >>> >> >> [email protected] > >>> >> >> https://lists.wikimedia.org/mailman/listinfo/analytics > >>> >> >> > >>> >> > > >>> >> > > >>> >> > > >>> >> > -- > >>> >> > Oliver Keyes > >>> >> > Research Analyst > >>> >> > Wikimedia Foundation > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Oliver Keyes > >>> >> Research Analyst > >>> >> Wikimedia Foundation > >>> >> > >>> >> _______________________________________________ > >>> >> Analytics mailing list > >>> >> [email protected] > >>> >> https://lists.wikimedia.org/mailman/listinfo/analytics > >>> > > >>> > > >>> > > >>> > _______________________________________________ > >>> > Analytics mailing list > >>> > [email protected] > >>> > https://lists.wikimedia.org/mailman/listinfo/analytics > >>> > > >>> > >>> > >>> > >>> -- > >>> Oliver Keyes > >>> Research Analyst > >>> Wikimedia Foundation > >> > >> > > > > > > > > -- > > Oliver Keyes > > Research Analyst > > Wikimedia Foundation > > > > -- > Oliver Keyes > Research Analyst > Wikimedia Foundation > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
