Alrighty; existing implementation updated with https://gerrit.wikimedia.org/r/#/c/198489/ which also exposes the isAppPageview method and associates a UDF with it (Marcel, this means you'll be able to do your stuff. Rockin'!)
Let me know when y'all have the necessary varnish and app changes proposed/committed/merged/etc so I can work on the updates to this method and the documentation in parallel. On 20 March 2015 at 21:11, Oliver Keyes <[email protected]> wrote: > Okay. So, distinct header, registering refresh=1 or [nothing] in the > x-analytics field when it hits the varnish layer? Rocking! Let me know > what happens/where it goes/when it goes/etc. > > The RESTbase services, yeah; they're going to make an impact. Not just > in format either (I have literally no idea how the caching setup there > works, if there is a caching setup, or which varnish cluster any such > caching setup would go through and so if we're even picking up those > requests at all). > > On 20 March 2015 at 19:11, Bernd Sitzmann <[email protected]> wrote: >> Adam: Thanks for including us Android devs. >> I think a distinct header is probably preferable. >> >> Oliver: >> I think you already got this, so just to confirm: for Android you can use >> sections=0 for counting pageviews, and sections=all for counting saved page >> refreshes. >> >> As a heads-up we're experimenting with Node.js/RESTBase services. Not sure >> when and if those will be used in production. We're at an early stage. Just >> wanted to mention that this since it will change the way we request pages >> from the server significantly (actually it would be from different servers, >> too). That's probably another pro for using HTTP request headers. >> >> The downside is that we're not using this special request header yet. >> >> Bernd >> >> On Fri, Mar 20, 2015 at 3:17 PM, Oliver Keyes <[email protected]> wrote: >>> >>> Sounds good for me, although I'd just go for the first option. There's >>> nothing contained in the second option not found in the first that we >>> need (that is: yes, it has device version and OS and all of that, >>> which we need for other things, but then so does the user agent, so >>> it's probably extraneous work to include all of that a /second/ time) >>> >>> On 20 March 2015 at 15:42, Adam Baso <[email protected]> wrote: >>> > Either use of a distinct header (e.g., X-WMF-Refresh: 1) or use of a >>> > distinct parameter (e.g., wprov=rfsi1 for "refresh from saved pages on >>> > iOS, >>> > version 1") from the client would work. The use of a distinct parameter >>> > is >>> > much easier. >>> > >>> > Then the VCL code could be updated to look for the field and enrich the >>> > X-Analytics header accordingly. >>> > >>> > Okay? >>> > >>> > >>> > On Fri, Mar 20, 2015 at 11:37 AM, Oliver Keyes <[email protected]> >>> > wrote: >>> >> >>> >> Just a note that I'm started on a related patch now, using the (we now >>> >> know, not-futureproof) logic of: >>> >> >>> >> 1. if it's got sections=0 it's a pageview; >>> >> 2. if it's got sections=all and is from iOS it's a pageview. >>> >> >>> >> Would appreciate some feedback on the idea of sending refresh=1 in the >>> >> x_analytics header, so we know what to expect there. >>> >> >>> >> On 20 March 2015 at 00:44, Oliver Keyes <[email protected]> wrote: >>> >> > In the long-term we'll just be relying on x_analytics, yes, because >>> >> > the app will be sending through the pageID and namespace in the same >>> >> > way as desktop and the mobile web do; so it'll be >>> >> > pageid=50;ns=0;refresh-1 or pageid=50;ns=0, respectively. That's a >>> >> > different patch. In the short-term I'm not seeing how building out a >>> >> > complex ecosystem for this is a valuable use of time given that we >>> >> > know what we'll be switching to (and that it's standardised not just >>> >> > across apps, but also for desktop and the mobile web, to boot). All >>> >> > we >>> >> > care about distinguishing that we won't get for free as part of >>> >> > already-scheduled work is refreshes from pageviews. >>> >> > >>> >> > On 19 March 2015 at 23:59, Gergo Tisza <[email protected]> wrote: >>> >> >> On Thu, Mar 19, 2015 at 8:06 PM, Oliver Keyes <[email protected]> >>> >> >> wrote: >>> >> >>> >>> >> >>> As I see it, we basically have two possibilities here: >>> >> >>> >>> >> >>> 1. Make the URLs distinguishable; >>> >> >>> 2. Add additional metadata in a non-URL place >>> >> >>> >>> >> >>> 1 is undesirable because it ruins caching, and we like caching. So >>> >> >>> we >>> >> >>> look at 2, which realistically means the x_analytics field. Why >>> >> >>> don't >>> >> >>> we add a parameter there? refresh=1. And then, our app check boils >>> >> >>> down to (in pseudocode): >>> >> >>> >>> >> >>> if(other_checks & urlContains("sections=(0|all)" & >>> >> >>> !xAnalyticsContains("refresh")){ >>> >> >>> return true; >>> >> >>> } >>> >> >>> return false; >>> >> >>> >>> >> >>> Nice and simple and easy. It'll require some coordination with >>> >> >>> Ottomata because it means modifying the UDF parameters, and we're >>> >> >>> using said UDF in production so it'll have to be synced to a change >>> >> >>> in >>> >> >>> the relevant Oozie job, but it should be totally doable, and I >>> >> >>> can't >>> >> >>> see an easier way of doing it. >>> >> >>> >>> >> >>> Thoughts, people? >>> >> >> >>> >> >> >>> >> >> Why use anything other than X-Analytics at all? Source of the >>> >> >> pageview >>> >> >> is >>> >> >> exactly the kind of information it is meant for. Just set >>> >> >> source=AndroidAppPageView / source=IosAppSectionView / >>> >> >> source=AndroidAppSavedPageRefresh etc. (Or set up a mapping to >>> >> >> three-letter >>> >> >> acronyms if you want to be nice on the servers.) That puts logging >>> >> >> completely in the hands of the app developers, so there are no >>> >> >> information >>> >> >> flow problems and less organizational overhead; it also makes rules >>> >> >> more >>> >> >> explicit (and thus harder to mess up / easier to spot errors). >>> >> >> >>> >> >> _______________________________________________ >>> >> >> Analytics mailing list >>> >> >> [email protected] >>> >> >> https://lists.wikimedia.org/mailman/listinfo/analytics >>> >> >> >>> >> > >>> >> > >>> >> > >>> >> > -- >>> >> > Oliver Keyes >>> >> > Research Analyst >>> >> > Wikimedia Foundation >>> >> >>> >> >>> >> >>> >> -- >>> >> Oliver Keyes >>> >> Research Analyst >>> >> Wikimedia Foundation >>> >> >>> >> _______________________________________________ >>> >> Analytics mailing list >>> >> [email protected] >>> >> https://lists.wikimedia.org/mailman/listinfo/analytics >>> > >>> > >>> > >>> > _______________________________________________ >>> > Analytics mailing list >>> > [email protected] >>> > https://lists.wikimedia.org/mailman/listinfo/analytics >>> > >>> >>> >>> >>> -- >>> Oliver Keyes >>> Research Analyst >>> Wikimedia Foundation >> >> > > > > -- > Oliver Keyes > Research Analyst > Wikimedia Foundation -- Oliver Keyes Research Analyst Wikimedia Foundation _______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
