In case you didn't know, all API calls support 2 caching parameters, "maxage" and "smaxage". The first one controls client caching (basically sends headers back that will make the client cache the request), the second one controls server caching (varnish) across users. If you use the latter for things like mobileview, it's best if you take into account the fact that page content can change. In which case you'll probably want to add a bogus revisionid parameter (even if that's not a valid parameter for that particular API call), which will act as a cachebuster. Of course that means that you'll need at least one dynamic API call, or one with short caching values, to catch the fact that the revision id changed for that page. That is, if you care about not showing slightly stale wiki content... it really depends on the use case.
On Wed, Jun 3, 2015 at 10:43 PM, Brian Gerstle <[email protected]> wrote: > I'm definitely interested. I'll add some engineering tasks to the iOS > backlog t optimize siteinfo queries as a starting point. If we're going to > make some additional patches to mobileview, adding an extracts snippet and > adding cache headers seem like great improvements. > > > On Wed, Jun 3, 2015 at 2:39 PM, Adam Baso <[email protected]> wrote: > >> There are others interested in caching, too, and as third parties look to >> integrate api.php calls, it is important there as well. You're in good >> company! >> >> Anyone in Reading web engineering or infrastructure know how easy/hard it >> would be to have the caching related headers for action=mobileview mimic >> MFE HTML as well as whether that would be advisable? If we can shave, say, >> 100ms on respones and lower utilization at the Apaches, seems like a good >> investment while the mobile app content service is stood up and polished. >> >> Brian, from what I heard you say yesterday, the sitematrix and siteinfo >> stuff shouldn't be too risky. Interested in tackling the PHP? Need a >> pairing partner? >> >> -Adam >> >> >> On Wednesday, June 3, 2015, Brian Gerstle <[email protected]> wrote: >> >>> Also, I'm the guy complaining about cache control :-). >>> >>> The other related conversation is the future of mobileview itself, and >>> if/when/how the apps will interact w/ MW-core vs. RESTBase micro services. >>> Just something to keep in mind that might affect how much (if any) effort >>> we put into working on mobileview as opposed to investing in >>> RESTBase-powered services, which (IIUC) already have some caching >>> mechanisms to hook into. >>> >>> That said, some MW API's which are not page related could benefit from >>> cache-control, like sitematrix & siteinfo >>> <https://phabricator.wikimedia.org/T41061>. >>> >>> On Wed, Jun 3, 2015 at 10:09 AM, Brian Gerstle <[email protected]> >>> wrote: >>> >>>> Side note, do we need a lot of these headers? >>>> >>>> X-Powered-By: HHVM/3.6.1 >>>> >>>> X-Content-Type-Options: nosniff >>>> >>>> X-Frame-Options: SAMEORIGIN >>>>> X-Varnish: 2583358918, 1834266080 >>>>> Via: 1.1 varnish, 1.1 varnish >>>>> X-Cache: cp1047 miss (0), cp1060 frontend miss (0) >>>>> X-Analytics: https=1 >>>>> Set-Cookie: WMF-Last-Access=03-Jun-2015;Path=/;HttpOnly;Expires=Sun, >>>>> 05 Jul 2015 12:00:00 GMT >>>> >>>> >>>> Could save a few bytes down the wire if we nixed the unused ones. >>>> >>>> >>>> On Wed, Jun 3, 2015 at 9:28 AM, Adam Baso <[email protected]> wrote: >>>> >>>>> Hi team - >>>>> >>>>> There was a question yesterday if we had caching headers in the APIs >>>>> consumed by the apps. >>>>> >>>>> I did a quick cURL on a couple URLs for Android and iOS and it seems >>>>> like maybe requests are served cold from the Varnish cache (as with many >>>>> API responses), even if there may be some memcached/Redis hot storage at >>>>> the web server for the underlying content. >>>>> >>>>> Does anyone know if there's a safe approach to implement caching >>>>> headers (e.g., Cache-Control, Expires, ETag, etc.) to optimize for hot >>>>> cache hits for the API endpoints used by the apps and experimental >>>>> webapps? >>>>> In particular, would we benefit from doing so on action=mobileview? Is >>>>> there a way to canonicalize the request parameters to increase the odds of >>>>> a cache hit regardless of whether iOS or Android made the first hit >>>>> (either >>>>> client-side or perhaps at the edge with scare VCL)? What about for other >>>>> API endpoints? >>>>> >>>>> -Adam >>>>> >>>>> $ curl -s -D - " >>>>> http://en.m.wikipedia.org/w/api.php?action=mobileview&format=json&page=Fish&prop=text%7Csections%7Cthumb%7Cimage%7Cid%7Crevision%7Cdescription%7Clastmodified%7Cnormalizedtitle%7Cdisplaytitle%7Cprotection%7Ceditable&onlyrequestedsections=1§ions=0§ionprop=toclevel%7Cline%7Canchor&noheadings=true&thumbsize=640"; >>>>> -o /dev/null | grep -v GeoIP >>>>> >>>>> HTTP/1.1 200 OK >>>>> >>>>> Server: Apache >>>>> >>>>> X-Powered-By: HHVM/3.6.1 >>>>> >>>>> X-Content-Type-Options: nosniff >>>>> >>>>> X-Frame-Options: SAMEORIGIN >>>>> >>>>> Vary: Accept-Encoding,X-Forwarded-Proto,Cookie >>>>> >>>>> Content-Type: application/json; charset=utf-8 >>>>> >>>>> X-Varnish: 2886917462, 1761146385 >>>>> >>>>> Via: 1.1 varnish, 1.1 varnish >>>>> >>>>> Transfer-Encoding: chunked >>>>> >>>>> Date: Wed, 03 Jun 2015 13:20:08 GMT >>>>> >>>>> Age: 0 >>>>> >>>>> Connection: keep-alive >>>>> >>>>> X-Cache: cp1046 miss (0), cp1046 frontend miss (0) >>>>> >>>>> Set-Cookie: WMF-Last-Access=03-Jun-2015;Path=/;HttpOnly;Expires=Sun, >>>>> 05 Jul 2015 12:00:00 GMT >>>>> >>>>> Cache-Control: private, s-maxage=0, max-age=0, must-revalidate >>>>> >>>>> >>>>> $ curl -s -D - " >>>>> https://en.m.wikipedia.org/w/api.php?action=mobileview&format=json&noheadings=true&page=SM%20U-66&prop=sections%7Ctext%7Clastmodified%7Clastmodifiedby%7Clanguagecount%7Cid%7Cprotection%7Ceditable%7Cdisplaytitle%7Cthumb%7Cdescription%7Cimage§ionprop=toclevel%7Cline%7Canchor%7Clevel%7Cnumber%7Cfromtitle%7Cindex§ions=all&thumbwidth=640"; >>>>> -o /dev/null | grep -v GeoIP >>>>> >>>>> HTTP/1.1 200 OK >>>>> >>>>> Server: nginx/1.6.2 >>>>> >>>>> Date: Wed, 03 Jun 2015 13:22:23 GMT >>>>> >>>>> Content-Type: application/json; charset=utf-8 >>>>> >>>>> Transfer-Encoding: chunked >>>>> >>>>> Connection: keep-alive >>>>> >>>>> X-Powered-By: HHVM/3.6.1 >>>>> >>>>> X-Content-Type-Options: nosniff >>>>> >>>>> X-Frame-Options: SAMEORIGIN >>>>> >>>>> Vary: Accept-Encoding,X-Forwarded-Proto,Cookie >>>>> >>>>> X-Varnish: 2583358918, 1834266080 >>>>> >>>>> Via: 1.1 varnish, 1.1 varnish >>>>> >>>>> Age: 0 >>>>> >>>>> X-Cache: cp1047 miss (0), cp1060 frontend miss (0) >>>>> >>>>> X-Analytics: https=1 >>>>> >>>>> Set-Cookie: WMF-Last-Access=03-Jun-2015;Path=/;HttpOnly;Expires=Sun, >>>>> 05 Jul 2015 12:00:00 GMT >>>>> >>>>> Cache-Control: private, s-maxage=0, max-age=0, must-revalidate >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Mobile-l mailing list >>>>> [email protected] >>>>> https://lists.wikimedia.org/mailman/listinfo/mobile-l >>>>> >>>>> >>>> >>>> >>>> -- >>>> EN Wikipedia user page: >>>> https://en.wikipedia.org/wiki/User:Brian.gerstle >>>> IRC: bgerstle >>>> >>> >>> >>> >>> -- >>> EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle >>> IRC: bgerstle >>> >> > > > -- > EN Wikipedia user page: https://en.wikipedia.org/wiki/User:Brian.gerstle > IRC: bgerstle > > _______________________________________________ > Mobile-l mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/mobile-l > >
_______________________________________________ Mobile-l mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/mobile-l
