> > I worry a little bit about the performance without having a batch api, but > we can certainly try it out and see what happens. Basically we will be > requesting the page view information for every NS_MAIN article in every > wiki once a week. A quick sum against our search cluster suggests this is > ~96 million api requests. >
Oh, sorry, I thought you meant you were just querying 100 or so titles! In the case of huge queries like these, you should just query the wmf.pageview_hourly table directly. You can do so with plain SQL via Hive or maybe Impala if we end up setting that up. But those queries should be really fast in that table. We can help you write the query if you send us an attempt and a spec of exactly what you need.
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
