My personal use cases which are primarily using the visualization tools would appreciate more dimensionality in daily and weekly views (which increases storage). I think you should definitely degrade the resolution, possibly more aggressively than you propose. RRDTool has been doing this for decades so people are used to it.
-Toby On Fri, Jul 29, 2016 at 5:40 AM, Dan Andreescu <[email protected]> wrote: > Dear Pageview API consumers, > > We would like to plan storage capacity for our pageview API cluster. > Right now, with a reliable RAID setup, we can keep *18 months* of data. > If you'd like to query further back than that, you can download dump files > (which we'll make easier to use with python utilities). > > What do you think? Will you need more than 18 months of data? If so, we > need to add more nodes when we get to that point, and that costs money, so > we want to check if there is a real need for it. > > Another option is to start degrading the resolution for older data (only > keep weekly or monthly for data older than 1 year for example). If you > need more than 18 months, we'd love to hear your use case and something in > the form of: > > need daily resolution for 1 year > need weekly resolution for 2 years > need monthly resolution for 3 years > > Thank you! > > Dan > > _______________________________________________ > Analytics mailing list > [email protected] > https://lists.wikimedia.org/mailman/listinfo/analytics > >
_______________________________________________ Analytics mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/analytics
