On Fri, 2014-01-10 at 17:10 +0400, Nadya Privalova wrote: > Idea: > The goal is to improve performance when user gets statistics for > meter. Now we have fixed list of statistics (min, max and so on). > During request a user may specify the following params: > 1. query > 2. group_by > 3. period > > The idea of bp is to pre-calculate some kind of requests and store > them to a separate table in database.
Pre-calculate when? :) During processing of samples, or during some periodic job? > The pre-calculated statistics is called aggregates. The term "aggregate" really just means a generic grouping or summarization. If you are looking for a term that represents the rules/heuristics for maintaining rolling calculations, perhaps the term "report" is better? > Aggregates may be merged among each others and with any Statistics' > objects. > Note, that aggregates will be transparent for users. No changes in api > is required during get_statistics. > > Example: > Let's assume we have 6 Samples about 'image' meter. All of them belong > to one day (e.g. 1st May) but have happened in different times: > 11.50, 12.25, 12.50, 13.25, 13.50 and 14.25. User would like to get > statistics about this meter from start = 11.30 till end = 14.30. So we > need to process all samples. > But we may process these samples earlier and already have > pre-calculated results for full hour 12.00 and 13.00. In this case we > may get Sample 11.50 and 14.25 from "meters" table and merge > statistics for them with already calculated Statistic result from > "aggregates" table. > This example "saved" only 2 reads from DB. But if we consider metrics > from pollsters with interval = 5 sec (720 Samples per hour) we will > save 719 reads with aggregate usage. Hmm. So, aggregation and grouping are the domain of data warehousing and OLAP Servers. I don't believe that putting this functionality directly in to Ceilometer is a good idea. I believe it would be better to delegate this kind of functionality to well-known and used tools like Pentaho [1], which can use a variety of different backend storage systems. Bottom line, I believe Ceilometer should focus strictly on the collection of samples/alarms, the pre-processing of those things, and the storage of those things, however I do not believe that Ceilometer should become an OLAP analytics tool when existing ones already fill that need. Best, -jay [1] http://www.pentaho.com/5.0 & http://en.wikipedia.org/wiki/Mondrian_OLAP_server _______________________________________________ OpenStack-dev mailing list [email protected] http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
