Hi all, Actually we have a mechanism for cleaning up the data, but I didn't test it. There is time-to-live parameter. It's implemented in MySQL and MongoDB backends. I'm not sure about MongoDB perf testing but I know that Sandy Walsh and his Rackspace team have plans about performance testing all Ceilometer's DB backends.
Thanks, Nadya On Thu, Dec 12, 2013 at 3:33 PM, Roman Sokolkov <[email protected]>wrote: > Sorry, > > link<https://www.zabbix.com/documentation/2.0/manual/config/items/history_and_trends> > - > how zabbix stores old data. > > > On Thu, Dec 12, 2013 at 3:24 PM, Roman Sokolkov <[email protected]>wrote: > >> Bogdan, >> >> Thanks for your comment. Please see my thoughts: >> >> - this driver not merged yet. >> - this a prototype of driver for young technology. >> - Ceilometer developers prefer MongoDB. >> >> So i don't think we should research something regarding ElasticSearch >> right now. I suggest to focus on MongoDB. >> >> Thanks, Roman S. >> >> >> On Thu, Dec 12, 2013 at 12:38 PM, Bogdan Dobrelya <[email protected] >> > wrote: >> >>> On 12/12/2013 08:51 AM, Roman Alekseenkov wrote: >>> >>> Nadya, >>> >>> Thanks a lot for sharing the results. >>> >>> Roman S., >>> >>> What is your perspective on the things described below? I'm worried >>> about Ceilometer + MySQL support in Fuel 4.0. As you can see, the DB fell >>> apart in a matter of one day. We can't really afford similar behavior in >>> customer's environments. So, I'm interested in the following: >>> >>> 1. Given that Ceilometer generates so much data, will MongoDB really >>> help to handle it better or it's a matter of time until it breaks like >>> MySQL? >>> 2. Is there a cleanup mechanism in Ceilometer to clear historical >>> data? We don't want the database to grow indefinitely. >>> 3. Are we pointing Ceilometer to the same Galera instance as we use >>> for OpenStack, or we deploy a different MySQL for it in Fuel? >>> >>> I'd like to add here: 4) Would it make the sense to use ElasticSearch >>> backend driver prototype for ceilometer >>> https://blueprints.launchpad.net/ceilometer/+spec/elasticsearch-driver? >>> Here is some more considerations about ES as NoSQL usage: >>> https://www.found.no/foundation/elasticsearch-as-nosql/ >>> I believe, this is a good point for deeper research. >>> >>> >>> 1. What are our recommended/default settings in Fuel for Ceilometer? >>> It is stats collection every 5 seconds, or something different? How safe >>> are our defaults? >>> 2. Do we allow user to tweak essential Ceilometer settings through >>> Fuel UI or manually? >>> >>> Thanks, >>> Roman >>> >>> On Tuesday, December 10, 2013, Nadya Privalova wrote: >>> >>>> Hi colleagues, >>>> >>>> The purpose of this letter is to describe my Ceilometer tests and just >>>> experience in it's deployment. >>>> >>>> First of all let me describe what I wanted to achieve. There were a lot >>>> of discussions on summit about Ceilometer's collector performance: >>>> community was not sure that it's worth to start improving performance of >>>> fetching data from Ceilometer's DB _before_ fixing some issues during data >>>> collection. I decided to check how Ceilometer collects data and see that >>>> 'issues' in real life. For this purpose I used the following env (copied >>>> from letter to community): >>>> >>>> Lab description: >>>> 3 controllers >>>> 187 computes >>>> HA: Galera for MySQL >>>> memcached is on, RabbitMQ in HA mode >>>> >>>> Ceilometer processes are running as follows: >>>> 1 controller: ceilometer-api, ceilometer-agent-central, >>>> ceilometer-collector, ceilometer-agent-compute >>>> 2 and 3 controller: ceilometer-collector >>>> all computes: ceilometer-agent-compute >>>> >>>> My first step was just create 200 instances and monitor it every 5 >>>> sec. During 2 days was collected about 9 000 000 entries in DB. Lab worked >>>> rather ok. But 200 instances for such the lab is very little load. >>>> >>>> The second step was to try Rally. Two goals were achieved here: >>>> measure the Ceilometer's influence on Nova (please read the letter till the >>>> end for details about this) and load the lab and check how many entries >>>> will be stored in DB. >>>> Nova test in several words: scenario boot+delete instance becomes >>>> about 2 times slower if Ceilometer is up and monitors 200 instances. But >>>> Julien provided good comments about deployment strategies and I think it's >>>> ok to duplicate it here: >>>> "If you store them [metrics] in the same MySQL DB that's used by Nova >>>> for example, it's likely that's the problem is the load Ceilometer puts on >>>> the MySQL cluster slows Nova >>>> down." >>>> And my summary: >>>> "So if we want to use Ceilometer + MySQL in production we need to use >>>> separate controllers with Ceilometer's MySQL only. And each controller may >>>> run it's own collector which will write data into "local" MySQL. And only >>>> one instance of central-agent may be started (WIP >>>> https://wiki.openstack.org/wiki/Ceilometer/blueprints/tasks-distribution >>>> )" >>>> >>>> How the lab was feeling after Rally test? Bad. During the test was >>>> created and deleted about 2000 instances and besides this 200 instances >>>> were up permanently. So after two days: >>>> mysql> select count(*) from meter; >>>> +--------------+ >>>> | count(*) | >>>> +--------------+ >>>> | 18177507 | >>>> +--------------+ >>>> So one of MySQL servers failed and sync between servers failed too. I >>>> managed with it only by creating a new MySQL cluster (old cluster >>>> reconfiguration). >>>> >>>> I think that the load simulated by Rally is not very high actually. >>>> E.g. if we consider Savanna EDP use case than the same load will be >>>> achieved by the following scenario: "20 users runs 33 hadoop-jobs during >>>> two days" (cluster with 3 nodes will be created and deleted). >>>> >>>> So to summarize: if we want to use MySQL+Ceilometer in production we >>>> should install it on separate controllers and use separate MySQL for it. >>>> And one more note: I'm talking only about data collection now, data >>>> fetching (smth like "show ceilometer statistics for 2 days") works very >>>> slow on MySQL and a little bit better on MongoDB. >>>> >>>> Please if you have any questions, suggestions about further tests - >>>> you are welcome! Now we decided just to run 1000 instances and check >>>> Ceilometer behavior. >>>> >>>> Thanks for attention, >>>> Nadya >>>> >>>> >>> >>> >>> >>> -- >>> Best regards, >>> Bogdan Dobrelya, >>> Researcher TechLead, Mirantis, Inc. >>> +38 (066) 051 07 53 >>> Skype bogdando_at_yahoo.com >>> 38, Lenina ave. >>> Kharkov, [email protected] >>> >>> >> >> >> -- >> Best regards, Roman Sokolkov >> > > > > -- > Best regards, Roman Sokolkov >
-- Mailing list: https://launchpad.net/~fuel-dev Post to : [email protected] Unsubscribe : https://launchpad.net/~fuel-dev More help : https://help.launchpad.net/ListHelp
