Forgot to copy the list ---------- Forwarded message ---------- From: Luis Gervaso <l...@woorea.es> Date: Wed, May 9, 2012 at 7:14 PM Subject: Re: [Openstack] ceilometer (java implementation) To: Doug Hellmann <doug.hellm...@dreamhost.com>
I asked for these fields in my previous email, they are not clear enough. After analyze the problem, i ended that metrics can not be calculated per event. We need at least 2 events to calculate the duration (start/end) in some metrics. We can afford this on the counter processes, but it requires an update of the previous pesisted start event. So as we can calculate this info everywhere, and i don't like updates on the counters logic i'm calculating the duration as part of agreggation/mediation process (this will be only a view of the raw data). This way we maintain a very simple logic on the counters: * counters listen for specific events * the interpretation of raw data can be outside On Wed, May 9, 2012 at 7:01 PM, Doug Hellmann <doug.hellm...@dreamhost.com>wrote: > > > On Wed, May 9, 2012 at 12:46 PM, Luis Gervaso <l...@woorea.es> wrote: > >> That's fantastic! >> >> What I mind is that the counters only should gather event info. >> >> Maybe multiple counters listening, collecting info about the system. But >> only one persist the event. >> > > That makes sense. It seems like we will need to split the event-based > processing from the polling. The agent that runs on the compute node should > poll libvirt (and any other services on that node) and generate counter > messages using a "metering" exchange. A separate set of daemons running in > a more central location will watch messages on the metering exchange and > save them to the database. Those same processes can watch for notification > messages and convert them directly to metering data to be written to the > database. That saves the extra traffic from notification messages resulting > in several extra metering messages. > > >> >> Other process agreggates the data and creates different >> views/reports/billing, this should be outside of openstack (since it >> depends on the business rules, not depends on the infrastructure) >> > > If I understand Nick's proposal for the API, we will have some minimal > aggregation there but you are right that any other interpretation will > probably need to be handled by a custom app. > > >> >> The agreggator, also should offer an api for polling or can be configured >> throught drivers to push the the 3rd >> systems >> >> Now, in my implementation i'm putting all the stuff on a directory but it >> can be as well: >> >> - AMQP >> - SQL / NoSQL >> >> (the persistent layer shold be interchangeable) >> >> So as response to your duration question, this think is out scope for the >> counter tasks. > > > I think you misunderstood my question. The "duration" to which I referred > was the field in the counter schema ("counter_duration"). Only the "exists" > notification for instances includes enough information to calculate the > duration. The "create" notification should result in a "0" duration, so > that can be ignored. The "delete" notification should record the data since > the end of the last cycle, but does not today (at least it does not seem > to). > > >> >> >> >> >> On Wed, May 9, 2012 at 5:30 PM, Doug Hellmann < >> doug.hellm...@dreamhost.com> wrote: >> >>> >>> >>> On Tue, May 8, 2012 at 7:19 PM, Luis Gervaso <l...@woorea.es> wrote: >>> >>>> Hi, >>>> >>>> I have uploaded a toy version of ceilometer (java implementation). >>>> >>>> It does implement the first two counters (instance : rabbitmq listener >>>> and cpu : polling from libvirt) >>>> >>>> i need more clarification on the meaining: >>>> >>>> counter_volume >>>> counter_duration >>>> counter_datetaime >>>> >>>> I hope this helps to figure out how to agreggate these data. >>>> >>>> http://github.com/woorea/ceilometer-java >>> >>> >>> Nice! >>> >>> I have also been experimenting. I have some Python code in >>> https://github.com/dhellmann/metering-prototype that listens for >>> notifications related to instances (create, delete, exists) and converts >>> them to counter output (see spy.py). There is also a pair of scripts for >>> recording an event stream and playing it back (useful for testing, see >>> recorder.py and player.py). I don't have any libvirt polling, yet, though. >>> >>> In the course of looking at the notifications being generated for >>> different scenarios, I discovered that the instance delete messages do not >>> have any duration information right now. I don't know if that is a bug, or >>> if the idea is to figure out the durations from looking at the most recent >>> "exists" event. What do other people think? >>> >>> Doug >>> >> >> >> >> -- >> ------------------------------------------- >> Luis Alberto Gervaso Martin >> Woorea Solutions, S.L >> CEO & CTO >> mobile: (+34) 627983344 >> luis@ <luis.gerv...@gmail.com>woorea.es >> >> > -- ------------------------------------------- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO & CTO mobile: (+34) 627983344 luis@ <luis.gerv...@gmail.com>woorea.es -- ------------------------------------------- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO & CTO mobile: (+34) 627983344 luis@ <luis.gerv...@gmail.com>woorea.es
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp