On 05/06/2012 10:36 PM, Tomasz Paszkowski wrote: > Hi, > > I'd like to share my thoughts on metering openstack resources usage. > Except those data available from SystemUsageData and those mentioned > in blueprints some of the cloud providers charge for I/O on disk > drives (to prevent users from dd if=/dev/zero nosense and to teach > them properly implementing cache strategies). Hi Tomasz,
I could not agree more and this is the reason why I/O shows in the list of meters shown in http://wiki.openstack.org/EfficientMetering (c5) "disk IO in megabyte per second has a high impact on the service availability and could be billed separately ". > Those data along with > network card usage can be gathered from libvirt using domblkstat and > domifstat. Thanks for the hint http://wiki.openstack.org/EfficientMetering?action=diff&rev2=60&rev1=59 > My idea is to gather data using agent (or modified > nova-compute) and send them to messaging queue using following jsoned > data schema: > > {'instance': 'instance-0000003b', > 'host':'tytan-1','zone':'r4cz1','counters': {'interface': {'vnet0': > (80796L, 1212L, 0L, 0L, 53403L, 621L, 0L, 0L)}, 'disk': {'vda': (629L, > 11699200L, 58L, 219136L, 0L)}}} > > interface is a result of: interfaceStats(), disk is a result of: blockStats() > > > Those messages are consumed from queue and stored in mysql tables. I > assume that instance is a parent resource for each disk (ephemeral or > volume) and for each network interface. > > So for message mentioned earlier we have three resources: > > 1) instances resource: instance-0000003b > 2) child resource: vnet0 (network) > 3) child resource: vda 'ephemeral > > > Mysql table for resources is like below: > > CREATE TABLE resources ( > id BIGINT UNSIGNED NOT NULL AUTO_INCREMENT, > parent BIGINT UNSIGNED, > type VARCHAR(255) NOT NULL, > value VARCHAR(255) NOT NULL, > zone VARCHAR(255) NOT NULL, > added TIMESTAMP, > ) ENGINE=INNODB; > > Counters are stored also in Mysql using following table: > > > CREATE TABLE counters ( > resource BIGINT UNSIGNED NOT NULL, > type VARCHAR(255) NOT NULL, > value BIGINT UNSIGNED, > delta BIGINT UNISGNED, > added TIMESTAMP NOT NULL, > prev TIMESTAMP NOT NULL, > ) ENGINE=INNODB; > > Where prev is a reference to previous counter value. Process which is > reading data from queue is puting raw counter value into the table and > if possible (reference to previous entry present) evaluates delta > value. > > > By using this model of stroing usage counter's it very easy for > billing system to evaluate charges. We just run SUM(delta) on each > counter for given time range. > > This model could be very easy adopted to other counters (IP Traffic > external/internal counters from iptables). > It looks like you already have a codebase that could be useful for the metering implementation. Would you be willing to share it ? Cheers > > > > On Mon, Apr 30, 2012 at 12:15 PM, Loic Dachary <[email protected]> wrote: >> Hi, >> >> To prepare for the next meeting ( thursday 3rd, may 2012 >> http://wiki.openstack.org/Meetings/MeteringAgenda ) I cleaned up and >> reorganized the Metering blueprint so that it ( hopefully ) incorporates all >> the information temporarily stored in the etherpad ( >> http://etherpad.openstack.org/EfficientMetering revision 67 in case it is >> vandalized ). >> >> We could start a discussion from the content of the following sections: >> >> http://wiki.openstack.org/EfficientMetering#Counters >> http://wiki.openstack.org/EfficientMetering#Storage >> >> and come up with a list of the counters that should exist by default and how >> they should be stored. >> >> This morning we had a discussion with Zhongyue Luo on >> irc.freenode.net#openstack-metering about how Dough could use the metering >> service. Since it already knows about instance creations, counter c1 that >> records how long a given instance was up is of no interest. However, other >> counters such as the external bandwidth used would be useful. I advocated >> that one of the advantages for Dough to rely on metering to collect counters >> is that it does not need to know about each OpenStack component and can rely >> on metering to figure out how to extract such counters from nova-compute, >> nova-network soon to be quantum, nova-volume soon to be cinder, swift, >> glance and free it from the burden of tracking structural changes. >> >> Cheers >> >> -- >> Loïc Dachary Chief Research Officer >> // eNovance labs http://labs.enovance.com >> // ✉ [email protected] ☎ +33 1 49 70 99 82 >> >> >> _______________________________________________ >> Mailing list: https://launchpad.net/~openstack >> Post to : [email protected] >> Unsubscribe : https://launchpad.net/~openstack >> More help : https://help.launchpad.net/ListHelp > > -- Loïc Dachary Chief Research Officer // eNovance labs http://labs.enovance.com // ✉ [email protected] ☎ +33 1 49 70 99 82 _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
