On Mon, May 14 2012, Loic Dachary wrote: >> Each set of metering data will need to be associated with the appropriate >> metadata from the resource at the time the metering information was >> collected. The rate of change of metadata and metering events are >> different, though, so the timestamps of the metadata records are unlikely >> to match exactly with the values in the metering records. Depending on the >> clock resolution, it would be possible to have metadata changes and meter >> data with the same timestamp, resulting in an incorrect association. > Indeed, good point. >> >> We can work around that by maintaining proper foreign key references using >> the metadata version field as you describe in the schema above (so the >> resource id and metadata version value point to the correct metadata >> record). It will make recording the metering data less efficient because >> we will need to determine the current version for the resource metadata, >> but we can optimize that eventually through indexes and caching. >> >> Aggregation will also need to take the metadata version into account, so >> everywhere in the list of queries we say "by resource_id" we need to change >> that to "by resource_id and version". > I added the idea of a format version for when the payload format changes and > tried to write down a description of the metadata storage matching this > thread in the wiki. > > http://wiki.openstack.org/EfficientMetering?action=diff&rev2=80&rev1=78 > > What do you think ?
I'm jumping in a bit late in the discussion, but there may be a point I miss in the current definition because, I think it's getting too complicated. We now have 2 "payload" fields: one for meter and one for metadata. For example, if you look at the c1 counter (instance) you need to store the "type" as payload of the meter. This is a metadata of the instance, but it's not currently defined as being stored in metadata, but in the "payload" field of the meter. Moreover, I'm rather sure there will soon be a counter with the need of 2 different "payload" information, and we'll have a problem since we can only store one in the current meter schema, so we'll store the second one as a metadata or something. So clearly the initial "payload" solution is not enough. OTOH I find the metadata proposal in another table too much complicated. Why not storing what metadata in the meter.payload field in the same table (e.g. as a JSON string)? I miss the point of the introduction of a dedicated metadata table with version string. It sounds to me like early optimization, which is the root of all evil. :) But I might miss something. -- Julien Danjou // eNovance http://enovance.com // ✉ julien.dan...@enovance.com ☎ +33 1 49 70 99 81 _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp