> In current Alarm implementation, Ceilometer will send back Heat an > 'alarm' using the pre-signed URL (or other channel under development).
By the other channel, do you mean the trusts-based interaction? We discussed this at the mid-cycle in Paris last week, and it turns out there appear to be a few restrictions on trusts that limit the usability of this keystone feature, specifically: * no support for cross-domain delegation of privilege (important as the frontend stack user and the ceilometer service user are often in different domains) * no support for creating a trust based on username+domain as opposed to user UUID (the former may be predictable at the time of config file generation, whereas the latter is less likely to be so) * no support for cascading delegation (i.e. no creation of trusts from trusts) If these shortcomings are confirmed by the domain experts on the keystone team, we're not likely to invest further time in trusts until some of these issues are addressed on the keystone side. > The alarm carries a payload that looks like: > > { > alarm_id: ID > previous: ok > current: alarm > reason: transision to alarm due to n samples outside thredshold, > most recent: .... > reason_data: { > type: threshold > disposition: inside > count: x > most_recent: value > } > } > > While this data structure is useful for some simple use cases, it can be > enhanced to carry more useful data. Some usage scenarios are: > > - When a member of AutoScalingGroup is dead (e.g. accidently deleted), > Ceilometer can detect this from a event with count='instance', > event_type='compute.instance.delete.end'. If an alarm created out of > this event, the AutoScalingGroup may have a chance to recover the > member when appropriate. The requirement is for this Alarm to tell > Heat which instance is dead. Alarms in ceilometer may currently only be based on a statistics trend crossing a threshold, and not on the occurrence of an event such as compute.instance.delete.end. Near the end of the Icehouse cycle, there was an attempt to implement this style of notification-based alarming but the feature did not land. Another option would be for Heat itself to consume notifications and/or periodically check the integrity of the autoscaling group via nova-api, to ensure no members have been inadvertently deleted. This actually smells a little some of the requirements driving the notion of "convergence" in Heat: https://review.openstack.org/#/c/95907/6/specs/convergence.rst TL;DR: make reality the source the truth in Heat, as opposed to the approximation of reality expressed in the template > - When a VM connected to multiple subnets is experiencing bandwidth > problem, an alarm can be generated telling Heat which subnet is to be > checked. Would such a bandwidth issue be suitable for auto-remediation by the *auto*scaling logic? Or would it require manual intervention? > We believe there will be many other use cases expecting an alarm to > carry some 'useful' information beyond just a state transition. Below is > a proposal to solve this. Any comments are welcomed. > > 1. extend the alarm with an optional parameter, say, 'output', which is > a map or an equivalent representation. A user can specify some > key=value pairs using this parameter, where 'key' is a convenience > for user and value is used to specify a field from a Sample whose > value will be filled in here. > > e.g. --output instance=metadata.instance_id;timestamp=timestamp While such additional context may be useful, I'm not sure your examples would apply in general because: * there wouldn't be a *single* distinguished instance ID that caused the alarm statistic to go over-threshold (as the cpu_util or whatever metric is aggregated across the entire autoscaling group in the alarm evaluation) * there wouldn't be a discrete timestamp when the statistic crossed the alarm threshold due to perioidization and sampling effects > 2. extend the Ceilometer alarm-evaluator service, so that when an alarm > is seen requiring output values, it will try matching the 'value' > specified above to the fields in a sample, and replace the output > entry with 'key=<real_value>'. > > e.g. "output": { > "instance": "bd56bb53-d07f-49a6-8f60-6f8ef1336060", > "timestamp": "2014-07-0102: 21: 13.002155", > } > > The above data is passed back to the alarm_url as part of its > existing payload. > > If alarm-evaluator cannot find a matching field, it can fill in an > empty string, or just "None". > > 3. extend the OS::Ceilometer::Alarm resource type in Heat so that an > optional property (say, 'output') of type map can be used to specify > what are expected from the Alarm. And would the logic to consume such additional context be baked into the heat.engine.resources.autoscaling module? Or would that be plugable somehow? > Since it is an additional field in the 'details' argument, the impact to > existing Heat template/users will be negligible. However, the > expressive power of carrying back additional fields would be a great > help to some scenarios we yet to know. > > Because this is a cross-project proposal, comments from both communities > are valuable and thus appreciated. If it is a viable approach, should > we raise two specs in both projects repectively? I'm unconvinced as yet as to the viability, on the basis of my comments above. Though I'll keep an open mind with regard to your responses. Thanks, Eoghan _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev