Excerpts from Steven Hardy's message of 2014-06-15 02:40:14 -0700: > Hi all, > > So, I stumbled accross an issue while fixing up some tests, which is that > AFAICS since Icehouse we continually revalidate every property every time > they are accessed: > > https://github.com/openstack/heat/blob/stable/havana/heat/engine/properties.py#L716 > > This means that, for example, we revalidate every property every time an > event is created: > > https://github.com/openstack/heat/blob/stable/havana/heat/engine/event.py#L44 > > And obviously also every time the property is accessed in the code > implementing whatever action we're handling, and potentially also before > the action (e.g the explicit validate before create/update). > > This repeated revalidation seems like it could get very expensive - for > example there are several resources (Instance/Server resources in > particular) which validate against glance via a custom constraint, so we're > probably doing at least 6 calls to glance validating the image every > create. My suspicion is this is one of the reasons for the performance > regression observed in bug #1324102. > > I've been experimenting with some code which implements local caching of > the validated properties, but according to the tests this introduces some > problems where the cached value doesn't always match what is expected, > still investigating why but I guess it's updates where we need to > re-resolve what is cached during the update. > > Does anyone (and in particular Zane and Thomas who I know have deep > experience in this area) have any ideas on what strategy we might employ to > reduce this revalidation overhead?
tl;dr: I think we should only validate structure in validate, and leave runtime validation to preview. I've been wondering about what we want to achieve with validation recently. It seems to me that the goal is to assist template authors in finding obvious issues in structure and content before they cause a runtime failure. But the error messages are so unhelpful we basically get this: http://cdn.memegenerator.net/instances/500x/50964597.jpg What holds us back from improving that is the complexity of doing runtime validation. To me, runtime is more of a 'preview' problem than a validate problem. A template that validates once should continue to validate on any version that supports the template format. But a preview will actually want to measure runtime things and use parameters, and thus is where runtime concerns belong. I wonder if we could move validation out of any runtime context, and remove any attempts to validate runtime things like image names/ids and such. That would allow us to remove any but pre-action validation calls. _______________________________________________ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev